Taxman Cometh: A Brief History of Taxes

Hello and welcome back to Breaking the Dollar. I'm your host, Everett Millman. This week's episode, we're going to focus on something that everybody loves to hate and that is taxes. One of my co-workers, Luis Cordero, had an interesting discussion last week about, you know, where do some of these taxes come from? How long have they been around? And it sort of harkens back to my episode on funny money on different forms of currency throughout history because pretty much for as long as human civilization has existed, some form of taxes have been with us.

And what's interesting, again relating to funny money, is that back in ancient times, it was often just some portion of your property that you gave up to the government. Now obviously that still goes on. We just use currencies to be an intermediary for that exchange. But in ancient Greece and ancient Egypt, for example, you might have your agriculture, a portion of your farming output might be taken away. Another fascinating example comes from France. In the 1700s, late 1700s, around the time of the French Revolution, you could actually pay your taxes to the government with your labor. So you would go and work on some public works project. Maybe you were building a road or building homes and you did that for x amount of days. I think maybe it was three days or a few weeks, something like that, and that was your payment of tax for the whole year.

Later on in the episode when I briefly am going to touch on property taxes, that's an idea that goes all the way back to feudalism in medieval Europe where technically your property was being leased from the lord. You might think of it as the landlord if you want to connect it to a concept that still exists today. And so a recurring theme will be how our forms of taxes and the things that we believe or the law believes are appropriate to tax really have a grounding in history that goes back literally thousands of years. Now, that doesn't mean that we have to like it or enjoy paying taxes, but it isn't something that has completely sprung out of nowhere.

So I'm going to start by talking about the income tax because that is the big controversial one. And I don't mean to be a hypocrite or contradict myself because I did bring up how some form of tax has been around basically forever. But what's interesting is that a lot of the forms of tax that exist today in the United States have really only been codified in the law. They've only been specifically laid down in the Constitution or in legislation for about a hundred years, or in some cases, even less. And this is certainly true of the income tax. Now the big thing that everyone focuses on is the 16th Amendment to the US Constitution. That amendment reads sort of like it is giving the federal government the power to tax income. But it's a little more nuanced than that. What it does is it clarifies that the government can tax your income without apportioning it. And that means calculating it as a percentage based off the population of your state.

So the 16th Amendment, which was passed in 1913, which is not coincidentally the same year that the Federal Reserve was established, it basically just expanded the taxation powers of the federal government so that they weren't held back by the fact that different types of taxation are treated differently under the law. Income was one of them. Now that was by no means the first time an income tax had ever been levied in the United States. Throughout the second half of the 1800s, you actually had several instances where the federal government or a state government stepped in to try and tax income. The difference is that this was always done on a temporary basis for five years or so. And it was always done with a specific type of income in mind.

So something that you find in the history of taxes is that a lot of times it was based on a specific type of item. In this way, you can think of tariffs, which is basically an import duty. It's a tax on products that you're importing into your state or country. This is the original idea behind most taxes. You tax a very specific thing and in addition, the revenues from that tax might go to a very specific thing. And essentially what the 16th Amendment said was that you didn't have to worry about apportionment when you gathered an income tax. And that it didn't matter what the source of that income was. So really, that's the big powerful distinction that makes the 16th Amendment important enough to get an amendment in the Constitution.

Now the whole idea of taxes is very philosophical. It causes a lot of debate, a lot of controversy. And as I said, the income tax is sort of the main battleground for this. There are lots and lots of people who not only think that the income tax is unconstitutional, and they find all sorts of little loopholes or caveats with the 16th Amendment, but they often rally under the banner of taxation is theft. You hear this a lot in the precious metals industry as well. Anyone who doesn't trust our currency, doesn't trust the government, they're going to be predisposed not to like any tax on their income. There really is an interesting and rich history of people making these arguments in court about why you cannot charge an income tax. Yes, there are people out there who have gone and done this. Most of them are now in jail.

So whatever the merits of the idea that taxation is theft, it's pretty much a losing battle to try and argue against the government in this respect. And again, just to draw in the broader history of this before there were established currencies issued by governments, it didn't mean that you paid no tax at all on your income. Eventually, you would in the form of giving up some of your possessions and giving it to the government. That was the case in every empire and kingdom you can think of throughout history. People are being taxed. And part of the idea behind it was that the king was entitled to everything in his kingdom. So it was only by the goodwill and grace of the king that you got to keep anything back then. So really, the philosophical basis was more obvious to people back then who believed in monarchy, but also from a practical sense, what were you going to do? The king had absolute power. You really had no say in the matter. And so it does make sense that the idea of not being taxed arbitrarily goes hand-in-hand with democracy and not being under the rule of a king.

Now something that's interesting in this whole topic of taxes is, as I said before, there are many different kinds of taxes. The government is basically going to find any way possible to raise revenue from economic and commercial activity. So it's not just income that can be taxed. You also have property taxes. You have an estate tax, which is similar. Essentially, it's like a wealth tax. If you inherited items or you already had valuable things that really aren't income, it's not like you earned it in a given year. Those things can still be taxed. The government is going to get its pound of flesh. Similarly, you also have capital gains taxes, which is basically a tax on investment profit. And you can go all down the list, probably the most familiar tax that everybody is aware of is sales tax. And that's because you see it every time you buy something on the bottom of the receipt, it will show you how much money you're paying in tax on top of the price of that item.

What I think is important for everyone to keep in mind is that the history of general sales taxes is rather similar to that of the income tax, in that it's a somewhat recent development in the 1800s and 1900s, but that it was something that was going on really all throughout history by different names, through different forms. You had several states try it on their own throughout the 1800s, but again, it would be for a temporary time period, or it would be only on certain types of sales. It wasn't until the Great Depression that sales tax became such a widespread thing, because the government really, really needed the money at the time. They needed a way to raise revenue. Even today, there are only five states out of 50 in the United States that do not have any statewide sales tax.

Now for the other 45 states, sales tax has been its their largest form of tax revenue since 1947, so it didn't take long for this to catch on and become a real boon for the state treasury. Like the income tax, though, there are people who are opposed to sales taxes, as you can imagine. In the vast majority of countries around the world outside of the United States, they have almost eliminated sales taxes as we understand them and replace them instead with what's called a value added tax, or VAT for short. Sometimes you'll hear this referred to as GST, a goods and services tax. Really the main difference between a VAT and a sales tax is that the sales tax is only on the finished good or product. It's only at the final point of sale. So for some people, they consider it a consumption tax. You are taxing the person who is consuming some item. By contrast, the VAT accomplishes basically the same thing, but it taxes everyone down the line in the supply chain. This is similar to what's called an excise tax, which is an indirect tax at the time something is manufactured. So that's different from a sales tax because like I said, only the person at the final -- the end user, it's called -- the person at the very end of the line who's buying some finished product, they're the only one who pays sales tax.

In these other schemes, basically everyone is paying that tax at every point. So they simply pass it on to the next person in the chain. The formats of these tax schemes are different, but the final result is essentially the same of raising revenue for the government based on how much commerce is happening in the economy. Now there are a couple of things that are complicating factors for sales taxes. One is that lots of things are exempted and this sort of becomes a political battle to get your preferred class of things exempted from sales tax. One benefit in the precious metals industry that I can tell you about from experience is that in many, many states, including the state of Florida, if you purchase over a certain dollar amount of gold or silver, you don't have to pay any sales tax on it. Another caveat is that if you buy gold and silver in the form of legal tender, meaning an actual coin issued by a government, you also do not pay any additional sales tax.

Now that's just one example from my industry, but the examples start to multiply and multiply for all the different exemptions. In a lot of cases you can get tax credits so that if you don't make very much money, you aren't being taxed, you have to get a big refund and no matter what tax bracket you're in, if you do pay more than you owe in taxes, you get a refund at the end of the year, which I'm sure almost all of us celebrate and are familiar with, but it's complicated. So one thing that is very difficult to put a sales tax on is a service and that's one of those problems that the VAT or GST tries to get around. But one of the things that this discussion is starting to reveal is that the tax code is very complex and it is only growing more complex as time goes on. More things are being taxed and more things are being taxed in more sophisticated ways.

So in the instance of property taxes, it used to simply be that it was done by the acre. It was how much land you had, you paid a certain tax on that. It was a flat rate, but people began to realize that some forms of land are more valuable than others. So now the common practice is to tax property based on its value, not simply on how large the amount of land is. But even beyond the greater complexity in property taxes and other forms of tax, you can see very simply just in the 1040 form that you have to fill out when you're filing your federal income tax with the IRS. When the 16th Amendment passed in 1913, so we're saying roughly a hundred years ago, all of the instructions for how to fill out your 1040 form took up one page. Today, the instructions are nearly 200 pages long if you printed them all out. Think about that. That is a massive increase in complexity.

And it's not just getting more complicated for consumers and the individual paying their income tax. There's also a increasing number of hoops that businesses have to jump through with respect to their tax burden. So another example that I can give you from the precious metals industry is a court case, a Supreme Court case called the Wayfair case that was decided about a year ago in 2018. And essentially, it overturned the precedent of taxing interstate sales based on the state of the seller. That was the original decision that happened at a previous case in the 1990s. But of course, they could not have foreseen the incredible expansion of internet sales that allow people to sell things across state lines without physically being in that state.

So in short, what the Wayfair decision did is it reversed that and said you did have to collect taxes based on the laws in the place where the person bought the item. That's had an effect on all online sellers, but especially so in online precious metals sales. Because as I said, there's all this minutiae about tax exemptions that you have to know. But then you also have all the complications of local taxes, municipal taxes. And now essentially every company that's in this industry, every company that sells gold and silver coins online has to become a tax expert for all 50 states and every locality in every city. It really is a much greater burden on doing business. And because of it, unfortunately, there are now some states where it is unprofitable for a gold and silver seller to operate in. I mean, this Supreme Court decision is having a really seismic effect on the whole industry. And so I think that's a fair example of how burdensome tax law can be on people who are just trying to engage in regular business transactions.

Now one of the kind of obvious drawbacks of that is as you have more and more taxes and more ways of taxing people, it does encourage them to try and evade those taxes. So not only do we see that obviously in the United States and it's a big political battle about how we want corporations and the wealthy to not evade any taxes. But even in countries where you wouldn't normally associate it with being a tax haven or tax evasion, they have a lot of problems with tax compliance. And this isn't just businesses. So one very prominent example is in Greece. In Greece, they went through a particularly bad banking crisis and financial crisis over the past five or six years. And so they had to institute very high taxes in order to keep the government afloat, to collect enough revenue to keep the government operating. And those taxes had to get higher and higher because less and less people were complying with the law. It was sort of just a dirty little secret that most people avoided paying their taxes. They simply didn't do it. And that the bureaucrats in the government who had to go and collect those taxes often did not do it.

So there are definitely some inefficiencies involved with levying taxes to begin with. It absolutely changes the math for businesses and individuals about how much money do they need to make or how much money are they actually taking home. And you will hear that point advanced by people who argue against aggressive taxes because it is true that without quite as many of these types of taxes, governments around the world functioned pretty well. You have hundreds of years of history where, as I said, there were forms of taxation, but not nearly as widespread as they are now. So I do sympathize with that side of the, you know, taxation is theft argument. I'll also point out that you would expect as there are more types of taxes and more things are being taxed, and as the world has become infinitely more wealthy and economically productive over the past 50 years, that that would mean that tax rates could go down. If you're taxing more people, you're taxing more things, and the people have more wealth, you should be able to generate more money by taxing them at a lower percentage. That's just common sense.

But in fact, the opposite has happened, at least in the United States. So back in 1970, the average state sales tax was about 3.5%. Today about 50 years later, it's closer to 6%. It's nearly doubled. That doesn't really add up, at least in my opinion. To reiterate, it's not just that there are more taxes and there are more people to tax now. It's also undeniable that the average individual and the average corporation have much more wealth than they did 50 or certainly 100 years ago. So think about that. If the average tax rate stayed the same, it's 3.5% of a bigger pool of money. So tax revenues would increase for the government anyway. So it just seems very odd that tax rates have been rising, even though there's more money out there for them to potentially tax. I think those are the part of this debate that are interesting to look at. I really don't get behind the more radical side that no taxes are acceptable and that all taxes should be abolished because as good as that sounds, there obviously haven't been many viable replacements proposed. And I just don't think you're going to win that battle with your government no matter what government in the world that is.

Anyway we will now dip into our mailbag and take a question from the listeners. This week's question comes from Jaspreet in Culver City. And Jaspreet asks, what is the Fed going to do this week? Ah, that's good. That means that this person was paying attention to the fact that the Federal Reserve has its monthly meeting coming up this week where it decides what to do about interest rates. My gut feeling is that the Fed is going to cut interest rates again by 0.25%, which in insider lingo is called 25 basis points. Basis point is just a hundredth of a percent. But anyway, I do think they're going to cut rates again. Although what we have seen is that the Fed has not been unanimous in cutting rates recently. There's been a little bit of dissent from some of the more hawkish members of the Fed's Open Market Committee.

And so what I think everyone should really be watching for is what kind of language does the Federal Reserve use when it talks about the rate cut that I expect to see. If the Fed is a little bit cautious or includes any language about being close to neutral or symmetrically balanced, that would be a sign that this might be the last rate cut for a while. Or at least that's what the Fed is hoping. The way I see it, the Fed really doesn't want to have to keep cutting rates. It doesn't want rates to go all the way to zero because then it doesn't have any room to cut rates in the future if we have a recession. So that's sort of the direction I expect things to go is that we will get a rate cut, but the Fed will try and signal to everyone and imply that this will be the last one for a while. Now whether or not they're able to actually do that, that's another question.

That brings us to the end of today's show. Appreciate everyone out there listening. Thank you so much for tuning in.