How Much Does ChatGPT Know—and Not Know—About Coins?

ChatGPT is a new artificial-intelligence tool that seems to have caught everyone’s attention. It is a chatbot powered by a large language model (LLM). These kinds of AI models are nothing like Skynet from the Terminator film series, but you wouldn’t know that by some of the hyperbolic reactions to this emerging technology.

Everybody seems to think AI is on the cusp of eliminating millions of jobs—mine included. In order to cut through all of the hype, I decided to put ChatGPT to the test to see how much it understands about my area of expertise: coins and precious metals.

I asked ChatGPT several popular questions about numismatics, precious metals, and the history of coins in general. The questions spanned from merely testing its logical coherence to more philosophical musings about the monetary system.

Although ChatGPT gave some impressive correct information, it also provided several glaringly incorrect answers! It was a fun experiment, but I concluded that artificial intelligence has a long way to go before it can convincingly approach human reasoning.

First, here's a quick primer about what ChatGPT does.

Image via Shutterstock

How ChatGPT Works

ChatGPT was developed by a company called OpenAI. As the name suggests, OpenAI started out as an open-source project in 2015. It had several prominent seed investors, such as Elon Musk and Peter Thiel. As of now, OpenAI is no longer as open or transparent about how the software works.

ChatGPT is a large language model (LLM). Large language models are trained through machine learning by being fed massive amounts of text. They use a neural network to extrapolate rules about human speech, grammar, and writing. An LLM can then recognize patterns, understand relationships between words, memorize facts, and perform text-based tasks.

This could be as simple as solving a math problem or providing an encyclopedia-style answer to a history question. ChatGPT can even write poetry. It doesn't have access to actively crawl the internet in real-time, but that may change soon. As the technology rapidly improves, it seems that LLMs are capable of even more sophisticated feats.

Even though it’s only existed for a few months, this technology is indeed being used across a number of industries already. The chatbot has been shown to write functional code in a variety of programming languages, provided it is given the right prompt or input. It threatens to shake up the online search business, which has largely been monopolized by Google up to now. It's worth noting that since the emergence of ChatGPT, Google has developed a similar chatbot called Bard, and Microsoft has its own chatbot through Bing.

ChatGPT has even been featured in a recent episode of the comedy show South Park, and many voices in the media fear that it could change writing and literature forever. It perhaps has the potential to transform other sectors of the economy through automation, as well.

Let's see for ourselves how the AI handled questions about coins and precious metals.

What AI Got Wrong: Critiquing ChatGPT's Answers

AI chatbots, ChatGPT included, still have certain quirks and bugs that have been well-documented. They have been known to "hallucinate" and spit out some clearly incorrect logic.

When I initially posed the questions above to a previous version of ChatGPT, known as ChatGPT-3.5, I got a few of those kinds of responses.

Question 1: What Makes the 1944 Penny Rare?

First, it gave some misleading information about the 1944 Lincoln penny last time I asked:

ChatGPT's earlier answer about the 1944 penny.

Most of this information is correct. Unfortunately, the chatbot contradicts itself in point #2. It doesn't make sense for the government to melt down copper pennies if indeed the copper supply had been replenished. Moreover, the blanket claim that the U.S. Mint made less one-cent coins in 1944 is also incorrect.

Question 2: Has the Value of the U.S. Dollar Risen Over Time?

Next, when I had previously asked about the value of fiat currencies like the dollar, ChatGPT-3.5 completely mischaracterized the general trend.

ChatGPT's earlier answer about the value of the U.S. dollar.

Sure, much of what it says about currencies is true. Sadly, the AI would have us believe that the U.S. dollar has generally appreciated in value over time—the exact opposite of what has happened. It is patently incorrect.

Question 3: Why Does a Buffalo Nickel Have No Date?

Most egregiously, ChatGPT resorted to an outright fabrication when I asked about Buffalo nickels with a missing date. No matter what way you look at it, its answer here (except for the very first sentence) is entirely made-up!

ChatGPT "lies" about Buffalo nickels with no date.

Even more bizarrely, when I asked what the source for this erroneous information was, the chatbot cheerfully supplied me with a real reference book. (The information wasn't there, of course.) When I pointed this out, ChatGPT compounded its error by generating a second lie—albeit a more plausible one than the first. Nonetheless, it was a blatant error all the same.

I wrote an entire article detailing ChatGPT's failures with the topic of dateless Buffalo nickels that you can read. I was amused and fascinated by its mistakes.

What AI Got Right: ChatGPT's Correct Answers

Aside from the issues with the answers above, everything else I asked ChatGPT passed muster.

Question 1: What Were the First Coins in Human History?

I began with a straightforward softball question:

Question #1 to ChatGPT

This response was spot-on. It aligns well with a heavily-researched article I wrote on this exact topic for the World History Encyclopedia in 2015. ChatGPT gets full credit for getting this right.

Question 2: What Is the Most Valuable Canadian Coin?

Next, I posed a question about a more contemporary topic:

Question #2 to ChatGPT

Maybe we could nitpick here and point out that there is a novelty 100-kilogram Gold Maple Leaf coin worth more. But, in terms of the most expensive Canadian coin you could actually own, this is the right answer.

You can read our article about the most valuable Canadian coins by following the link.

Question 3: What Nickels Are Silver?

Another topic that I have written an authoritative article on is silver war nickels. Given that it was originally posted in 2019, I wasn’t terribly surprised that the chatbot was well-versed in the topic.

Question #3 to ChatGPT

All of the specific context provided in the answer is accurate. It even went above-and-beyond by pointing out the exact metallic compositions and the presence of the distinguishing mintmarks. I doubt I could have written a more comprehensive summary myself!

You can read our article explaining what nickels are silver in greater detail by following the link.

Question 4: Why Were Coins Historically Made of Gold and Silver?

Thus far, ChatGPT has nailed the simple factual questions. But what if we ask it something a bit more open-ended and philosophical?

The first thing you will notice in all of the subsequent responses is that ChatGPT tends to become a lot more verbose. This does result in some rather complete answers, but it is certainly a big difference between artificial intelligence and normal human conversation. (It’s also a dead giveaway to schoolteachers that a student may have used ChatGPT to write an essay!)

Question #4 to ChatGPT

Question #4 to ChatGPT (continued)

There is nothing I can dispute in the response above. Interestingly, the closing sentence even anticipates the potential follow-up question about why we no longer use gold and silver for circulating coins.

You can learn why precious metals were so well-suited for their role as coinage by listening to our podcast on the topic.

Question 5: Are Coins Today Worth Less Than They Used to Be?

I came up with a logical follow-up question on this same topic. One of the interesting things about ChatGPT is that it “remembers” what has been said earlier in the conversation. It appears to perform well at building upon previous prompts and inputs.

Question #5 to ChatGPT

Question #5 to ChatGPT (continued)

This answer passes with flying colors. It may as well have come directly from a textbook on monetary economics.

Question 6: Has the Value of the U.S. Dollar Risen Over Time?

This query was where I really expected to trip up the chatbot. When I initially ran this experiment with an earlier version of ChatGPT, it made some glaring mistakes.

However, this time ChatGPT got it right and balanced out its answer with a healthy amount of context.

Question #6 to ChatGPT

Question #6 to ChatGPT (continued)

I also asked a similar question in a more generic sense about all fiat currencies, and likewise received a reasonably balanced response.

Question 7: What Makes the 1944 Penny Rare?

Asking about rare pennies was another case that was tricky for the older version of the software. Its second effort was much more accurate.

Question #7 to ChatGPT

Question #7 to ChatGPT (continued)

For specific pricing information about the value of the 1944 penny, check out our comprehensive article on the topic.

Question 8: Why Is Platinum Cheaper Than Gold?

You might expect artificial intelligence to struggle with more nuanced questions about commodity markets and pricing. In fact, it seemed to have a pretty good grasp on where the prices for these two different precious metals come from.

Question #8 to ChatGPT

Question #8 to ChatGPT (continued)

Our blog post about why platinum is so cheap provides a lot of extra details about this question. But there's no denying that ChatGPT's summary is good.

Question 9: Why Is Silver So Valuable?

This might be the kind of question you have wondered yourself. Of course, there are several different reasons why silver has been a valuable metal for thousands of years. The chatbot did well to touch upon each of these different factors:

Question #9 to ChatGPT

Question #9 to ChatGPT (continued)

For a more holistic answer, be sure to read our article about why silver is so valuable by following the link.

Conclusion: AI Is a Great Tool, But Isn't Close to Replacing Human Intelligence

It's worth pointing out that when I asked these questions a second time to the new-and-improved ChatGPT-4, it didn't make the same mistakes. One reason GPT-3.5 failed where GPT-4 succeeded is that the latter version handled leading questions better. If I ask why a coin is rare, or if the dollar's value has risen, the older version of the software seemed to assume that the coin in question must be rare and the dollar must have risen in value. The possibility that the assumptions built into the question could be wrong was not considered.

Since the recent release of ChatGPT-4, the artificial intelligence has undoubtedly gotten better at parsing a question and anticipating potential objections to its answer. It also provided a more comprehensive treatment of the question. By comparison, the earlier version was at times susceptible to giving a ham-fisted response that confirmed the assumptions of the query at essentially any cost.

This technology is obviously improving. The AI that powers ChatGPT seems to be capable of some impressive feats of human-like intelligence. It still has a tendency to give long-winded answers when something more concise will do. Yet the extra context it provides tends to be quite sophisticated.

There are still plenty of skeptics regarding the level of "consciousness" or "intelligence" that ChatGPT exhibits. Count the famous linguist Noam Chomsky among them. (Say what you will about his polarizing political views, but Chomsky's understanding of linguistics is unassailable.) In a New York Times op-ed with two co-authors, Chomsky opined:

"The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data and extrapolating the most likely conversational response or most probable answer to a scientific question. On the contrary, the human mind is a surprisingly efficient and even elegant system that operates with small amounts of information; it seeks not to infer brute correlations among data points but to create explanations.”

According to the criticisms made by Chomsky et al, even the most cutting-edge AI still lacks a "theory of mind"—that is, it can't empathize with or understand the mental states of living entities, much like children that have not reached certain developmental milestones. It bears repeating: we're not dealing with something sentient like Skynet.

Not yet, at least!

I came away from the experience (and I hope you do as well, reader) rather impressed by the level of sophistication of the current incarnations of artificial intelligence. (Follow the link to watch an excellent video that explains what chatbots are doing "beneath the hood," so to speak.) I still don’t buy into the hyperbole and the exaggeration about how it’s going to destroy a huge chunk of white-collar jobs. ChatGPT doesn't think creatively on its own.

I do believe, however, AI has awesome potential to transform the way that many industries work, especially in the financial services, marketing, and administrative sectors. It's darn good at streamlining mundane tasks that have been done many times before.

Technological advancement over the last half-century has made all of our lives immeasurably better. We shouldn't necessarily be afraid of AI; we simply should remain conscious of how the technology is applied—an issue known in the AI industry as "alignment."

Read more about coins and precious metals from the human experts at Gainesville Coins:

What Is Bullion?: Definition, Uses, and Complete Guide

Guide to Numismatics: Discover the Thrill of Coin Collecting

Is the World on the Brink of a Banking Crisis?

The Hierarchy of Money and the Case for $8,000 Gold

How To Sell Your Coin Collection

What Are the Purest Gold Coins in the World?

1919 Lincoln Wheat Penny Value