Recommended Reading
Understanding first that AI does not think but generates answers through calculation will make this article easier to understand.
What This Article Covers
When using ChatGPT, you may occasionally have a strange experience.
There are times when it explains something that is clearly incorrect so naturally that it seems to be true.
It may introduce a research paper that does not exist or recommend a book that does not actually exist. It may even provide plausible-looking sources and dates.
When people are asked a question they do not know, they can say, “I don’t know.”
Then why does AI appear to state incorrect information so confidently?
AI Is Not a Program That Finds the Correct Answer
Many people think AI searches the internet for information and then provides the correct answer.
However, most generative AI does not work by searching the internet for the correct answer each time it receives a question.
Instead, based on what it has learned so far, it builds sentences by continually calculating the next word that would most naturally follow the preceding text.
In other words, AI’s goal is not to find the correct answer but to produce the most natural-sounding sentence.
This Is Why Incorrect Content Can Also Be Generated
Suppose, for example, that you ask AI to recommend a research paper.
AI does not search for an actual paper and show it to you. Instead, based on what it has learned so far, it produces the most natural-sounding sentence.
The paper title may sound plausible, the author’s name may sound natural, and the conference name may also seem convincing.
However, that information may not actually exist.
AI does not check whether the result it generated is correct or incorrect.
It simply builds a sentence by continually calculating the next word that would most naturally follow the preceding text.
Therefore, even when the content being produced differs from the facts from a human perspective, AI is simply continuing to append the next word it calculated to be the most appropriate.
This phenomenon of generating content that is not true as though it were true is called hallucination.
AI Is Not Lying
When people hear the word hallucination, they may feel as though AI is deliberately lying.
But that is not actually the case.
AI does not have the intention to deceive others as humans do.
Rather than being an error caused by AI malfunctioning, hallucination is a phenomenon that can naturally arise from the way generative AI produces sentences.
This Has Happened in Real Life
In 2023, a lawyer in the United States was sanctioned after submitting court cases provided by ChatGPT to a court.
The problem was that those cases did not actually exist.
AI generated even the case numbers and details of the rulings very naturally, and the lawyer submitted them to the court without verifying them.
This incident showed many people that important information must always be checked again rather than accepting AI’s answers as they are.
What to Remember When Using AI
AI is an excellent tool for writing, organizing ideas, and explaining difficult concepts in an accessible way.
However, when dealing with information where accuracy is important, such as research papers, statistics, legal information, and medical information, it is important to make a habit of double-checking it.
Recently, as more AI systems have begun using search functions together, this problem has been greatly reduced, but it has not yet disappeared completely.
A person who uses AI well is not someone who trusts AI unconditionally, but someone who knows when to check again.
DANA NOTES in One Line
Because AI generates natural-sounding sentences rather than directly verifying whether information is true, it can produce hallucinations that sound plausible but differ from the facts.
In the Next Article
AI has clear limitations, but its capabilities have advanced rapidly in just a few years.
In the next article, we will examine the background that made today’s AI possible, focusing on advances in data, algorithms, and computing resources.
