Recommended Reading
What This Article Covers
When you continue using the same AI service, there may come a point when you start thinking:
“Does it seem to answer better than before?”
“Does it seem to perform better now that I am using a paid plan instead of the free version?”
“Does it seem to understand what I want better now than when I first started using it?”
On the other hand, even with the same AI, responses may sometimes feel fast and at other times slow.
It is easy to think that all of these changes mean that the AI model itself has improved, but that is not always the case.
What we commonly call “AI performance” actually includes several different kinds of changes.
The capabilities of the AI model itself may change, or the conditions under which the AI service is used may change.
An AI service may also provide answers that are better suited to the user by referring to previous conversations or preferences, while internet or device conditions may make the service feel faster or slower.
Therefore, when AI performance seems to have changed, it is important to distinguish what has actually changed.
Companies improve the performance of AI models
First, let’s look at the performance of the AI model itself.
An AI model does not continuously grow on its own simply because a user keeps asking it questions.
Companies that develop AI create new models or update and adjust existing ones.
Through this process, capabilities such as understanding sentences, reasoning, writing, coding, and understanding images may change.
When a new or improved model is then applied to an AI service, users may receive better answers than before.
In this case, the capabilities of the AI model itself have actually changed.
The conditions available within the same AI service can differ
However, what we actually use is usually not the AI model itself, but an AI service that allows us to use the model.
For example, GPT is an AI model, while ChatGPT is an AI service that allows users to use that model.
Therefore, the performance of an AI model and the performance we experience while using an AI service do not always mean the same thing.
An AI service may provide multiple models and features, and the range of what users can access may differ depending on whether they use a free or paid plan.
Depending on the service, higher-tier plans may provide access to higher-performing models, more features, or greater usage limits.
In this case, a user may naturally feel:
“The AI got smarter after I switched to a paid plan.”
However, the AI model itself did not become smarter specifically for that user because they paid for the service.
The models, features, usage limits, and other service conditions available to the user have changed.
AI may seem to understand me better
As you continue using an AI service, you may also feel that it gives answers that suit you better than when you first started using it.
This can also be different from an improvement in the performance of the AI model itself.
If an AI service can use previous conversations, information provided by the user, saved preferences, or memory, the same AI model may be more likely to generate answers that are better suited to that user.
For example, imagine asking an AI you are using for the first time:
“Write something for me.”
The AI does not know what writing style the user prefers, how long the text should be, or whether the user wants a technical explanation or an easy-to-understand one.
However, if the user previously said:
“Don’t use too much technical terminology. Explain it so that non-developers can understand it.”
and the AI service can refer to that information, it may later produce writing that better matches those conditions.
The user may feel that the AI now understands them better than before.
However, this does not necessarily mean that the AI model itself has learned from the user again and become smarter.
The amount of user context that the AI can refer to when generating an answer has increased.
Internet and device conditions can affect how fast AI feels
The user’s environment can also affect whether AI feels faster or slower.
Generative AI services that we use generally do not perform all AI computations directly on the user’s computer. Instead, the AI model runs on servers, and the results are delivered to the user through the internet.
Therefore, when the internet connection is fast and stable, sending a question and receiving a response may also feel smoother.
When uploading files to an AI service, the time required to send the file can also vary depending on the internet upload speed.
The condition of the computer, smartphone, or browser being used can also affect how smoothly the screen is displayed or the service is operated.
However, faster internet does not improve the reasoning ability of the AI model.
Replacing a computer with a newer one also does not make the AI model generate more accurate answers.
In this case, what mainly changes is not the performance of the AI model, but the speed and experience of using the AI service.
What we call “AI performance” includes several different factors
The points discussed so far can be summarized in the following table.
| What changed? | Actual change | What the user experiences |
|---|---|---|
| AI model | The company develops a new model or updates and adjusts an existing one | Better answer quality |
| Service plan | Available models, features, usage limits, and other conditions change | Access to higher performance or more features |
| User context | Previous conversations, memory, preferences, and other information are used | The AI seems to understand the user better |
| Usage environment | Differences in internet connection, device, browser, and other conditions | Faster and smoother use |
All four can affect the experience a user has while using AI.
However, the reasons behind those changes are different.
An improvement in the AI model itself is different from gaining access to another model or additional features through a higher-tier plan.
An AI giving answers that better match a user’s preferences can also be different from an improvement in the model’s reasoning ability itself.
The same applies when better internet or device conditions make an AI service faster and smoother to use.
In that case, what has changed is not the capability of the AI model, but the usage environment.
Therefore, what we commonly call “AI performance” can include the performance of the model itself, service conditions, user context, and the usage environment.
Understanding these differences makes it easier to determine whether the AI itself has actually become smarter or whether the experience of using the AI service has changed.
DANA NOTES in One Sentence
When we feel that “AI performance has improved,” the change may be influenced not only by improvements in the AI model itself, but also by service conditions, user context, and the internet and devices used to access the service. The important point is to distinguish the performance of the AI model itself from the performance experienced while using an AI service.
