In Models and Harnesses, you saw that the model does the thinking. This lesson looks at where that thinking comes from, and what a model knows before your app hands it anything.
How a Model Learned
A model learned by reading an enormous amount of text: books, articles, websites, and conversations, far more than a person could read in many lifetimes. This is called training.
During training, the model practiced one task over and over: read some text and predict what comes next. To get good at that, it had to pick up grammar, facts, writing styles, and ways of reasoning, because all of those help it predict better. The company then trains it further on examples of good replies, so that it answers questions and follows instructions like an assistant.
Once training ends, the model is finished. It doesn’t keep reading the internet, and your conversations don’t change the model you’re talking to.
Writing One Piece at a Time
When you send a message, the model writes its reply the same way it trained. It predicts a likely next word, or piece of a word, adds it, and predicts the next one, until the reply is done.
You can feel what that’s like. Finish this line from a work email: Please find the report … You probably had attached in mind before you’d finished reading. You didn’t look anything up. The word was simply the most likely one to come next. A model does something similar, at a much larger scale.
This explains two things about AI replies:
- They sound natural. The model learned from text written by people, so its replies read the way people write.
- They don’t come from a list of facts. The model doesn’t look anything up in a stored collection of facts. It writes what’s likely, based on everything it read. For well-known things, the likely answer is usually the right one. For something rare, very specific, or that the model never read about, it can still write a smooth, confident answer that’s wrong.
That second case has a name. When a model makes something up and presents it as fact, it’s called a hallucination. Made-up details, like quotes, studies, or sources that don’t exist, are a known problem that the AI companies point out themselves.
The Knowledge Cutoff
Training uses text up to a certain date, called the knowledge cutoff. A model is released some months after that, and people keep using it for a long time afterward, so there’s always a gap between what the model learned and what’s happening now.
Ask a model on its own about last week’s news, and it has nothing to go on. It might tell you so, or it might answer from older information without mentioning that it’s out of date.
That’s why many AI apps can search the web. Searching is a harness job: the app runs the search and hands the results to the model, often with links to the pages it used. When you see those links, the answer came from a search, and you can open them to check.
When a Smooth Answer Is Wrong
Because a model always writes what’s likely, a wrong answer reads just as smoothly as a right one. A confident tone tells you nothing about whether the answer is right. You’ll come back to this in Habits for Checking Results, with habits for catching it.
Where Would the Answer Come From?
A good answer can come from three places: what the model learned in training, a search the app runs, or your own material that the app has to hand over. For each request below, decide where a good answer would have to come from:
The third bin, your own material, is where the next lesson picks up. A model can’t reach your files, email, or accounts by itself, and the next lesson shows how the app gets them to it.
Next: The Model’s Limits
