Quest 2 of 15
How AI Reads Language
Peek under the hood: tokens, probabilities, and temperature. Understand why the same prompt can give different answers—and how to control creativity when using AI writing and study tools. For example, you will compare two precise and two creative responses to the same revision question.
Start here
A language model is a model trained on large amounts of text to predict and generate language. You do not need to know programming: this part follows how it turns your words into an answer, one small piece at a time.
Big idea
Language models do not read whole sentences the way you do. They chop text into tokens, guess the next token using probability, and settings like temperature control how creative or predictable the answer feels.
How text becomes tokens
The sentence you type to Gemini.
Learn one idea at a time
Read, explore, then mark each idea when you can explain it.
Idea 1 of 6
Hold the full path in mind before zooming into the details: your prompt is the input; a language model processes it using patterns it learned during training on enormous amounts of text; and the generated reply is the output. Notice what is not on that path — there is no library lookup, no database of verified facts, no step where the model checks whether its sentence is true. It predicts language; it does not retrieve answers. This single picture explains most of the model's behaviour: why it is fluent about almost anything, why it can be confidently wrong, and why the way you word your prompt changes the reply so much — the prompt is the pattern the whole prediction builds on.
Choose a deep dive
Open the topics you want to explore. The detail stays folded until you need it.
Deep dive 1Why a word is not always one token
A model breaks text into small pieces that may be whole words, word fragments, spaces, or punctuation. This lets it handle unfamiliar names and many languages, but it means a 100-word answer is not exactly 100 tokens. Token counts matter because they affect limits, speed, and cost.
Deep dive 2Temperature changes variety, not knowledge
Ask for a definition of evaporation at low temperature and the replies should stay similar. Ask for poem ideas at higher temperature and you may see more surprising combinations. If the model lacks a fact, increasing temperature only changes the style of its guess; it does not make the fact more reliable.
Deep dive 3Follow one sentence through the model
Type 'The capital of Zimbabwe is' and the model splits your text into tokens, converts each token to numbers, and passes them through many layers that weigh how every token relates to every other. The output is a probability for each possible next token: 'Harare' scores extremely high because that pattern dominates its training text. The model appends the winner and repeats the whole process for the next token. Nothing in this loop consults a map or a fact database — the right answer emerges because the pattern was overwhelmingly consistent in training data. Ask about something rare or recent, and the same confident machinery produces a guess instead.
Deep dive 4Why models have a knowledge cut-off
A language model learns from a snapshot of text collected up to a certain date, then training stops. Events after that date simply are not in its weights, which is why a model may not know this year's exam syllabus change or last month's election result. Tools work around this by giving the model fresh text at question time — search results, a pasted document, your notes — which it can read inside its context window. Knowing the difference between what a model memorised in training and what it was handed in the prompt helps you predict which questions it will fumble.
Deep dive 5Tokens are pieces of text
Models chop words into tokens and guess what comes next. That is why long answers cost more and why temperature changes how adventurous the wording feels.
One-minute challenge
Connect this lesson to real life
Name one situation where this idea could help, and one thing a person should still check.
Explore a real-world example
Use the arrows to connect the idea to a visible situation.
Photo example
Example: software is still physical
Even “in the cloud” means real computers somewhere else are doing the maths for each reply.
Key terms
Tap a term to flip and read the definition.
Optional further learningFree textbooks and trusted online resources
These sources informed the course structure. Use them to revisit a concept or study it in more depth.
Ready check
Tick each idea only when you could explain it without looking back.
Ready for practice? Use the match game and temperature lab to see these ideas in action — run the same prompt twice and compare.
Extra context (audience, logistics, curriculum notes)
Built for: No advanced math required—focus on intuition that makes you a smarter tool user.
Formats: Mini-lecture + demo · AI chat lab with temperature modes · Token counting exploration
AI Architect Module 2 — tokens, embeddings intuition, next-token prediction, temperature (AI_Architect_10_Part_Course.md).
Next up
Ready for the next part?
When you've finished the reading, inline exercises, and knowledge check for this part, check the box to continue.