Level 1Unit D · Session 25
25

Unit D · AI and chatbots

What a chatbot is, and what it is not

What we are making: an accurate picture of what is happening when you type a question into a chatbot. Most people, including adults, have this wrong.

The idea

A chatbot is extremely good at one thing: guessing which word comes next. That is not a simplification for your benefit — it is the whole of what it does. It read more text than a person could read in a thousand lifetimes, learned which words follow which, and now guesses one word at a time until it has an answer.

YOUThe capital of France isTHE CHATBOTParis.
It guessed 'Paris' because in everything it read, that word followed those words.

Why that matters

Guessing well produces answers that sound right whether or not they are. It cannot tell the two apart, and it is not lying when it is wrong — lying needs you to know the truth first.

New words
AIArtificial intelligence. Programs that do things we used to need people for.
ChatbotAn AI you type to, that types back.
ModelThe trained program itself. Its 'brain', if you like.
TrainingThe long process of it reading everything and learning the patterns.
PromptWhat you type. The question or instruction you give it.

What it is not

What people thinkIt looks things upIt knows what is trueIt understands youIt has opinions of its ownIt remembers you afterwardsWhat is actually trueIt predicts likely wordsIt has no idea what is trueIt matches patterns in textIt reflects what it readUsually it forgets completely
Every item on the left is a reasonable guess. Every one of them is wrong.

A chatbot that tells you something false is not malfunctioning. It is doing its job: producing text that looks like a good answer.

💡 Tip
This is why you still matter. The chatbot produces something plausible. Deciding whether it is actually correct is your job, and it cannot be handed over.

Do it — the prediction game

No computers. You are going to be the chatbot.

  1. Your teacher says the first half of a sentence. Everybody writes down the next word. Just one.
  2. Compare. Most of the class will have written the same word. That is what a chatbot does.
  3. Do it again with a sentence nobody in the room knows the ending of.
  4. Notice: you still wrote something. It still sounded like a sentence. It was still a guess.

Do it — meet one

One chatbot, on the projector, the whole class together.

  1. Ask it something you already know the answer to. Check it.
  2. Ask it to explain something from Unit B — what a loop is. Is the explanation good?
  3. Ask it something about your own school. Watch what it does.
  4. Ask exactly the same question twice. Compare the two answers. Are they identical?
💡 Tip
That last one is revealing. Ask a person a question twice and you get the same answer. Ask a chatbot and you often get two different ones, because it is guessing each time, not remembering a fact.

Try it — three questions to discuss

  • If it does not know what is true, why is it right so often?
  • Where did all the text it read come from? Who wrote it?
  • What kinds of question would you expect it to be worst at?

What you learned

  • A chatbot predicts the next word, over and over.
  • It does not look things up and does not know what is true.
  • Sounding right and being right are different things.
  • The same question can get different answers.
  • Checking the answer is your job, not its.
Challenge optional — only if you finish early
  • Ask a chatbot the same question in English and in Arabic. Are the answers the same?
24. Sense, decide, act