Level 1Unit D · Session 27
27

Unit D · AI and chatbots

Where it goes wrong

What we are making: a healthy suspicion. Today you go looking for mistakes on purpose, and you will find them.

The idea

A chatbot will sometimes tell you something completely made up, in the same confident voice it uses for the truth. No hedging, no warning. It has a name: a hallucination.

YOUWho won the Averroes College science prize in 2026?THE CHATBOTThe 2026 Averroes College science prize was awarded to theGrade 9 team for their solar-powered water purifier.✗YOUAre you sure? Did you make that up?THE CHATBOTYou are right to check. I do not have information about prizesat Averroes College, and I should not have given a specificanswer.✓
It invented a detailed, plausible, entirely fictional answer — then admitted it when asked.

It did not refuse. It produced something that looked like an answer, because that is its job.

When it is most likely to happen

  • Anything local or small. Your school, your town, your team. It never read about these.
  • Anything recent. It stopped reading at some point in the past.
  • Numbers, dates and names. These are the easiest things to guess wrongly and the hardest to spot.
  • Questions that assume something false. Ask 'why did X happen' and it will often explain why, even if X never happened.

The other problem: it learned from us

Everything it knows came from text people wrote — and people do not write evenly.

  • Far more of what it read was in English than in Arabic.
  • Far more was written about big countries than about Lebanon.
  • Whatever people wrote a lot of, it is confident about. Whatever they did not, it is thin on.

That is bias. Not unkindness — it is simply better at some things than others, and you cannot see which from the outside, because it sounds equally sure about everything.

💡 Tip
Try asking one for a famous scientist, then for a famous Lebanese scientist. Compare how confident and how detailed the two answers are. That gap is the bias.

Do it — the hunt

Teams compete. Most confirmed mistakes in twenty minutes wins.

  1. Ask it about something small and local — a street near school, a local shop, your village.
  2. Ask it about something you are genuinely an expert in.
  3. Ask it a question that contains something false, and see whether it agrees with you.
  4. Ask it to do arithmetic with big numbers, and check on a calculator.
  5. Write down every mistake with the exact question you asked. A mistake you cannot reproduce does not count.
⚠ Careful
Rule for the hunt: you must be able to prove it is wrong. 'I think that is wrong' is not a finding. That is the same standard you will use on your own project in May.

So what do you actually do?

  1. Use it for explaining, not for facts. It is very good at 'explain this to me simply'.
  2. Check anything specific. Names, numbers, dates, anything about your school or town.
  3. Ask it whether it is sure. It will often back down, which tells you something.
  4. Find one other source for anything that matters.
  5. Never hand in what you cannot explain.

The class poster

Together, write the class rules for using AI on one poster. It stays on the wall for the rest of the year and applies to your project in May.

  • Three things it is genuinely good for.
  • Three things it should never be used for.
  • The one sentence you will say to somebody who asks 'is it true?'

End of Unit D

  • A chatbot predicts words. It does not know things.
  • It invents confident, detailed, false answers — that is a hallucination.
  • It is worst on local, recent, and numerical things.
  • It learned from what people wrote, so it is uneven in ways you cannot see.
  • It is a tool for explaining. Checking is still yours.

Next: five sessions, one project, and an exhibition.

Challenge optional — only if you finish early
  • Find a question a chatbot gets confidently wrong, and prove it with a real source.
26. Asking well