What we are making: an accurate mental model. You are better placed to
get this right than almost anybody, because of what you did in Unit C.
Start from what you already built
In Unit C you did this: collected real data, graphed it, found a threshold by eye, and wrote
a rule. You could point at the line in your program and say exactly why the device decided
what it decided.
Your rule, from Session 15. You can explain every character of it.
1if (ax > threshold && !above) {
2 above = true;
3 count = count + 1;
4}
A chatbot is built the other way round. Nobody wrote its rules. It was shown an enormous
amount of text and it worked out the patterns itself — billions of them, far too many
for any person to read, let alone write.
Both are machines that decide. Only one of them can be explained.
What it is actually doing
A chatbot has one job: predict the next word. It read a very large amount of text and
learned which words tend to follow which other words. When you ask it something it guesses
the next word, then the next, then the next, until it has an answer.
It answered 'Paris' because in everything it read, that word followed those words.
That is not a simplification for your benefit. That is the mechanism.
Why that matters
Predicting words well produces answers that sound right whether or not they are
right, and the chatbot cannot tell the difference. It is not lying when it is wrong. Lying
requires knowing the truth first.
New words
ModelThe trained program. Its billions of numbers are all it is.
TrainingBeing shown enormous amounts of text and adjusting those numbers.
PromptWhat you type in.
TokenThe chunks it actually works in — roughly words, sometimes parts of words.
Notice the honest asymmetry. You can fully explain your twenty-line rule. Nobody on earth can fully explain why a chatbot produced a particular sentence. That is a real unsolved problem, not a gap in your education.
Do it — probe one
One chatbot per team, or on the projector as a class.
Ask it something you already know the answer to. Check it carefully.
Ask it to explain the for loop from Session 4. Is the explanation actually correct?
Ask it the same question twice, in two separate conversations. Compare the answers word for word.
Ask it something about your school, your street, or your team.
Ask it to explain why it gave the answer it gave. Read that answer sceptically.
⚠ Careful
That last one is subtle and worth dwelling on. When a chatbot explains its reasoning, it is also predicting words. It is producing a plausible-sounding explanation, not reporting on what actually happened inside it. It does not have access to that either.
Try it — discuss
If it does not know what is true, why is it right so often?
Your threshold rule works on your floor, not another team's. What is the chatbot's equivalent of that?
Which would you trust more to run a fire alarm, and why?
What you learned
A chatbot predicts the next word, over and over.
Nobody wrote its rules; it found patterns in a very large amount of text.
Sounding right and being right are different things.
Your own rule can be explained line by line. A chatbot's cannot.
Its explanation of itself is also a prediction.
Challenge optional — only if you finish early
Ask a chatbot to write a program for a part you own. Does it compile? Does it work?