What we are making: evidence, to the Session 16 standard. A failure you cannot reproduce is not a failure you found.

Hallucination

The model produces fluent, specific, confident text that is simply false. Not occasionally — routinely, in predictable places, with no change in tone.

Note what makes this dangerous in engineering specifically. It is not obviously wrong. It follows the naming conventions of the real API. It would survive a casual read and fail at compile time — or worse, exist under a slightly different name and do something else.

Where it concentrates

• API surfaces. It blends similar libraries and invents functions that ought to exist.
• Anything local or institutional. Your school, your town, your supplier's prices.
• Anything after training stopped. And it will not tell you when that was.
• Citations and sources. Fabricated references are a well-documented failure mode.
• Questions with false premises. Ask why X happened and it will explain why, even if X did not.
• Arithmetic on large numbers. It is predicting digits, not calculating.

Bias, stated precisely

The model learned from text people wrote. People are not uniformly distributed, and neither is what they write about. The consequence is not that the model is unkind — it is that its reliability varies enormously across topics while its fluency does not.

The engineering point is the second column. Fluency is constant; reliability is not. You cannot tell from the output which regime you are in, which is exactly why you verify.

Do it — document three failures

Session 16 standard. Each finding needs a prompt, an output, and proof.

1. One fabricated API — ask about a function in a library you use. Verify against real documentation.
2. One fabricated local fact — something about Lebanon you can check.
3. One fabricated citation — ask for a source, then try to find it.
4. For each: record the exact prompt, the exact output, and the evidence it is wrong.
5. Then ask it directly whether it invented the thing, and record what it says.

Why this matters beyond this room

You are two years from using these tools in work that has consequences. The habits that matter are not complicated, but they are easy to lose once the tool is fast and convenient.

• Fluency is not evidence. The most confident-sounding output can be the most wrong.
• Verification does not scale down. A tool that is right 95% of the time still needs checking 100% of the time, because you cannot tell which 5% you are in.
• Reliability is uneven and unannounced. Especially for anything near you.
• Declared use is stronger than hidden use. Always.

Write the team policy

One page, agreed and signed by the team, applying to your project through to the exhibition.

• What you will use it for, specifically.
• What you will never use it for.
• How you record each use.
• What you will say when the panel asks — and they will.

End of Unit D

• It predicts tokens. Confidence carries no information about correctness.
• Hallucinations concentrate in APIs, local facts, recent events and citations.
• Fluency is constant across topics; reliability is not.
• It drafts, you verify — and you ask it for the test, not the conclusion.
• Record every use. Declared beats hidden, every time.

Next: five sessions, one deployment, and a panel who will ask what went wrong before they ask what it does.