Next-token prediction
Next-token prediction is the only thing a model does: pick the next chunk of text, add it, and go again.
There is no plan and no draft. It writes one piece at a time, and each piece is chosen based on everything before it.
Everything that looks like reasoning comes out of this loop. Knowing that is what makes the failures make sense.
Was this page helpful?