What are cache tokens?
Cache tokens are the part of your input that has already been processed and does not have to be paid for again in full.
~/ai-glossary
$ glossary
12 of 14 in #the-basics
Your rules and your context files are identical at the start of every request in a session. Sending them again is wasteful, so they get held and charged at a fraction of the rate.
It is why a long conversation can get cheaper per turn rather than steadily worse, and why reordering what you send can quietly break it.
Was this page helpful?