Cache tokens
Cache tokens are the part of your input that has already been processed and does not have to be paid for again in full.
Your rules and your context files are identical at the start of every request in a session. Sending them again is wasteful, so they get held and charged at a fraction of the rate.
It is why a long conversation can get cheaper per turn rather than steadily worse, and why reordering what you send can quietly break it.
Was this page helpful?