Blog

Why repeated reading should cost less

A terminal session repeats itself. The same instructions, the same files, the same plan, then one new question. If every repeat is billed like a first reading, the tokens go to memory rather than to the change.

What caching is for

Prompt caching keeps a repeated prefix warm. The next step pays less to look at that prefix again than it would to read it as new text. Toknary leaves caching on. You do not turn it on per request.

What we are studying

Which parts of a coding session actually repeat. A stable plan repeats. A file that changes every step does not. The research is to keep the stable part warm and to stop dragging the changed part through every later call. We are not publishing a savings figure. We are watching real sessions, which is why the tokens are free while we learn. The reading habit that makes caching useful is in read the lines you need.

All notes