Prompt Caching: Where the Saving Goes Missing
Prompt caching cuts the cost of repeated AI input to about a tenth. The mechanism is simple to describe and unforgiving in practice: it matches on exact bytes, fails silently, and can cost more than it saves. Here is how it works and where the money actually leaks.