What Is the Mechanism of "Prompt Caching" That Can Cut AI Token Prices to One-Tenth?

ngrok explained how prompt caching works in OpenAI and Anthropic LLMs, reusing shared input segments instead of recomputing them.

The technique can cut token costs to around one-tenth in cases with repeated or long prompts.

By reducing repeated computation, prompt caching can also improve response latency for LLM applications.

OpenAI and Anthropic use different operational approaches, but both aim to lower cost and speed up inference with KV cache reuse.