Prompt caching cuts input cost up to ~90% — but it fails silently. A broken cache still returns correct responses; you just quietly pay full price. cachedoctor tells you whether your cache is working, why it isn't, and how much it's costing you.
Run observe as a local pass-through proxy and set your SDK's base URL to it. Requests are forwarded unchanged — read-only, any language, key never stored — and every call gets a live diagnosis.
$ cachedoctor observe $ export ANTHROPIC_BASE_URL=http://localhost:7070 # or OPENAI_BASE_URL $ python your_app.py 🔴 #1 anthropic /v1/messages → 200 · hit 0% · No prompt caching enabled 🔴 #2 anthropic /v1/messages → 200 · hit 0% · New cacheable prefix (drifted?) ── session summary ── 2 requests · hit rate 0% · spent ~$0.42 · recoverable ~$18.30 (this session)
One static Go binary. No Python environment, no LLM calls, no telemetry — deterministic rules, runnable air-gapped against logs you already have.
cachedoctor observeLive proxy: diagnose real traffic as it flows. Anti-patterns, prefix drift between calls, running hit rate, dollars recoverable.
cachedoctor check req.jsonScan one request body for cache anti-patterns — volatile values in the prefix, missing or misplaced breakpoints, below-minimum prefixes, models with no caching at all.
cachedoctor diff a.json b.jsonTwo consecutive calls that should have hit the cache but didn't? See the exact byte where the prefix identity broke.
cachedoctor analyze usage.jsonlPoint it at response logs you already collect: real hit rate and $/month recoverable, priced per model.
Caching applies wherever the same tokens lead the prompt on consecutive calls — in most production apps that's 80–95% of every request. Every row below has a way to silently break byte-identity.
| Use case | What silently kills the cache |
|---|---|
| Support / chat bots | a timestamp in the system prompt |
| Coding agents, IDE assistants | prefix drift between call types |
| Document Q&A | re-serializing the doc per question |
| RAG | shuffled retrieval order |
| LLM-as-judge / evals | rebuilt few-shot example order |
| Multi-agent fan-out | tools serialized from an unordered map — random per worker |
| Scheduled batch jobs | call cadence longer than the TTL: all writes, no reads |
| Voice / real-time agents | per-utterance session metadata up top |
check and diff exit 2 on a high-severity issue — a strict exit-code contract (0 clean · 1 unreadable input · 2 finding · 64 usage error), so a flag typo can never masquerade as a passing build. There's a ready-made GitHub Action that checks saved request bodies, diffs them against the base branch, and comments the impact on the PR.
$ cat req.json | cachedoctor check - 🔴 HIGH — Volatile content in the cached prefix Your prompt prefix contains a volatile value (timestamp: "2026-09-14T07:12:03Z"). The provider caches the longest byte-identical prefix, so anything that changes near the front means the cache never engages — silently. fix: Move changing values toward the end; keep system and tools byte-stable. $ echo $? 2
The CLI tells you where the money leaks. cachedoctord is the commercial companion proxy that closes the leaks continuously — and proves the savings on your real traffic.
--exact. Errors are engineered to only ever over-warn — a below-minimum prefix is never silently missed.cache_control breakpoints and automatic prefix caching are different failure surfaces — both are covered, including models that predate caching entirely.diff doesn't say "your prefix changed" — it shows the first diverging byte, with context.