Prompt caching & context optimization for harnesses

Addendum: what MCP 2026-07-28 changes

I wrote the above against MCP 2025-06-18. The current revision is 2026-07-28, and it is a rewrite rather than a patch. Protocol sessions are gone. The initialize handshake is gone. Three things I treated as harness problems are now protocol rules, and one of my bullets above is wrong.

The correction

Newly activated tools are sorted into the tool list, not simply appended at the end

That is wrong as a single rule. The spec now ships a Client Best Practices page with a section on prompt caching, and it says the opposite:

Append newly discovered definitions after the cache breakpoint rather than re-sorting the tools array, or route every call through a single stable call_tool({name, args}) meta-tool so the array never changes.

Both rules are right. They apply at different moments, and I collapsed them into one:

  • Sort at run start. A canonical order makes the next run with the same tool set hit the cache. Order the array once, before the prefix is warm.

  • Append mid-run. Once a prefix is warm, never touch the array. An insert and an append cost the same, because tools sits first. That is exactly the cost you are trying to avoid.

The third option in that quote beats both. Expose one stable call_tool({ name, args }) and the array never changes at all. The prefix then survives every discovery, for the whole run. You pay for it by losing provider-side argument validation and constrained decoding, so the harness has to validate every call itself.

Three harness problems became protocol rules

The tool set can no longer drift per connection. SEP-2567 removed protocol sessions and the Mcp-Session-Id header. The tools page now requires that the set "MUST NOT vary per-connection or as a side effect of other requests on the connection". It may vary by the authorization on the request, because credentials are per-request input. A server that needs cross-call state mints an explicit handle and takes it as an ordinary tool argument. So the tool list is a function of (server, auth) and never of history. That is the cache-stability property, written into the protocol.

Deterministic order is now a SHOULD, and prompt caching is the reason given. Verbatim from the tools page:

Deterministic ordering enables clients to reliably cache the tool list and improves LLM prompt cache hit rates when tools are included in model context.

It is still a SHOULD, so the harness canonicalizes anyway. Sort by code point rather than localeCompare if you hash the result: ICU collation depends on the build and the locale, so the same tool set can otherwise hash differently on two machines.

Freshness hints are mandatory. SEP-2549 added a CacheableResult interface. ttlMs and cacheScope are now required on tools/list, prompts/list, the resources/* lists, resources/read, and server/discover. public is the recommended scope for a tool list. The cache key is the method plus the parameters that affect the result, so each page caches on its own clock and there is no cross-page consistency guarantee.

One more: invalidation is opt-in now. notifications/tools/list_changed reaches you only if you opened a subscriptions/listen stream with toolsListChanged: true. The old HTTP GET endpoint is gone. The harness decides whether it wants the interrupt at all.

The new failure mode: invalidation is lossy

SEP-2575 removed SSE resumability and the Last-Event-ID header. A broken stream loses the in-flight request. Notifications are not redelivered. The server holds no subscription state across reconnections, so the client has to re-send subscriptions/listen itself.

A missed list_changed is therefore undetectable, and the consequence inverts the intuition. A lossy invalidation channel makes your prompt cache live longer and your catalog wronger. You can hand the model a schema for a tool the server dropped an hour ago.

The spec plans for this. Clients may re-fetch before the TTL expires on "an unexpected error on a tool call indicating the method was not found or the parameters were invalid". So the trigger set is three-way, not one-way:

  1. the notification, when it arrives
  2. ttlMs expiry, which is the only guaranteed path back to the truth
  3. a tool call that fails with an unknown method or invalid parameters

Treat ttlMs as load-bearing. It is not a nicety on top of the notification any more.

The cost that now dominates the one this post optimizes

Multi round-trip requests (SEP-2322) let tools/call return resultType: "input_required". The client answers on a retry that carries inputResponses and requestState, under a new JSON-RPC id. Those results must not be cached. Tasks moved out of the core protocol into an extension that polls with tasks/get.

Both make a run park, and park in the middle of a call. A park longer than the provider cache lifetime costs a full cold write on resume, whatever your tool discipline is. That single event can cost more than every token the lazy catalog saves.

Tool laziness protects the prefix. Only park discipline protects the cache. I optimized the first above and never mentioned the second.

There is still no tools/get

Worth saying plainly, because the docs make it look otherwise: get_tool_details in the three-layer example is a harness meta-tool, not a protocol method. Nothing in MCP fetches one tool’s schema by name. Layer 2 is you, paging tools/list and filtering locally, which is why the docs also tell you to memoize the definitions host-side, separate from what currently sits in model context.

The same page puts a number on when to bother: switch to progressive discovery once tool definitions take 1% to 5% of the context window. Below that, load everything and stop thinking about it.

Where Alfred landed

Alfred took the strongest option before the doc recommended it. There is one mcp.call tool, so the native tools array never grows with the catalog. The catalog revision travels as a tool argument, in the tail, so a catalog change never edits a schema. The objection that normally kills a single meta-tool is risk: one registered tool cannot carry a static risk tier per remote tool. Alfred resolves the effective tier at the dispatch gate from (connection, remote name, descriptor hash) and fails closed on drift, so the caching win costs nothing in safety.