NVIDIA shows KV caches can move between different-sized models
· 9 min read
A week ago on this blog we called agent memory its own product category: LangMem, Mem0, Zep, all racing to give agents something that remembers past turns. That's memory as recall, deciding what's worth keeping. There's a more literal kind of memory sitting underneath all of it: the notes a model takes while it's reading your prompt, called the KV cache. Normally those notes get thrown away the moment you send that same conversation to a different model. While researching prompt caching for our Token & Cost Management chapter, I found a new paper out of NVIDIA, "Cross-Model KV Cache Transfer in LLM Families" (arXiv:2608.03893), that treats that waste as a problem worth solving instead of just how things are.