Valkey shipping forecast – 9.2 arrives, the CVE tsunami gets a drill, and AI inference bills take a hit

MMS Founder
MMS • RSS
Posted on nosqlgooglealerts. Visit nosqlgooglealerts
image

Valkey 9.2 ships today, and it’s a release about money. Not the project’s – the operator’s. Distributed key-value (KV) caching, out alongside the core update, claims up to 40% lower AI inference costs and time-to-first-token shaved by up to 80%. Replication compression knocks ongoing cross-region traffic down by roughly 50% to 65%. Forkless snapshots end the memory doubling that has traditionally come with in-memory backups. And large sorted sets, rebuilt on a B+ tree, run updates up to 105% faster and use 36-43% less memory once the set hits millions of entries.

Valkey is an open source in-memory datastore governed by the Linux Foundation. It powers caching, session stores, message queues, leaderboards and the latency-sensitive paths users feel the moment something is off. The project emerged as a fork after Redis’s license change, and the ecosystem of contributors around it has grown to include Aiven, Alibaba Cloud, AWS, DigitalOcean, Ericsson, Google Cloud, Momento, NetApp Instaclustr, Oracle, Percona and others. With 9.2, Valkey is showing why it wants to be read as AI infrastructure just as much as a general-purpose datastore.

Speaking at Open Source Summit Europe in Prague a month before the release, Madelyn Olson – Valkey project maintainer and Principal Engineer for AWS in-memory databases – was clear about why standards matter here:

If you ask AIs today – ‘hey, go use a cache’ – we have some data that shows about 15% of them just build their own cache. And that’s even more concerning, because that’s harder to maintain. It’s not very interoperable. So just making sure that the technologies that folks and enterprises and organizations are choosing are standards is really important.

Distributed KV caching – a shared cache across the GPU fleet

The AI inference part of the release ships as a separate open-source capability rather than inside the core datastore, and it’s there to tackle a critical issue seen across agentic applications in production: GPU time wasted reprocessing the same prompt instructions, tool definitions and conversation history on every turn. Each turn reinvents work the last one just did. Distributed KV caching creates a large, shared cache across an entire GPU fleet, so inference engines can reuse that work rather than recomputing it. Teams running open-weight models on self-managed inference connect a distributed KV cache through LMCache, which plugs into engines such as vLLM and SGLang with a single configuration change. LMCache is the first open source inference project involved in the capability.

The claims are a cut in AI inference spend of up to 40% – serving more users on the GPUs teams already have – and a drop in time-to-first-token of up to 80%. Deployment is cloud-agnostic and needs no application code changes. Those are big claims without a production workload behind them, so Ennio Pastore, Director of AI Platform at Multiverse Computing, provided a data point in the release:

At Multiverse Computing, we help customers scale inference workloads with compressed models that cut cost and latency. We cut both even further and served more prompts on existing GPUs by adding distributed KV caching on Valkey. With Valkey, we processed 50% more requests on the same GPUs within our one-second target for median time to first token. We achieved a 23% reduction in our median time to first token at normal traffic, and a 95% reduction at peak traffic. We contributed these improvements to the open-source community so every builder gets the same gains.

Yihua Cheng, CTO and co-founder of Tensormesh, covered the networking side:

KV cache objects are large and latency-sensitive, which is exactly where traditional networking falls short. With RDMA [Remote Direct Memory Access], Valkey can move multi-megabyte KV chunks directly into GPU-serving nodes with minimal CPU overhead. That makes it a strong shared backend for LMCache deployments where many inference engines need to reuse the same context.

Inside the 9.2 feature set

A shared cache is only useful if the inference engine can find the right piece of it, fast. Until now that’s meant teams building and maintaining their own custom lookup code to route a new request to the cached prefix that most closely matches it. Valkey 9.2 ships Path Hash as a new native data type to replace that code. A single command returns the longest matching prefix and the location of its cached results. Large AI data stays in GPU or host memory, with Valkey serving as the fast, lightweight index pointing to it. Path Hash is also a general-purpose building block for matching on any hierarchical data, not just AI prefix routing.

Olson had been previewing the primitive in Prague as a hierarchical hash inspired by LLM KV cache routing – finding the node with the deepest matching prompt prefix, and factoring in load when sending a new request. Path Hash is that primitive, shipped. It was contributed to 9.2 by Alibaba Cloud’s Tair team. Guoyun Zhu, Head of Tair R&D at Alibaba Cloud, explained why they contributed it:

With representation on Valkey’s Technical Steering Committee, Alibaba Cloud is committed to advancing Valkey through open collaboration. We are proud to have contributed Path Hash to Valkey 9.2, bringing native support for hierarchical data and expanding what developers can build with Valkey. Drawing on our Tair team’s research and practical experience in KV cache for AI inference, we see significant opportunities for Valkey in AI infrastructure.

Taking a snapshot of a busy in-memory datastore has traditionally caused memory spikes that can double usage. Snapshot operations ran in a separate process with their own copy of the data, which left operators sizing hardware for a worst-case backup window rather than a normal workload.

Jacob Murphy, Valkey maintainer, walked through why 9.2’s approach is different when he and Olson sat down with me in Prague. Traditional Linux fork-based snapshots cause copy-on-write memory spikes, he said. The new in-database approach runs in a background thread instead, so memory stays flat, and operators can size infrastructure for what the application actually needs. Murphy noted that awareness inside the database lets the project make better trade-offs than leaning on kernel behavior.

Cross-region replication is another line item Valkey 9.2 is trying to shrink. Continuous stream compression cuts ongoing replication traffic by roughly 50% to 65%, with backup files up to 80% smaller using the built-in LZ4 option. New replicas synchronize about 35% faster. The cost is roughly 2% to 4% of a CPU core per replica. What that tackles, in practice, is cloud transfer fees – the place where in-memory datastores can very easily become expensive at scale.

Sorted sets power real-time rankings, gaming leaderboards and priority queues – applications where memory and latency scale with the dataset. Valkey 9.2 rebuilds large sorted sets on a B+ tree, so updates run up to 105% faster and memory efficiency improves by 36-43% on sets with millions of entries.

Governance and vendor-neutrality

Olson set out what the release is really trying to achieve in the announcement:

A major ethos of Valkey is doing more with less, which means getting more functionality with less hardware. We designed Valkey 9.2 around this principle to deliver reduced memory overhead, lower cross-region network costs, and worry-free performance. With each release, Valkey continues to prove its value as an exceptionally fast, efficient, and reliable datastore.

Her position on this hasn’t changed during Valkey’s releases – open, collaborative API design, and critical infrastructure built on open source. Khawaja Shams, CEO of Momento, credited the open governance model, observing:

Valkey has consistently delivered improved availability, performance and efficiency with each release. Valkey 9.2 is no exception, with forkless saves, hot key detection and even more efficient sorted sets. Momento and AWS have worked closely, and in the open, to advance RDMA for Valkey. It’s a great example of the open source ecosystem working! Valkey proves that true open source doesn’t lock innovation behind enterprise-license paywalls.

Security operations in an AI-era project

“Assume bug-finding keeps getting cheaper.” That was one of Olson’s slides at Open Source Summit Europe in October, co-presenting with fellow Valkey maintainer Harkrishn Patro about how open source projects are supposed to cope now that security advisories arrive faster than anyone can triage them. So Valkey turned AI on its own code to find the bugs first – same thinking that drove the 9.2 release features.

Patro described the triage problem:

When everything is a CVE, nothing is. We set clear boundaries to filter what we consider a true CVE. For example, an authenticated client causing high memory usage is not a CVE, but accessing another client’s data is. Issues on the internal, trusted cluster bus port are treated as regular bugs, not CVEs. We shortened the default disclosure embargo from 30 to 14 days to get patches out faster, and formed a dedicated Security Group to scale our response.

Olson walked through the adversarial-testing pipeline Valkey uses to hunt for bugs before external researchers find them:

We use adversarial testing with LLMs. We start with a threat model to define security boundaries. Then, an LLM reviews code, especially where boundaries are touched, to identify potential bugs. Another LLM critically evaluates these findings to reduce false positives. Finally, we generate a reproducible test case – like a crash or memory error – then flag it for human review.

Early runs of the pipeline funneled around 1,200 flagged areas down through review and triage to 34 real, reproducible bugs. One was a use-after-free in C code iterating through pending Transport Layer Security (TLS) connections, exploitable for remote code execution (RCE). The AI surfaced the memory issue; the full RCE implication emerged in human triage, which, Olson stressed, stays the non-negotiable last step.

The very cool thing is the release pipeline sitting on top of all this. Olson and Patro walked through a July 2026 example in their presentation, where a merged fix at 16:03 UTC moved through an automated backport sweep across five supported branches in 72 minutes, with all five releases published by 23:41 that evening and downstream updates completing shortly after midnight. The whole fix-to-shipped-releases window ran under eight hours, with humans reviewing each step.

My take

Here’s the thing that actually matters about Valkey 9.2 – the release’s own copy doesn’t give it enough weight, so I want to do it justice.

In 9.2, Valkey stops trying to store the AI cache data. The data sits on the GPU or in the host memory right next to it, which is where inference already needs it. Valkey is there to tell the inference engine where in that memory the piece it needs is sitting. The split between lookup and storage is the change. If you’ve worked in observability or any other data-heavy space for more than about five minutes, you already know this pattern. The databases that scale stop pretending they own the bytes and admit straight out that they’re really just the lookup. Prometheus did it, ClickHouse did it when it kicked storage out from under the query engine, and Valkey is doing it for the AI inference stack. The primitive it’s shipping – Path Hash, a single command that returns the longest matching prefix plus the location of its cached results – is the whole architecture pattern folded into a data type.

Why that matters right now is because we are watching, in real time, the AI infrastructure stack decide whether to standardize or fragment. Olson’s Prague line about 15% of AI-generated code just rolling its own cache instead of reaching for an existing one is the quiet warning. Every month we go without a common primitive for distributed KV caching is a month where every inference team is writing its own slightly-broken version of the same thing. Valkey 9.2 is pushing for standardization. LMCache first, vLLM and SGLang integrations today, Tair in the Technical Steering Committee – a coalition is forming around one standard. Earlier in November, the US government’s cloud platform, cloud.gov, announced its migration from Redis to Valkey, citing the ability to offer the same in-memory datastore its customers already rely on under an open-source license it can commit to long-term. It’s a good indicator of where this release is finding its audience. Let’s see who else joins it before the end of the year.

Subscribe for MMS Newsletter

By signing up, you will receive updates about our latest information.

  • This field is for validation purposes and should be left unchanged.