Posted on nosqlgooglealerts. Visit nosqlgooglealerts

The database market is undergoing significant changes, driven by increasing demands for scale, resilience, and the burgeoning era of AI agents. 

Speaking exclusively to AIM, CockroachDB CEO Spencer Kimball stated that the shift towards distributed SQL databases built on a solid PostgreSQL foundation is becoming increasingly crucial for businesses of all sizes, not just tech giants.

The core difference offered by CockroachDB lies in its horizontal scaling capabilities. While it strives to maintain a PostgreSQL-like interface, distributed operations require a different approach. 

“Cockroach didn’t reject Postgres. It re-architected it from the ground up to meet the scale, distribution, and the consistency AI demands,” Kimball said.

He further added that scaling 100x on a monolithic architecture is utterly impossible. This, he explained, is where distributed SQL databases like CockroachDB come in, built for “serious scale, like hundreds of terabytes into petabytes” of operational data. “Postgres may be eating the world, but AI needs a database that can digest.”

Kimball said that he is particularly referring to operational databases and not the analytical ones. “It’s about the metadata that tracks the product or service, all the activity, and the high level of concurrent operations that demand strong consistency,” he added.

He explained that both humans and agents would have access to the data. These agents operate at high speed and are continuously active, performing the same tasks multiple times daily or even hourly. They work on behalf of both consumers and businesses, resulting in a steadily increasing volume of traffic.

What’s Next from CockRoach

Kimball sees AI playing a role in observability and support. “AI can move much faster. If you give it the right scenarios and train it, then what could have taken several hours to fix might only take several minutes,” he said.

Vector indexing is another area of focus for CockroachDB. “Customers want nearest-neighbour search in high-dimensional spaces at scale. They want it fast and consistent, even as data changes,” Kimball said.

But he clarified that CockroachDB is not trying to become a general-purpose vector database. Cockroach isn’t trying to compete with OpenSearch, Elastic, or MongoDB on vector search. “If you’re already using CockroachDB for mission-critical relational workloads, you want vector support there. Not everyone needs that, but for our users, it’s essential.”

He further added that they are not trying to win the market for the vector index. “We’re not a vector database. However, it’s a very important modality.”

Moreover, Kimball talked about reducing costs. “Nobody wants to pay 10x more because their workload scales 10x. CockroachDB can improve utilisation with multi-tenancy.” He explained that if a customer has 100 use cases on a large cluster, the peaks and troughs average out, allowing them to move from 10% to 50-60% utilisation.

The company is also working on using cloud cost efficiencies. Kimball said CockroachDB’s architecture allows the use of spot instances, disaggregated storage, and storage tiering. “We believe we can reduce costs by 10 to 16x in the next few years.”

Moat of Cockroach

Kimball said that CockroachDB’s strength is in geographic scale. “We have customers in the EU, the US, and India. If you want to make your service span all of those places, Cockroach has some really interesting capabilities that are different.”

He provided one example from the US sports betting sector. “Customers use Cockroach nodes in multiple states to comply with data locality laws. Data is processed where bets are placed.”

Moreover, he added that CockroachDB is cloud-agnostic and supports hybrid deployments. “Big banks and tech companies use private data centres and all three major clouds. We let customers run the database wherever their business needs it.”

One key challenge, he pointed out, is integrating AI into database operations. “It’s not easy to run distributed systems. When something goes wrong, you want the root cause before a human even looks at it. AI can help.”

On competing with cloud vendors, he noted, “They’re both competitors and partners. Big clouds don’t want to serve self-hosted enterprise customers, and those customers don’t want to be tied to one cloud. CockroachDB fits well there.”

He added that clouds often refer such customers to CockroachDB. “They say, ‘We can’t run this in your data centre, but CockroachDB can.’ That’s why the partnership works.”

As the era of AI agents increases data scale and complexity, CockroachDB is positioning itself to meet those demands through distributed design, cross-cloud flexibility, and AI-enhanced tooling.

Why Postgres 

Kimball explained how CockroachDB tries to stay close to the Postgres experience but adapts key behaviours to function at scale in distributed environments.“So well, it tries to look as much like Postgres as possible.”

One clear example was ID generation. Traditional Postgres allows for monotonically increasing sequences, such as auto-incrementing IDs for user records. In monolithic systems, this works smoothly, but things break down at a massive scale.

“In a monolithic system… that counter, it’s all just in one place… But once you say, I want to do 10 million of these concurrently… you don’t want them all going to one node that holds a counter.”

CockroachDB distributes the sequence generation process differently, making it scale-friendly but less linear. “It will look the same as a sequence. But… we have a more distributed mechanism to assign IDs… they’re not just counting 1,2,3,4,5.”

He acknowledged differences between Postgres and MySQL users as well. “Postgres does structured data, too. There’s room for both. 

Kimball said that the bigger challenge lies in how the databases are operated, not how they are used by applications. He said that system administrators and DBAs familiar with one will have a steeper learning curve when switching to the other, due to differences in tools, management styles, and best practices. 

“If you’re very good as a system administrator or like a DBA using Postgres, then it’s a lot more new stuff to learn. 

Kimball said that it often comes down to what teams are already used to operating. “If you’re good at MySQL, moving to distributed MySQL, then TiDB makes sense.” He was referring to TiDB CTO Ed Huang, who said that he believes MySQL will power AI agents.

Journey of the Cockroach

Cockroach Labs was founded in 2015 by ex-Google employees Kimball, Peter Mattis, and Ben Darnell. It draws inspiration from Google’s Bigtable and Spanner databases.

Kimball said that in the early 2000s, systems like Google’s Bigtable avoided SQL not out of dislike, but to keep things simple while focusing on scalability. “It was just easier not to have to do all that stuff and also build something that is elastically scalable and more survivable.”

However, over time, the industry began adding SQL features again. MongoDB added transactions. Google layered SQL on top of Spanner with F1. 

“They created a whole new distributed architecture, but they left all of the hard stuff and started adding the hard stuff back on top of it,” said Kimball. 

He added that NoSQL systems, such as Cassandra, offer flexibility and scalability but fall short in terms of consistency and schema management. “If you have 50 people working on a complex, mission-critical product… it just becomes impossible.”

By 2015 the CockroachDB team had a clear understanding of their target users which included big banks, major tech firms and other high-stakes organisations. 

Instead of building a new SQL dialect, they chose PostgreSQL. “Postgres felt like the cleanest and the most appropriate, and had the most upward velocity momentum.”