In this new age of generative AI (GenAI), NoSQL hybrid databases have new relevancy.

Enterprises need to encode unstructured data as numerical vector embeddings and GenAI applications need to perform similarity searches among those vectors to provide contextual information when sending prompts to large language models (LLMs).

This technique, known as retrieval augmented generation (RAG), has led a number of hybrid database platforms to host vector storage and search workloads. And while “pure play” vector databases are also competing in the market, the hybrid folks are out to prove that they are just as rigorous, while being far more versatile.

DataStax, the vendor behind the open source Cassandra project, one of the first databases on the NoSQL scene, offers both the customer-managed DataStax Enterprise platform and managed cloud service counterpart Astra DB.

On Wednesday, DataStax released to general availability its new Data API, aimed squarely at developers building vector-first RAG/LLM applications.

An API for GenAI

The New Stack spoke with DataStax chief product officer Ed Anuff, who gave us the lowdown not just on the Data API itself, but the specific context and inspiration surrounding it, as well.

Anuff set the stage this way: “As we looked at revving the API, we said, okay, we’ve got a lot of people building RAG applications that are definitely vector-first… [so] we said, can we use this as the basis of… a RAG-first API.”

The result is a new API for both JavaScript and Python developers that, according to DataStax’s press release, “provides all the data and a complete RAG stack for production GenAI apps with high relevancy and low latency.”

Based on the JVector search engine, DataStax says the new API provides up to 20% higher relevancy, 9x higher throughput and up to 74x faster response times than other vector databases. Perhaps the most revolutionary facet of all is that the new API was built to minimize the need for deep Cassandra knowledge in order to use it.

Explaining DataStax’s inspiration to design the API that way, Anuff told The New Stack that, during a period that began in the third quarter of 2023, “close to half of the people that would sign up for Astra DB, our cloud service, were doing so with no previous Cassandra experience and were just looking to build-RAG based applications.”

Anuff added: “For us, it was a wakeup call.”

Other GenAI/RAG Use Tools

Along with the API, DataStax is also launching an updated developer experience for Astra DB, which the company says adds a dashboard, data loading and exploration tools, and integration with leading AI and machine learning frameworks.

DataStax says the new environment simplifies integration with tools like LangChain, OpenAI, Vercel, Google’s Vertex AI, AWS, and Azure. In addition, developers can support RAG techniques, like forward-looking active retrieval (FLARE) and ReAct, that synthesize multiple responses, and do so with minimal latency.

Anuff explained that the new Data API can be also used as a general-purpose document API, despite its primary intent of enabling vector/RAG applications. For non-vector applications, the Cassandra Query Language (CQL) API remains in service.

Conversely, while Astra DB’s inclusion of Pulsar enables non-vector streaming data applications, it can also be used for the ingestion of vector data and the accompanying processing that may be necessary.

Anuff summed it up this way: “We do have a technology stack called RAGStack that…builds on top of LangChain, but also uses LlamaIndex as well as a few other open source packages,” adding that RAGStack “…handles the ingestion problem and Pulsar is one of the mechanisms we use for that.”

NoSQL’s Grown up

There was a time when NoSQL databases seemed like an oddity, with a raison d’etre of simply being unlike the relational databases that preceded them. They were rebels, often without a cause. They also lacked rigor: even if you could get past their lack of declarative query languages and inability to join tables, the absence of real indexing on anything other than a primary key made them more of a developer toy than anything. But NoSQL databases matured.

The query languages were built, and solid secondary indexing was added. And as each type of NoSQL database added elements of the other types, they really came to be hybrid databases for data of varying levels of formal structure.

Now, these hybrid databases provide more than a mere alternative to the relational model. Generative AI and vectorization make unstructured data more usable and more critical to business systems, and hybrid NoSQL databases are the optimal platforms to host and query this data.

Vector, graph and document data models are no longer just an intellectually interesting approach, but instead represent a necessary paradigm shift in database management. Previously dark data is now contextual data, and it’s indispensable in mitigating LLM “hallucinations” and creating better GenAI output overall. That means the databases and APIs needed to work with this data need to become mainstream developer tools.

That’s the wave DataStax is riding here, and a wave more developers can now ride as well.