Month: July 2025

Microsoft has released version 0.29 of DevProxy, a command-line tool for simulating APIs. The new version includes AI support for configuring and using the tool.

DevProxy (formerly known as Microsoft 365 Developer Proxy) helps developers to add resilience to their API-related code by simulating a vast number of API and network behaviours. By default, the tool acts as a proxy that fails half the time. It can also simulate throttling, rate-limiting or slow API responses. It can be used to mock responses to specific APIs. Combined with dev tunnels CLI, a ngrok-like tool, it can also inspect cloud services communication to understand what messages are being passed in cloud calls.

To support DevProxy configuration using natural language, the development team has released a MCP server. It allows AI agents such as GitHub Copilot to automatically correctly configure DevProxy for user’s needs. The MCP is exposed via DevProxy Toolkit Visual Studio Code extension or via a standalone npm package called @devproxy/mcp.

DevProxy internally uses language models (LLMs) to generate OpenAPI or TypeSpec files from API call samples. In this release, the prompts that DevProxy uses are exposed in the prompt folder directory where DevProxy is installed. The prompts are saved in Prompty specification format.

The new version brings breaking changes that will require some adjustments for developers building DevProxy plugins. The new architecture uses a different base class coming from DevProxy.Abstractions assembly, with the standard plugins contained in the DevProxy.Plugins assembly.

For troubleshooting issues with the tool, this version includes categories for debug and trace messages, helping developers quickly identify where the error message is originating.

Lastly, when DevProxy is uninstalled, it automatically uninstalls the root certificate that’s installed for decrypting HTTPS traffic.

An updated DevProxy Toolkit extension for Visual Studio Code editor is released together with the tool.

DevProxy is an open-source project on GitHub and a new member of the .NET Foundation non-profit organisation. The complete release notes for this version are available on the site. There are 26 contributors and 53 open issues at the moment. The project has been starred 676 times.

Perplexity has introduced Comet, a new web browser designed to integrate natural language interaction directly into the browsing experience. Unlike conventional browsers built around navigation and search, Comet aims to support users in research, comparison, and task execution by combining browsing with persistent context and AI assistance.

The browser includes a built-in assistant that remains available across sessions and tabs, allowing users to ask follow-up questions, summarize content, and perform actions such as booking meetings or writing emails. Instead of switching between applications or copying information across tools, Comet supports an interaction model where queries can be posed and refined inline, with the assistant handling tasks in context.

According to Perplexity, Comet is designed to address common friction points in how people interact with information online. Instead of opening dozens of tabs or copying content between tools, users interact with an integrated assistant that can interpret intent, maintain context, and take action.

Some of the browser’s core capabilities include:

• Conversational browsing: Users can ask questions about what they’re reading, compare information across sources, or automate tasks without leaving the page.
• Session memory: The assistant maintains context across tabs, allowing multi-step reasoning and cumulative research.
• Task execution: Basic actions, such as emailing, summarizing, or product comparison, can be triggered by natural language instructions.

Comet is currently available to Perplexity Max subscribers via an invite-based rollout. Broader access is planned over the summer.
Some early adopters have noted concerns with the current distribution model and performance:

Only issue is this invite thing. I have received mine & use it. However, this model of distribution is flawed. I think ARC’s lack of meaningful traction was in part due to this… There isn’t time to be cute. Get it out to as many people who want to try it.

Others report mixed experiences with real-world usage:

I was using it at work for a bit, but I’ve switched back to Chrome/Edge. Comet is slower than both and doesn’t really add any value yet. Plus, my IT department is confused or something because some pages are blocked when I access with Comet, but work in other browsers.

Despite some limitations at launch, Comet reflects a broader trend toward embedding assistant technologies into core productivity tools. Perplexity has stated that future updates will expand capabilities and incorporate user feedback, with the long-term goal of making the browser a more dynamic and contextual interface for interacting with the web.

Spring AI 1.0, a comprehensive solution for AI engineering in Java, is now available after a significant development period influenced by rapid advancements in the AI field. The release includes many essential new features for AI engineers. Here’s a quick rundown of some of the most prominent features. We’ll introduce these concepts throughout this article.

• Portable service abstractions that allow easy, familiar, consistent, and idiomatic access to various chat models, transcription models, embedding models, image models, etc.
• Rich integration with the wider Spring portfolio, including projects such as Micrometer.io, Spring Boot, Spring MVC and GraalVM.
• Support for the day-to-day patterns of AI engineering, including Retrieval Augmented Generation (RAG) and tool calling that informs an AI model about tools in its environment.
• Support for the MCP that allows Spring AI to be used to build and integrate MCP services.

As usual, you can get the bits when generating an application on Spring Initializr.

Spring AI is your one-stop-shop for AI engineering

Java and Spring are in a prime spot to jump on this AI wave. Many companies are running their applications on Spring Boot, which makes it easy to plug AI into what they’re already doing. You can basically link up your business logic and data right to those AI models without too much effort.

Spring AI provides support for various AI models and technologies. Image models can generate images given text prompts. Transcription models can take audio and convert them to text. Embedding models can convert arbitrary data into vectors, which are data types optimized for semantic similarity search. Chat models should be familiar! You’ve no doubt even had a brief conversation with one somewhere. Chat models are where most of the fanfare seems to be in the AI space. You can get them to help you correct a document or write a poem (just don’t ask them to tell a joke). They are very useful, but they have some issues.

Let’s go through some of these problems and their solutions in Spring AI. Chat models are open-minded and given to distraction. You need to give them a system prompt to govern their overall shape and structure. AI models don’t have memory. It’s up to you to help them correlate one message from a given user to another by giving them memory. AI models live in isolated little sandboxes, but they can do really amazing things if you give them access to tools, functions that they can invoke when they deem it necessary. Spring AI supports tool calling, letting you tell the AI model about tools in its environment, which it can then ask you to invoke. This multi-turn interaction is all handled transparently.

AI models are smart but they’re not omniscient! They neither know what’s in your proprietary databases, nor would want them to! Therefore, you need to inform their responses by stuffing the prompts, basically using the string concatenation operator to put text in the request that the model considers before it looks at the question being asked (background information, if you like).

You can stuff it with a lot of data, but not infinite amounts. How do you decide what should be sent and what should not? Use a vector store to select only the relevant data and send it in onward. This technique is called Retrieval Augmented Generation (RAG).

AI chat models like to, well, chat! And sometimes they do so so confidently that they can make stuff up, so you need to use evaluation, using one model to validate the output of another, to confirm reasonable results.

And, of course, no AI application is an island. Today, modern AI systems and services work best when integrated with other systems and services. MCP can connect your AI applications with other MCP-based services, regardless of what language they’re written in. You can assemble and compose MCP services in agentic workflows to drive towards a larger goal.

And you can do all this while building on the familiar idioms and abstractions that any Spring Boot developer has come to expect: convenient starter dependencies for basically everything available on the Spring Initializr. Spring AI provides convenient Spring Boot autoconfiguration that gives you the convention-over-configuration setup you’ve come to know and expect. And Spring AI supports observability with Spring Boot’s Actuator and the Micrometer project. It plays well with GraalVM and virtual threads, too, allowing you to build fast and efficient AI applications that scale.

Meet the Dogs

To demonstrate some of the cool possibilities in action, we’re going to build a sample application. I always struggle with deciding upon a proper domain for these things. Let’s use one that’s relatable: we’re going to build a fictitious dog adoption agency called Pooch Palace. It’s like a shelter where you can find and adopt dogs online! Just like most shelters, people will want to talk to somebody and interview them about the dogs. We will build an assistant service to facilitate that.

We’ve got a bunch of dogs in the SQL database that will install when the application starts. We aim to build an assistant to help us find the dog of somebody’s dream, Prancer, described rather hilariously as “Demonic, neurotic, man-hating, animal-hating, children-hating dogs that look like gremlins”. This dog is awesome. You might’ve heard about him. He went viral a few years ago when his owner was looking to find a new home for him. The ad was hysterical, and it went viral! Here’s the original ad post, in Buzzfeed News, in USA Today, and in The New York Times.

We’re going to build a simple HTTP endpoint that will use the Spring AI integration with a Large Language Model (LLM). In this case, we’ll use Open AI though you can use anything you’d like, including Ollama, Amazon Bedrock, Google Gemini, HuggingFace, and scores of others – all supported through Spring AI – to ask the AI model to help us find the dog of our dreams by analyzing our question and deciding after looking at the dogs in the shelter (and in our database), which might be the best match for us.

The Build

Using Spring Initializr, let’s generate a new project. I’m going to be using the latest version of Spring Boot. Choose Open AI (the 1.0 or later release). I’ll be using a vector store. In this case, it is the PostgreSQL-powered vector store called PgVector. Let’s also add the Spring Boot Actuator module for observability. Add the Web support. Add support for SQL-based ORM-like data mapping using Spring Data JDBC. You should also choose Docker Compose. I’m using Apache Maven, too. I’ll use Java 24 in this article, but you should use the latest reasonable version. If Java 25 is out when you read this, then use that! I’m also using GraalVM native images. So, make sure to add GraalVM and the dependencies.

Many different options exist here, including Weaviate, Qdrant, ChromaDB, Elastic, Oracle, SQLServer, MySQL, Redis, MongoDB, etc. Indeed, Spring AI even ships with a SimpleVectorStore class, a vector store implementation that you can use. However, I wouldn’t use this implementation in production since it’s just in-memory and not very good at retaining data. But it’s a start.

We’ll need to support the aforementioned RAG pattern, so add the following dependency to the pom.xml:

    org.springframework.ai
    spring-ai-advisors-vector-store

Are you using GraalVM? No? Well, you should be! If you’re on macOS or Linux or Windows Subsystem for Linux, you should manage your JVM-based infrastructure using the amazing SDKMAN.io project. Once installed, installing GraalVM is as simple as:

$ sdk install java 24-graalce

Mind you, I’m using the latest version of Java as of this writing. But, you do you! Always remember to use the newest version of the runtime and technology. As my late father used to say, “It’s a cinch by the inch, hard by the yard”. If you’re not using freedom units, somebody might translate it as “It’s easy by the centimeter, difficult by the meter”. If you do things as they come, they don’t accrue into an insurmountable technical debt.

Click Generate, unzip the resulting .zip, and then open pom.xml in your favorite IDE.

We added Docker Compose support, so you’ll have a compose.yml file in the folder. Open it up and make sure to change the port export line, where it says the following:

ports: 
    - '5432'

Change it to be:

ports:
    - '5432:5432'

This way, when you start the PostgreSQL Docker image, you can connect to it from your external clients. This is handy for development, as you can see what’s being done.

The Configuration

First, Spring AI is a multimodal AI system that allows you to work with many different models. In this case, we’ll be interacting principally with a ChatModel, OpenAI. It’s perfectly possible and reasonable to use any number of dozens of different supported models like Amazon Bedrock, Google Gemini, Azure OpenAI, and even local models such as those enabled via Docker’s Model Runner and Ollama. Running your model is non-negotiable if you’re in a privacy-sensitive domain, like a bank or much of Europe. There are options for you, too! If no explicit support works for you, use the OpenAI API and swap out the base URL gains you purchase with new models, many of which implement the OpenAI API.

To connect to the model, we’ll need to specify an OpenAI key. Add the following to application.properties:

spring.ai.openai.api-key=

If you don’t have an OpenAI API key, you can get one from the OpenAI developer console.

Spring Boot has support for running Docker images on startup. It will automatically run the compose.yml file in the root of the folder. Unfortunately, PostgreSQL isn’t serverless, so we only want Spring Boot to start the image if it’s not running. Add this to application.properties:

spring.docker.compose.lifecycle-management=start_only

Spring Boot will automatically connect to the Docker image on startup. But this is only for development. Later, we’re going to build a GraalVM native image which is a production build of the code. Now let’s specify some spring.datasource.* properties so that the application can connect to our local SQL database running in the Docker daemon.

spring.datasource.password=secret
spring.datasource.username=myuser
spring.datasource.url=jdbc:postgresql://localhost/mydatabase

Again, you won’t need this for now, but you will eventually want to connect your application to a SQL database. Now you know how. Also note, that in a production environment, you wouldn’t hard code your configuration. Set up environment variables, e.g., SPRING_DATASOURCE_USERNAME.

When Spring Boot starts up, it launches the SQL database. We also want it to install some data into this database, so we have two files: data.sql and schema.sql. Spring Boot will automatically run schema.sql and then data.sql if we ask.

spring.sql.init.mode=always

Now we’re ready to go!

Show Me the Code!

The code is the easy part. We’ll build a simple HTTP controller that will field inquiries – to interview, in effect – the shelter about the dogs in the shelter.

package com.example.assistant;

import io.modelcontextprotocol.client.McpClient;
import io.modelcontextprotocol.client.McpSyncClient;
import io.modelcontextprotocol.client.transport.HttpClientSseClientTransport;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.PromptChatMemoryAdvisor;
import org.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;
import org.springframework.ai.chat.memory.InMemoryChatMemoryRepository;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;
import org.springframework.ai.document.Document;
import org.springframework.ai.mcp.SyncMcpToolCallbackProvider;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.data.annotation.Id;
import org.springframework.data.repository.ListCrudRepository;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

@SpringBootApplication
public class AssistantApplication {

    public static void main(String[] args) {
        SpringApplication.run(AssistantApplication.class, args);
    }
}

Any new Spring Boot project will already have something that looks like that class automatically generated for you. In this application, the generated class is called AssistantApplication.java.

We need data access functionality to connect our application to the underlying SQL database, so let’s dispense with that right now. Add the following underneath the main class:

interface DogRepository extends ListCrudRepository { }

record Dog(@Id int id, String name, String owner, String description) { }

That’s our data access layer. Now, let’s move on to the actual controller. Add the following controller underneath everything else.

@Controller
@ResponseBody
class AssistantController {

    private final ChatClient ai;

    AssistantController(ChatClient.Builder ai,
                        McpSyncClient mcpSyncClient,
                        DogRepository repository, VectorStore vectorStore
    ) {
        var system = """
                You are an AI powered assistant to help people adopt a dog from the adoption
                agency named Pooch Palace with locations in Antwerp, Seoul, Tokyo, Singapore, Paris,
                Mumbai, New Delhi, Barcelona, San Francisco, and London. Information about the dogs 
                available will be presented below. If there is no information, then return a polite response 
                suggesting we don't have any dogs available.
                """;
        this.ai = ai
                .defaultSystem(system)
                .build();
    }


    @GetMapping("/{user}/assistant")
    String inquire (@PathVariable String user, @RequestParam String question) {
        return this.ai
                .prompt()
                .user(question)
                .call()
                .content();
    }
}

We’ve already got a few critical pieces here. In the constructor, we inject the ChatClient.Builder and use that builder to configure a default system prompt. In the inquire() method, we accept inbound requests and then forward those requests from clients as a String to the underlying model.

Try it out. I’m using the handy HTTPie command line tool.

$ http :8080/jlong/assistant question=="my name is Josh"

It should respond with something confirming it understands what you’ve just said. But did it?

$ http :8080/jlong/assistant question=="what's my name?"

This time, it should respond by basically saying that it has already forgotten you. (Or maybe it’s just me. I seem to have that effect on people!)

Anyway, the way to fix that is to configure an advisor, a Spring AI concept that allows you to pre- and post-process requests bound for the model. Let’s configure an instance of the PromptChatMemoryAdvisor class, which will keep track of everything sent to the model and then re-transmit it to subsequent requests. We’ll use the user path variable passed in the URL as the key by which we distinguish each transcript. After all, we wouldn’t like some random person to get your chat transcript! We’re going to use an in-memory implementation, but you could just as easily use a JDBC-backed implementation.

Let’s create a concurrent map to store the multi-tenant requests. So, add this to the class.

    //...    
    private final Map memory = new ConcurrentHashMap();
    //...

Let’s configure the advisor and then pass it into the ChatClient at the call site. Here’s the updated method.

@GetMapping("/{user}/assistant")
String assistant(@PathVariable String user, @RequestParam String question) {

    var inMemoryChatMemoryRepository = new InMemoryChatMemoryRepository();
    var chatMemory = MessageWindowChatMemory
            .builder()
            .chatMemoryRepository(inMemoryChatMemoryRepository)
            .build();
    var advisor = PromptChatMemoryAdvisor
            .builder(chatMemory)
            .build();
    var advisorForUser = this.memory.computeIfAbsent(user, k -> advisor);

    return this.ai
            .prompt()
            .user(question)
            .advisors(advisorForUser)
            .call()
            .content();
}

Relaunch the program and try the two requests above again. It should remember you!

But, it still doesn’t know about the dogs! Prove it by issuing a more specific request.

$ http :8080/jlong/assistant question==" do you have any neurotic dogs?"

Remember, we want to find Prancer, the “demonic, neurotic, man-hating, animal-hating, children-hating dog that looks like a gremlin”.

We will need to integrate the data from the SQL database and send that along with the request, but not all of the data. There is no need. Instead, let’s use a vector store to find only the most germane data for our query. Recall that we were using Spring AI’s implementation of the vector type in a table designed by Spring AI. Let’s set up a table in PostgreSQL first.

spring.ai.vectorstore.pgvector.initialize-schema=true

Now modify the constructor to look like this:

    AssistantController(ChatClient.Builder ai,
                        DogRepository repository, VectorStore vectorStore
    ) {
        // be prepared to comment out from here...
        repository.findAll().forEach(dog -> {
            var dogument = new Document("id: %s, name: %s, description: %s".formatted(dog.id(), dog.name(), dog.description()));
            vectorStore.add(List.of(dogument));
        });
        // to here

        // ..as before... 
        this.ai = ai
            // as before
            .defaultAdvisors(new QuestionAnswerAdvisor(vectorStore))
            // as before ...
            .build();
    }

Now, when the application starts up, we’ll read all the data from the SQL database and write it to a VectorStore. Then, we configure a new advisor that will know to query the VectorStore for relevant results before embedding the results in the request to the model for final analysis.

Relaunch the program and try the last http call again.

It works!

Now, comment out the code above that initializes the VectorStore as it does us no good to initialize the vector store twice!

We’re making good progress, but we’re not nearly done! We may have found Prancer, but now what? Any red-blooded human being would leap at the chance to adopt this doggo! I know I would. Let’s modify the program to give our model access to tools to help schedule a time when we might pick up or adopt Prancer. Add the following class to the bottom of the code page.

@Component
class DogAdoptionScheduler {

    @Tool(description = "schedule an appointment to pickup or "+ 
                        "adopt a dog from a Pooch Palace location")
    String scheduleAdoption(
            @ToolParam(description = "the id of the dog") int dogId,
            @ToolParam(description = "the name of the dog") String dogName) {
        System.out.println("scheduleAdoption: " + dogId + " " + dogName + "".);
        return Instant
                .now()
                .plus(3, ChronoUnit.DAYS)
                .toString();
    }
}

The implementation returns a date three days later and prints out a message. Modify the constructor to be aware of this new tool: inject the DogAdoptionScheduler and then pass it into the defaultTools() method defined in ChatClient.Builder. Restart the program.

$ http :8080/jlong/assistant question==" do you have any neurotic dogs?"

It should return that there’s a neurotic dog named Prancer. Now, let’s get it to help us adopt the dog.

$ http :8080/jlong/assistant question== "fantastic. When can I schedule an  appointment to pick up or adopt Prancer from the San Francisco Pooch Palace location?"

You should see that it’s worked. (Neat, right?)

Now we’ve integrated our model and business logic with the AI models. We could stop here! After all, what else is there? Well, quite a bit. I’d like to take the tool calling support here just a few more steps forward by introducing the Model Context Protocol (MCP).

Anthropic introduced MCP in November 2024. It’s a protocol for models (in that case, Claude via the Claude Desktop application) to interoperate with tools worldwide. The Spring AI team jumped on the opportunity and built a Java implementation that eventually became the official Java SDK on the MCP website. The Spring AI team then rebased their integration on that. Let’s see it in action. We will first extract the scheduler into a separate MCP service (called scheduler), then connect our assistant.

Using Spring Initializr, name it scheduler, select Web and MCP Server, and hit Generate. Open the project in your IDE.

Cut and paste the DogAdoptionScheduler class from above and paste it at the bottom of the code page of the newly minted scheduler codebase. Ensure the service doesn’t start on the same port as the assistant; add the following to application.properties:

server.port=8081

We will also need to register a ToolCallbackProvider, which tells Spring AI which beans to export as MCP services. Here’s the entirety of the code for our new scheduler application:

package com.example.scheduler;

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;
import org.springframework.ai.tool.method.MethodToolCallbackProvider;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;

import java.time.Instant;
import java.time.temporal.ChronoUnit;

@SpringBootApplication
public class SchedulerApplication {

    public static void main(String[] args) {
        SpringApplication.run(SchedulerApplication.class, args);
    }

    @Bean
    MethodToolCallbackProvider methodToolCallbackProvider(DogAdoptionScheduler scheduler) {
        return MethodToolCallbackProvider
                .builder()
                .toolObjects(scheduler)
                .build();
    }
}

@Component
class DogAdoptionScheduler {

    @Tool(description = "schedule an appointment to pickup or"+
                        " adopt a dog from a Pooch Palace location")
    String scheduleAdoption(@ToolParam(description = "the id of the dog") int dogId,
                            @ToolParam(description = "the name of the dog") String dogName) {
        System.out.println("scheduleAdoption: " + dogId + " " + dogName + "".);
        return Instant
                .now()
                .plus(3, ChronoUnit.DAYS)
                .toString();
    }
}

Launch this service. Then return to the assistant. Delete references to DogAdoptionScheduler in the code – the constructor, its definition, the configuration on defaultTools, etc. Define a new bean of type McpSyncClient in the main class:

@Bean
    McpSyncClient mcpSyncClient() {
        var mcp = McpClient
                    .sync(HttpClientSseClientTransport
                            .builder("http://localhost:8081")
                            .build())
                    .build();
        mcp.initialize();
        return mcp;
    }

Inject that reference into the constructor and then change it to this:

 AssistantController(ChatClient.Builder ai, 
                        McpSyncClient mcpSyncClient, 
                        DogRepository repository, 
                        VectorStore vectorStore
    ) {
        // like before... 
        this.ai = ai
                // ...
                .defaultToolCallbacks(new SyncMcpToolCallbackProvider(mcpSyncClient))
                .build();
    }

Launch the program and inquire about neurotic dogs and at what time you might pick the dog up for adoption. You should see that this time, the tool is invoked in the scheduler module.

Production Worthy AI

The code is complete; now, it’s time to turn our focus toward production.

Security

It’s trivial to use Spring Security to lock down this web application. You could use the authenticated Principal.getName() as the conversation ID, too. But, what about the data stored in the SQL database, like the conversations? Well, you have a few options here. Most SQL databases have transparent data encryption. As you read or write values, it’s stored securely on disk. No changes are required to the code for this.

Scalability

We want this code to be scalable. Remember, each time you make an HTTP request to an LLM (or many SQL databases), you block IO, which seems to be a waste of a perfectly good thread! Threads should not simply sit idle and waiting. Java 21 gives us virtual threads, which can dramatically improve scalability for sufficiently IO-bound services. That’s why we set up spring.threads.virtual.enabled=true in the application.properties file.

GraalVM Native Images

GraalVM CE is an Ahead-of-Time (AOT) compiler that produces architecture- and operating system-specific native binaries. If you’ve got that setup as your SDK, you can turn this Spring AI application into a native image with ease:

$ ./mvnw -DskipTests -Pnative native:compile

This takes a minute or so on most machines, but once it’s done, you can easily run the binary.

$ ./target/assistant

This program will start up in a fraction of the time it did on the JVM. It starts up in less than a tenth of a second on my machine. The application takes a fraction of the RAM it would otherwise have taken on the JVM. That’s all very well and good, you might say, but I need to get this running on my cloud platform (CloudFoundry or Kubernetes, of course), and that means making it into a Docker image. Easy!

$ ./mvnw -DskipTests -Pnative spring-boot:build-image

Stand back. This might take another minute. When it finishes, you’ll see the name of the generated Docker image printed out. You can run it, remembering to override the hosts and ports of things it would’ve referenced on your host.

$ docker run -e SPRING_DATASOURCE_URL=jdbc:postgresql://docker.host.internal/mydatabase 
  docker.io/library/assistant:0.0.1-SNAPSHOT

Vroom!

We’re on macOS, and amazingly, when run in a virtual machine emulating Linux, this application runs even faster than – and right from the jump, too! – it would’ve if it were run on macOS directly! Amazing.

Observability

This application is so darn good that I bet it’ll make headlines, just like Prancer, in no time. And when that happens, you’d be well advised to keep an eye on your system resources and – importantly – the token count. All requests to an LLM have a cost, at least one of complexity if not dollars and cents. Thankfully, Spring AI has your back. Launch a few requests to the model, and then up the Spring Boot Actuator metrics endpoint powered by Micrometer: http://localhost:8080/actuator/metrics, and you’ll see some metrics related to token consumption. Nice! You can use Micrometer to forward those metrics to your time-series database to get a single pane of glass, a dashboard.

Conclusion/Wrapping up

AI has dramatically reshaped how we build software, unlocking new opportunities to make our applications more interactive, more usable, more powerful, and increasingly agentic. But take heart, Java and Spring developers: you don’t need to switch to Python to be part of this revolution.

At its core, AI integration often comes down to talking to HTTP endpoints, something at which Java and Spring have always excelled. Integration is our wheelhouse. Beyond that, Java and Spring are proven platforms for building production-grade software. With Spring, you get robust support for observability, security, lightning-fast performance with GraalVM, and scalability with virtual threads, everything you need to run real-world systems under real-world load. 

Most enterprises already run their mission-critical business logic on the JVM. The software that powers the world is already written in Java and Spring. And with Spring AI, it’s not just about adding AI, it’s about adding production-ready AI to systems built to last.

Resources

Mistral has released Voxtral, a large language model aimed at speech recognition (ASR) applications that seek to integrate more advanced LLM-based capabilities and go beyond simple transcription. For two variants of the model, Voxtral Mini (3B) and Voxtral Small (24B), Mistral has released the weights under the Apache 2.0 license.

According to Mistral, Voxtral closes a gap between classic ASR systems, which delivers cost-efficient transcription but lack semantic understanding, and more advanced LLM-based models, which provide transcription and language understanding. While this is similar to what other solutions like GPT-4o mini Transcribe, Gemini 2.5 Flash, and others provide, Voxtral stands out by making its model weights openly available, improving deployment flexibility and enabling a different cost model.

Besides being available for local deployment, the new models can be accessed via Mistral’s API, which also offers a custom version of Voxtral Mini optimized for transcription, helping reduce inference cost and latency.

Voxtral has a 32K token context, which enables it to process audios up to 30 minutes for transcription, or 40 minutes for understanding. Being LLM-based means it naturally lends itself to tasks like Q&A and summarization based on audio content without requiring to chain an ASR system with a language model. Additionally, it enables executing backend functions, workflows, or API calls based on spoken user intents. As usual for Mistral models, Voxtral is natively multilingual and supports automatic language detection with optimized performance for European languages. It goes without saying that Voxtral retains the text-only capabilities of its base model and can be used as a text-only LLM.

Speaking of transcription-only use cases, Mistral claims both cost and performance advantages over other solutions like OpenAI Whisper, ElevenLabs Scribe, and Gemini 2.5 Flash.

Voxtral comprehensively outperforms Whisper large-v3, the current leading open-source Speech Transcription model. It beats GPT-4o mini Transcribe and Gemini 2.5 Flash across all tasks, and achieves state-of-the-art results on English short-form and Mozilla Common Voice, surpassing ElevenLabs Scribe and demonstrating its strong multilingual capabilities.

When it comes to audio understanding, Voxtral can answer questions directly from speech thanks to its LLM foundation. This is a distinct approach compared to other LLM-based speech recognition models’. For instance, NVIDIA NeMo Canary-Qwen-2.5B and IBM’s Granite Speech have two distinct modes, ASR and LLM, that can be combined at different stages, such as using the LLM to summarize the textual output generated by the ASR step.

According to Mistral’s own benchmarking, Voxtral Small is competitive with GPT-4o-mini and Gemini 2.5 Flash across several tasks, and outperforms both in speech translation.

Besides offering Voxtral for download for local deployment or use via the API, Mistral also supports additional features specifically aimed at enterprise customers, including support for private deployment at production-scale, domain-specific fine-tuning, and advanced use cases such as speaker identification, emotion detection, diarization and others.

As AI adoption in Asia-Pacific moves from pilot projects to production, enterprise data systems are under pressure to adapt. Traditional stacks that are built by stitching together separate vector databases, search tools, and inference engines, often break down at scale, especially in multilingual and multi-region environments.

These fragmented setups add latency, duplicate data, and increase operational overhead. To solve this, CIOs are turning to composable AI architectures, i.e., modular stacks that integrate search, storage, and inference without sacrificing scalability.

A key design question now emerging: Should vector search sit inside the transactional database or live in a dedicated system?

MongoDB’s vice president and field CTO, Boris Bialek, told iTnews Asia that many teams are getting this balance wrong.

“Problems start when you try to run high-speed transactional workloads and vector search in the same system,” Bialek said. “Every time a new transaction happens, it updates the vector index too, and that slows everything down.”

AI architectures must not break in production

What works in a demo often breaks under real-world load. In multilingual, multi-region environments like APAC, rushed architectural choices quickly expose limits.

A common misstep is embedding vector search directly into the transactional database, said Bialek.

While this keeps everything in one place, it often leads to performance degradation.

“Many so-called ‘native’ vector features are just blobs (binary large objects) behind the scenes. When high-speed transactions run alongside compute-heavy vector queries, both slow down,” said Bialek.

In response, teams start splitting systems, duplicating data, and syncing changes through Kafka or ETL pipelines.

“It becomes what I call ‘management by Nike’ – everyone’s running between systems trying to keep them in sync. What started as a simple idea ends up as a fragmented setup that’s hard to scale,” he added.

Another alternative of adding a separate vector database, can also backfire.

It introduces glue code, near-real-time sync jobs, and risks of stale or inconsistent data.

Once you start duplicating vectors and managing sync jobs, you’ve lost the simplicity you were aiming for.

– Boris Bialek, VP and Field CTO, MongoDB

Instead, Bialek recommends a composable architecture, where modular systems are natively integrated into a unified stack.

In MongoDB’s case, that includes an operational database, a dedicated vector search layer, and built-in text search, coordinated internally, without external pipelines or duplication.

Such architecture eliminates friction and allows the engineering teams to build reliable, production-ready AI systems.

However, as CIOs modernise AI stacks, many still face strategic concerns, particularly around over-consolidation and the risk of vendor lock-in.

Avoid lock-in through openness and flexibility

Talking on the concern, Boris Bialek suggests reframing the discussion, not as risk management, but as a question of flexibility and long-term value.

“It’s not about being locked in or out, it’s about being able to adapt as needs evolve,” said Bialek.

Modern data architecture built on open standards, such as the JSON document model, allows organisations to move components in or out as needed.

In MongoDB’s case, the use of non-proprietary formats and interoperable components means teams can integrate open-source tools, extract modules, or migrate workloads without being tightly bound to a single vendor ecosystem.

This openness is essential as enterprises now expect not just functionality, but continuous innovation, operational simplicity, and scalable systems without added complexity.

However, meeting expectations isn’t just about architecture in theory; it’s about how systems perform under real-world conditions.

Lessons from real-world AI deployments

In multilingual, multi-regulatory environments like Southeast Asia, India or Europe, the ability to localise data, models, and inference workflows becomes essential.

Bialek mentions that ASEAN and India are similar to Europe in terms of cultural attitudes, different app usage patterns, and infrastructure challenges.

MongoDB’s document model supports type stability, applies schema where needed, and maintains consistent behaviour across languages.

This flexibility enables enterprises to build multilingual, domain-specific applications without adding operational burden.

Bialek said two factors that are critical in these environments include scalability and deployment flexibility.

“A major retail group based in Bangkok, for example, runs sharded clusters across Singapore, Kuala Lumpur, Jakarta, and Bangkok. Each region handles local writes and enforces data sovereignty, while the system maintains a unified customer view,” said Bialek.

This setup lets the business recognise a customer across countries, including Thailand and Malaysia, without disrupting service.

In India, banks deploy across Mumbai, Bangalore, and Hyderabad to support local writes and global reads. Even if one region goes offline, MongoDB’s architecture keeps operations running; no custom routing or failover tools are required.

Bialek mentions that non-functional requirements like high availability, encryption, key rotation, and vector scalability become critical.

These capabilities often get overlooked but are essential for long-term performance, compliance, and enterprise trust.

As enterprises scale AI beyond pilots, foundational capabilities like scalability and security become essential for delivering production-ready systems that meet both technical and business needs.

What production-ready AI requires

In ASEAN and similar regions, many organisations still experiment with AI, often prompted by boardroom directives to adopt a formal strategy.

Bialek said there is a growing transition toward structured, business-led implementations.

AI adoption today aligns closely with tangible business goals, like logistics optimisation, personalised customer experiences, and operational efficiency.

Business and technical leaders now work together, moving AI from exploratory phases into real-world production.

Despite such successes, Bialek mentions a major bottleneck: moving from prototype to production, as promising AI projects falter due to the absence of scalable infrastructure.

He emphasises the importance of AI-specific CI/CD pipelines that ensure data traceability, compliance, and governance, elements that are often overlooked in early-stage experimentation.

As full-stack RAG deployments begin to enter production across the region, Bialek sees signs of growing enterprise maturity.

However, he cautions that long-term success requires strong delivery pipelines and tight alignment between business priorities and technical execution.

Understand your priorities before rethinking the AI stack

As enterprises scale AI, the need for real-time context to reduce LLM hallucinations, especially in critical use cases like fraud detection, is essential, Bialek said.

Embedding live metadata such as payer, payee, and location helps ground model outputs in accurate, actionable data.

An effective AI stack should support hybrid search, combining vector and text search within a unified system.

Bialek says MongoDB’s integration with Voyage AI delivers real-time embeddings and retrieval without relying on external pipelines or complex system sprawl.

To future-proof AI architecture, enterprises need to prioritise real-time processing, unified data access, and simplified infrastructure.

They should avoid siloed systems and adopt composable platforms that strike a balance between flexibility and performance.

“AWS (US), Microsoft (US), IBM (US), Google (US), Oracle (US), Alibaba Cloud (China), Tencent Cloud (China), Twilio (US), Cloudflare (US), MongoDB (US), Netlify (US), Fastly (US), Akamai (US), Digitalocean (US), Datadog (US), Vercel (US), Spot by NetApp (US), Elastic (US).”

Serverless Computing Market by Service Model (Function as a Service, Backend as a Service), Compute (Functions, Containers), Database (Relational, Non-relational), Storage, Application Integration, Monitoring & Security – Global Forecast to 2029.

The serverless computing market is anticipated to expand from USD 21.9 billion in 2024 to USD 44.7 billion in 2029 at a Compound Annual Growth Rate (CAGR) of 15.3%. Because it increases project visibility, optimizes resource utilization, and facilitates better decision-making, the serverless computing sector is thriving globally. By using a single platform, businesses can efficiently manage and oversee several multinational projects, ensuring consistent performance and successful operations across multiple domains. By managing complex project portfolios more skillfully, this approach helps firms to remain competitive, adapt to changes in the market, and develop efficiently.

Download PDF Brochure@https://www.marketsandmarkets.com/pdfdownloadNew.asp?id=217021547

“As per service model, the backend-as-a-service (BaaS) will grow at the highest CAGR during the forecast period.”

Backend-as-a-service (BaaS) is essential in the serverless computing sector, making backend tasks easier through different managed services. BaaS provides fundamental functionalities like file storage and management, user authentication and management, database management, and push notification delivery. By outsourcing these backend duties to external firms, developers can focus on front-end development and application logic, leading to faster development and decreased operating expenses. This method facilitates smoothly incorporating different backend services, allowing for scalable and effective app development. With organizations looking to improve their development processes, BaaS offers a way to increase flexibility and streamline backend operations in serverless computing.

“As per vertical, IT & Telecom holds the largest share during the forecast period.”

The IT & telecom industry leads the way in the serverless computing market, utilizing its features to promote innovation and productivity. Serverless computing helps telecom companies improve operations by reducing the need to manage infrastructure and scaling as needed to accommodate changing demands. This vertical sees major advantages from serverless architectures in network function virtualization (NFV), real-time data processing, and content delivery networks (CDNs). Implementing serverless models enables IT & telecom providers to improve service delivery, decrease latency, and optimize resource usage. By incorporating serverless computing into their operations, these businesses can quickly implement new capabilities, uphold high availability, and meet the rising need for digital services, all while cutting operational costs and complexity to stay ahead in a fast-changing tech environment.

“As per region, Asia Pacific will grow at the highest CAGR during the forecast period.”

The rapid growth of the serverless computing market in the Asia-Pacific region is due to the quick uptake of cloud technologies, digital transformation projects, and strong tech communities in nations such as China, India, and Japan. This area, with its different economies and levels of technology, is starting to realize more and more the advantages of serverless computing for scalability, cost saving, and speeding up innovation. Effective and economical cloud options are vital in industries like e-commerce, finance, telecommunications, and manufacturing, where serverless designs are improving functions and service provision. However, in addition to the opportunity that challenges such as adhering regulations or where data needs to (or does not need to) reside may provide, there are also level differences of cloud infrastructure readiness across the region. Companies which provide clouds on a massive scale, AWS, Microsoft Azure and Alibaba Cloud are spending up large in Asia at the moment building data centers right across every country making sure they meet all of these key regulations. Furthermore, governments in the area are pushing for digitization and positive regulatory environments driving interest towards cloud technology as well. Continued digital infrastructure investments and growing adoption of cloud-native technologies in enterprises in Asia-Pacific are expected to drive strong growth in the market for serverless computing, particularly among companies in the digital economy sector.

Request Sample Pages@https://www.marketsandmarkets.com/requestsampleNew.asp?id=217021547

Unique Features in the Serverless Computing Market

One of the most defining features of serverless computing is its event-driven architecture. Applications are executed in response to events, such as HTTP requests, file uploads, or database changes. This design, combined with automatic scaling, enables serverless platforms to allocate computing resources dynamically based on demand, eliminating the need for manual provisioning and ensuring cost-efficiency and performance optimization.

Serverless computing abstracts server infrastructure, meaning developers don’t need to manage or maintain servers. This reduces operational overhead significantly and allows organizations to focus more on application development and innovation rather than infrastructure management. Updates, patching, and provisioning are handled by cloud providers, enhancing productivity.

A unique financial advantage of serverless is its micro-billing model—users are billed only for the exact execution time and resources consumed by functions, rather than for pre-allocated instances or idle time. This pay-per-use model is ideal for unpredictable or variable workloads, offering cost savings, especially for startups and small enterprises.

Serverless supports rapid application development through features like pre-built backend services, integrations, and reusable functions. This accelerates prototyping, testing, and deployment, allowing businesses to reduce time-to-market. Developers can independently deploy components, aligning well with DevOps and agile methodologies.

Major Highlights of the Serverless Computing Market

The serverless computing market is experiencing rapid growth as developers and enterprises increasingly embrace its simplicity, scalability, and efficiency. The model’s ability to eliminate server management and accelerate development cycles makes it a top choice for modern cloud-native applications. This demand is further propelled by the need for faster time-to-market and digital transformation initiatives across industries.

Major cloud vendors such as Amazon Web Services (AWS Lambda), Microsoft Azure (Azure Functions), Google Cloud (Cloud Functions), and IBM Cloud Functions are significantly investing in serverless platforms. Their continuous feature enhancements, global infrastructure support, and enterprise-grade service offerings are driving widespread adoption and trust in serverless technologies.

Serverless computing is being adopted in various sectors including finance, healthcare, retail, media, and IoT. From automating back-end processes and handling real-time data analytics to supporting AI/ML workloads and chatbot development, serverless is proving to be versatile and impactful across multiple domains.

With the rise of edge computing and hybrid cloud models, serverless frameworks are evolving to support execution closer to the data source. This enables low-latency applications and compliance with data residency regulations. Hybrid serverless platforms allow organizations to run functions both in the cloud and on-premises, ensuring greater flexibility and control.

Inquire Before Buying@https://www.marketsandmarkets.com/Enquiry_Before_BuyingNew.asp?id=217021547

Top Companies in the Serverless Computing Market

Some of the key players operating in the Serverless Computing Market are – AWS (US), Microsoft (US), IBM (US), Google (US), Oracle (US), Alibaba Cloud (China), Tencent Cloud (China), Twilio (US), Cloudflare (US), MongoDB (US), Netlify (US), Fastly (US), Akamai (US), Digitalocean (US), Datadog (US), Vercel (US), Spot by NetApp (US), Elastic (US), VMware (US), Backendless (US), Faundb (US), Scaleway (US), 8Base (US), Supabase (US), Appwrite (US).

Amazon Web Services (AWS)(US)

Amazon Web Services, known as AWS. In addition to the traditional services like compute, storage, and databases, it provides more advanced resources, including options for Artificial Intelligence (AI), Machine Learning suites with AWS, or Internet of Things solutions using their lambda-services. AWS serves enterprises at all stages with hyper-scalable, elastic solutions that run across a genuinely worldwide data center-based availability zone to reduce latency and increase reliability. Competitive product pricing, an expanded group of services offered, and a less disruptive ability to incorporate new features make AWS stand out in the cloud market due to its top-notch security, compliance with all regulations, and comprehensive customer support. On the list of serverless computing brands is AWS, which has a lauded selection of tools developers can use to run apps without managing servers. AWS Lambda is a serverless event-driven compute service; AWS Fargate lets you run containers without managing servers or clusters (like ECS by default); Amazon API Gateway enables creation, publishing, and securing of Restful APIs with ease; Step Functions from AWS are used for indeed executing workflows in the cloud ecosystem & DynamoDB which can be called as an enterprise-grade NoSQL database available to us. These tools help companies concentrate on building applications, scale quickly, and reduce costs by managing the underlying infrastructure to accelerate product launches.

Microsoft (US)

Microsoft Corporation was founded in 1975 and is a global software, services, devices & solutions provider with products that span the Desktop to Cloud: Windows operating system (OS), Office suite of productivity apps SharePoint servers or CRM Dynamics, Xbox gaming platform; Azure for hosting web-based workloads alongside LinkedIn an online network connecting professionals. Microsoft is a global technology leader that creates productivity and communication services for consumers, SMBs, and enterprise dashboards in sectors spanning AI to mixed reality to cybersecurity innovations. Azure Functions -Event-driven serverless computing and Azure Logic Apps for automated workflow; event-based notification solutions are core in Microsoft’s approach to flat-out solving workloads on top of their dynamic virtual machines. Developers can develop and launch apps with no infrastructure to manage, providing flexibility, scalability, and connection to other Azure services. Their cloud ecosystem, ease of use, and enterprise-level support by Microsoft make Azure an attractive option for businesses looking to deploy their serverless computing with many applications.

Alibaba Cloud (China)

Alibaba Cloud (China) provides serverless computing solutions that enable developers to build and deploy applications without managing server infrastructure. Their Function Compute service allows businesses to scale automatically, improve cost-efficiency, and accelerate time-to-market by only paying for the resources used during execution.

Oracle (US)

Oracle (US) provides serverless computing solutions through its Oracle Cloud Infrastructure, allowing businesses to run applications without managing servers. Their serverless platform automatically scales and manages resources based on demand, enabling faster development cycles and reducing operational complexity for developers.

Google (US)

Google (US) offers serverless computing solutions through its Google Cloud Platform, enabling developers to build and run applications without managing the underlying infrastructure. Google’s services, such as Google Cloud Functions and Firebase, provide scalable, event-driven execution, allowing businesses to focus on coding while the platform handles resource provisioning and scaling.

Media Contact
Company Name:MarketsandMarkets™ Research Private Ltd.
Contact Person: Mr. Rohan Salgarkar
Email:Send Email
Phone: 18886006441
Address:1615 South Congress Ave. Suite 103, Delray Beach, FL 33445
City: Florida
State: Florida
Country: United States
Website:https://www.marketsandmarkets.com/Market-Reports/serverless-computing-market-217021547.html

Press Release Distributed by ABNewswire.com
To view the original version on ABNewswire visit: Serverless Computing Market 2029 New Trends, Size, Share, Drivers, Latest Opportunities, Growth, and Future Outlook

MongoDB is experiencing annual revenue growth of over 20%, but investors are concerned about rising costs. The company’s stock price is impacted by these concerns.

MongoDB, a leading provider of cloud-based database services, has recently reported annual revenue growth of over 20%, showcasing its ability to maintain high growth rates at a substantial scale. However, investors have expressed concern over rising costs, which have impacted the company’s stock price. This article examines MongoDB’s financial performance and the factors contributing to its growth and cost challenges.

Revenue Growth and Scalability

MongoDB’s Q1 2026 results demonstrated a 22% year-over-year (YoY) growth in revenue, reaching $549 million [1]. This growth is particularly notable given that the company has scaled to a $2 billion annual recurring revenue (ARR) level. The key drivers of this growth include:

1. Product Expansion: Atlas, MongoDB’s cloud-based database service, now represents 72% of the company’s revenue, up from 71% in the previous quarter. This growth is driven by an increase in cloud adoption and the integration of AI workloads [1].
2. Customer Base Expansion: MongoDB added 2,600 net new customers in Q1 2026, the highest number in six years. This expansion is facilitated by a self-serve motion for mid-market customers and a developer-first approach [1].
3. Market Category Expansion: MongoDB is successfully penetrating new market categories, particularly in AI workloads, which are driving new use cases and adoption [1].

Operational Efficiency and Profitability

Despite the strong revenue growth, investors have been concerned about rising costs. MongoDB has managed to maintain profitability through various operational efficiencies:

1. Operational Margins: MongoDB reported a 16% non-GAAP operating margin in Q1 2026, up from negative margins in previous years. This improvement is attributed to disciplined hiring during the 2022-2023 downturn, revenue scaling, and the shift to high-margin products like Atlas [1].
2. AI Automation: MongoDB has implemented AI automation to reduce operational overhead, contributing to improved unit economics and higher margins [1].
3. Share Buybacks: MongoDB announced an additional $800 million share repurchase authorization, bringing the total to $1 billion. This move signals the company’s confidence in its cash generation capabilities and its commitment to returning value to shareholders [1].

Investor Concerns and Future Outlook

While MongoDB’s financial performance is impressive, investors remain concerned about rising costs and the potential impact on revenue stability. The company’s recent acquisition of Voyage AI and the subsequent release of Voyage 3.5 have strengthened its AI capabilities and enterprise data protection partnerships [2]. However, these advancements are not expected to materially impact the most pressing short-term risks, such as headwinds from a declining non-Atlas business and seasonality in Atlas consumption [2].

MongoDB’s narrative projects $3.3 billion in revenue and $205.5 million in earnings by 2028, requiring a 15.6% yearly revenue growth and a $291.6 million earnings increase from -$86.1 million currently [2]. Despite these projections, the company faces ongoing risks around a declining non-Atlas business and the need to maintain steady Atlas consumption growth to sustain overall growth targets.

Conclusion

MongoDB’s Q1 2026 results highlight the company’s ability to maintain high revenue growth rates at a massive scale. However, investors’ concerns about rising costs and the potential impact on revenue stability remain. As MongoDB continues to expand its product offerings and customer base, it will be crucial for the company to manage operational efficiencies and maintain profitability to address these concerns and support long-term growth.

References

[1] https://www.saastr.com/mongodb-at-2b-arr-5-epic-learnings-from-q1-2026-that-every-b2b-leader-should-study/

[2] https://simplywall.st/stocks/us/software/nasdaq-mdb/mongodb/news/why-mongodb-mdb-is-up-92-after-voyage-ai-acquisition-and-new