Author: Sergio De Simone

In a recent tech report, Apple has provided more details on the performance and characteristics of the new Apple Intelligence Foundation Models that will be part of iOS 26, as announced at the latest WWDC 2025.

Apple foundation models include a 3B-parameter version optimized to run on Apple Silicon-powered devices, as well as a larger model designed to run on Apple’s Private Cloud Compute platform. Apple emphasizes that both models were trained using responsible web crawling, licensed corpora, and synthetic data. A further training stage included supervised fine-tuning and reinforcement learning.

According to Apple, the 3B parameter model is designed for efficiency, low-latency, and minimal resource usage. The larger model, by contrast, aims to deliver high accuracy and scalability. Apple notes that, given its reduced size, the on-device model isn’t intended to implement a world-knowledge chat, but can support advanced capabilities such as text extraction, summarization, image understanding, and reasoning with just a few lines of code.

On the architecture side, the 3B-parameter model uses KV-cache sharing, a technique used to reduce the time-to-first-token, and is compressed using 2-bit quantization-aware training. Sharing the key-value caches between the two blocks the model is divided into enables a reduction of memory usage by 37.5%, says Apple. Quantization-aware training is a technique that allows to recover quality by simulating the effect of 2-bit quantization at training-time:

Unlike the conventional quantization scheme which derives the scale from weights W, we introduce a learnable scaling factor f that adaptively fine-tunes the quantization range for each weight tensor.

For the server-side model, Apple used a novel Parallel-Track Mixture-of-Experts (PT-MoE) transformer that combines track parallelism, sparse computation, and interleaved global–local attention. comprises multiple transformers that process tokens independently, each with its own set of MoE layers. Apple says that the combination of parallel token processing with the MoE approach delivers reduced synchronization overhead and allows the model to scale more efficiently.

To evaluate its foundation models, Apple researchers relied on human graders to assess each model’s ability to produce a native-sounding response. The results show that the on-device model performs well against Qwen-2.5-3B across all supported languages, and remains competitive with larger models like Qwen-3-4B and Gemma-3-4B in English. The larger server-side model performs favorably against Llama-4-Scout, but falls short compared to much larger models such as Qwen-3-235B and GPT-4o.

For image understanding, Apple followed the same approach by asking humans to evaluate image-question pairs, including text-rich images like infographics:

We found that Apple’s on-device model performs favorably against the larger InternVL and Qwen and competitively against Gemma, and our server model outperforms Qwen-2.5-VL, at less than half the inference FLOPS, but is behind Llama-4-Scout and GPT–4o.

As a final note, Apple researchers emphasizes their approach to Responsible AI, which includes enforcing a baseline of safety and guardrails to mitigate harmful model input and output. These safeguards were also evaluated through a combination of human assessment and auto-grading. Apple has also published educational resources for developers to apply Responsible AI principles.

As mentioned, Apple’s AI foundation models require XCode 26 and iOS 26 and are currently available as beta software.

Mistral has released Voxtral, a large language model aimed at speech recognition (ASR) applications that seek to integrate more advanced LLM-based capabilities and go beyond simple transcription. For two variants of the model, Voxtral Mini (3B) and Voxtral Small (24B), Mistral has released the weights under the Apache 2.0 license.

According to Mistral, Voxtral closes a gap between classic ASR systems, which delivers cost-efficient transcription but lack semantic understanding, and more advanced LLM-based models, which provide transcription and language understanding. While this is similar to what other solutions like GPT-4o mini Transcribe, Gemini 2.5 Flash, and others provide, Voxtral stands out by making its model weights openly available, improving deployment flexibility and enabling a different cost model.

Besides being available for local deployment, the new models can be accessed via Mistral’s API, which also offers a custom version of Voxtral Mini optimized for transcription, helping reduce inference cost and latency.

Voxtral has a 32K token context, which enables it to process audios up to 30 minutes for transcription, or 40 minutes for understanding. Being LLM-based means it naturally lends itself to tasks like Q&A and summarization based on audio content without requiring to chain an ASR system with a language model. Additionally, it enables executing backend functions, workflows, or API calls based on spoken user intents. As usual for Mistral models, Voxtral is natively multilingual and supports automatic language detection with optimized performance for European languages. It goes without saying that Voxtral retains the text-only capabilities of its base model and can be used as a text-only LLM.

Speaking of transcription-only use cases, Mistral claims both cost and performance advantages over other solutions like OpenAI Whisper, ElevenLabs Scribe, and Gemini 2.5 Flash.

Voxtral comprehensively outperforms Whisper large-v3, the current leading open-source Speech Transcription model. It beats GPT-4o mini Transcribe and Gemini 2.5 Flash across all tasks, and achieves state-of-the-art results on English short-form and Mozilla Common Voice, surpassing ElevenLabs Scribe and demonstrating its strong multilingual capabilities.

When it comes to audio understanding, Voxtral can answer questions directly from speech thanks to its LLM foundation. This is a distinct approach compared to other LLM-based speech recognition models’. For instance, NVIDIA NeMo Canary-Qwen-2.5B and IBM’s Granite Speech have two distinct modes, ASR and LLM, that can be combined at different stages, such as using the LLM to summarize the textual output generated by the ASR step.

According to Mistral’s own benchmarking, Voxtral Small is competitive with GPT-4o-mini and Gemini 2.5 Flash across several tasks, and outperforms both in speech translation.

Besides offering Voxtral for download for local deployment or use via the API, Mistral also supports additional features specifically aimed at enterprise customers, including support for private deployment at production-scale, domain-specific fine-tuning, and advanced use cases such as speaker identification, emotion detection, diarization and others.

Docker launched a new feature to let developers define, build, and run agents using Docker Compose, with the aim of streamlining the agent development process and reducing repetitive tasks. Additionally, Docker Offload, now in beta, provides a way to seamlessly offload building and running models to remote GPU compute.

Adding support for defining agents using Docker Compose is a further step in Docker’s strategy to position itself as a key tool provider for agent development, much like it did for container-based development. As Docker’s Mark Cavage and Tushar Jain note, this means simplifying repetitive and tedious tasks that agent development typically involves, such as iterating with different models, securely connecting to MCP tools, and packaging everything so teammates can easily run the same workflow.

The new feature allows developers to declare open models, agents, and MCP tools in a compose.yaml file, then build and run them using docker compose up. Docker Compose integrates with many current agentic frameworks, including LangGraph, Embabel, Vercel AI, Spring AI, CrewAI, Google ADK, and Agno.

To help developers get started with using Docker Compose for agent development, Docker has created a GitHub repository with sample projects for all supported frameworks. For instance, one example shows how to build a collaborative multi-agent fact checker using Google ADK:

The Critic agent gathers evidence via live internet searches using DuckDuckGo through the Model Context Protocol (MCP), while the Reviser agent analyzes and refines the conclusion using internal reasoning alone. The system showcases how agents with distinct roles and tools can collaborate under orchestration.

The corresponding compose.yaml file defines two services, adk and mcp-gateway, and includes a models section listing used models along with their arguments. Docker introduced the possibility to package and run local models with Model Runner in Docker Desktop 4.40, but you can also use remote or cloud-based models by providing the appropriate credentials.

Docker Compose modular architecture makes it easy to create compose overrides for multiple agent configurations. For example, you can define a variant that uses OpenAI instead of a local model, or one that targets Google Cloud Run. This lets you combine multiple compose files to easily switch your agent setup:

docker compose -f compose.yaml -f compose.openai.yaml up --build

Another new feature for agent development supported in the latest Docker Desktop is Docker Offload. This fully managed service can be used as a drop-in replacement for Docker Model Runner when local resources are not sufficient, allowing developers to run models and containers on a cloud GPU transparently using the same workflow as for local deployment.

Docker Offload frees you from infrastructure constraints by offloading compute-intensive workloads, like large language models and multi-agent orchestration, to high-performance cloud environments. No complex setup, no GPU shortages, no configuration headaches.

The service is currently available in beta, and Docker is providing 300 minutes of free usage to help developers get started.

Available in the Armv9-A architecture, Arm Scalable Matrix Extension 2 (SME2) is a set of advanced CPU instructions designed to accelerate matrix heavy computation. The new Arm technology aims to help mobile developers to run advanced AI models directly on CPU with improved performance and efficiency, without requiring any changes to their apps.

SME2 builds on the previously available SME extension, which introduced matrix operations and streaming vectors, by adding acceleration and support for multi-vector data-processing instructions, load to and store from multi-vectors, and a multi-vector predication mechanism.

While the performance benefits of SME2 are already available on the latest iOS devices and Apple M4-series chips, they will soon reach Android devices as well, says Alex Spinelli, Arm’s VP of AI and Developer Platforms and Services.

Matrix workflows are key for real-time mobile inference tasks such as image and language processing and voice generation. In particular, comparisons between SME2-enabled and non-SME2-enabled workflows shows a significant improvement, says Arm:

On SME2-enabled hardware, Google’s Gemma 3 model delivers 6x faster chat responses, and can start summarizing up to 800 words in under a second on a single CPU core.

Likewise, a 2.6x speed up has been measured for prompt processing on a vivo X200 Pro flagship smartphone running a 3.8B parameter Phi-3 Mini model.

To help developers take advantage of SME2, Arm provides a library called KleidiAI, which is integrated in Google’s XNNPACK. XNNPACK powers several machine learning and AI frameworks, including Alibaba’s MNN, Google’s LiteRT, Microsoft’s ONNX Runtime, and llama.cpp.

When SME2 is enabled and compatible, XNNPACK automatically routes the matrix heavy operations to SME2 via KleidiAI, so developers directly benefit with no changes needed in application logic or infrastructure.

KleidiAI is designed to be integrated easily into C and C++ codebases thanks to its micro-kernel based architecture.

A micro-kernel, in Arm’s parlance, refers to the “near-minimum amount of software to accelerate a given ML operator with high performance”, such as for example, packing or matrix multiplication. A key detail to explain why a micro-kernel is not simply a function, is that each micro-kernel processes only a portion of the output tensor, enabling the full operation to be dispatched across multiple threads.

In addition, KleidiAI has other features that will be welcome to developers, including it not relying on external dependencies, not using dynamic memory or requiring memory management, and a highly modular design where each micro-kernel is a stand-alone library consisting only of .c and .h files.

To help developers take advantage of SME2, Arm has released additional resources showcasing real-world examples of LLM-based apps using LiteRT, MNN, PyTorch and other supported frameworks.

Following the introduction of Model Runner a few months ago, Docker Desktop 4.43 expands its capabilities with improved model management and broader OpenAI compatibility. The release also debuts a new Compose Bridge to simplify the generation of Kubernetes configurations and upgrade the Gordon AI agent.

Docker Model Runner in 4.43 introduces a new user interface for inspecting models through model cards. These cards summarize all available variants within a model family, detailing their features such as number of parameters, quantization, format, size, and architecture.

For developers preferring to work from the command line, the docker model command now supports inspecting, monitoring, and unloading models. At the Docker Compose level, developers can now specify the context size to use for a given model as well as the llama.cpp runtime flags. Furthermore, Model Runner adds support several OpenAI API options, including tool support using {“stream”: “true”} and improved compatibility and security with custom CORS configuration.

Docker Desktop 4.43 also upgrades the Gordon AI agent, adding support for multi-threaded conversations and delivering a 5x performance improvement.

You can now run multiple distinct conversations in parallel and switch between topics like debugging a container issue in one thread and refining a Docker Compose setup in another, without losing context.

Compose Bridge is a new feature that enables converting compose.yaml files to Kubernetes configurations using a single command:

docker compose bridge convert

This innovation automatically generates comprehensive Kubernetes resources, ensuring that local development environments can be quickly and accurately mirrored in production-like Kubernetes clusters.

Compose Bridge is able to automatically create namespaces, configuration maps, deployments, services, secrets, network policies, and persistent volumes based on Compose file declarations. Developers can adjust how Compose Bridge creates Kubernetes resources by customizing a set of template files. To this aim, you can either export the template files used by the default transformation and modify them or build your own templates for resources not managed by the default transformation.

The compose.yaml model may not offer all the configuration attributes required to populate the target manifest. If this is the case, you can then rely on Compose custom extensions to better describe the application, and offer an agnostic transformation

For example, this lets developers add the x-virtual-host metadata to a Compose file and define how it should be translated into Kubernetes configuration by setting a custom ingress attribute in a custom template file. Using custom template files requires re-packaging the Docker image used by Compose Bridge.

As a final note on Docker Desktop 4.43, the MCP Toolkit now supports OAuth and offers improved integration with GitHub and Visual Studio Code.

In a recent paper, Stanford researchers Mason Kamb and Surya Ganguli proposed a mechanism that could underlie the creativity of diffusion models. The mathematical model they developed suggests that this creativity is a deterministic consequence of how those models use the denoising process to generate images.

In rough terms, diffusion models are trained to sort of uncover an image from an isotropic Gaussian noise distribution that is the outcome of the training process from a finite set of sample images. This process consists of gradually removing the Gaussian noise by learning a scoring function that points in gradient directions of increasing probability.

If the network can learn this ideal score function exactly, then they will implement a perfect reversal of the forward process. This, in turn, will only be able to turn Gaussian noise into memorized training examples.

This means that, to generate new images that are far from the training set, the models must fail to learn the ideal score (IS) function. One way to explain how this occurs is by hypothesizing the presence of inductive biases that may provide a more exact account of what diffusion models are actually doing when creatively generating new samples.

By analyzing how diffusion models estimate the score function using CNNs, the researchers identify two such biases: translational equivariance and locality. Translational equivariance refers to the model’s tendency to reflect shifts in the input image, meaning that if the input is shifted by a few pixels, the generated image will mirror that shift. Locality, on the other hand, arises from the convolutional neural networks (CNNs) used to learn the score function, which only consider a small neighborhood of input pixels rather than the entire image.

Based on these insights, the researchers built a mathematical model aimed at optimizing a score function for equivariance and locality, which they called an equivariant local score (ELS) machine.

An ELS machine is a set of equations that can calculate the composition of denoised images and compared its output with that of diffusion models such as ResNets and UNets trained on simplified models. What they found was “a remarkable and uniform quantitative agreement between the CNN outputs and ELS machine outputs”, with an accuracy of around 90% or higher depending on the acutal diffusion model and dataset considered.

To our knowledge, this is the first time an analytic theory has explained the creative outputs of a trained deep neural network-based generative model to this level of accuracy. Importantly, the (E)LS machine explains all trained outputs far better than the IS machine.

According to Ganguli, their research explains how diffusion model create new images “by mixing and matching different local training set image patches at different locations in the new output, yielding a local patch mosaic model of creativity”. The theory also helps explain why diffusion models make mistakes, for example generating excess fingers or limbs, due to excessive locality.

This result, while compelling, initially excluded diffusion models that incorporate highly non-local self-attention (SA) layers, which violate the locality assumption in the researchers’ hypothesis. To address this, the authors used their ELS machine to predict the output of a publicly available UNet+SA model pretrained on CIFAR-10 and found that it still achieved significantly higher accuracy than the baseline IS machine.

According to the researchers, their results suggest that locality and equivariance are sufficient to explain the creativity of convolution-only diffusion models and could form the foundation for further study of more complex diffusion models.

The researchers also shared the code they used to train the diffusion models they used in the study.

Launched in early preview last May, Gemma 3n is now officially available. It targets mobile-first, on-device AI applications, using new techniques designed to increase efficiency and improve performance, such as per-layer embeddings and transformer nesting.

Gemma 3n uses Per-Layer Embeddings (PLE) to reduce the RAM required to run a model while maintaining the same number of total parameters. The technique consists of loading only the core transformer weights into accelerated memory, typically VRAM, while the rest of the parameters are kept on the CPU. Specifically, the 5-billion-parameter variant of the model only requires 2 billion parameters to be loaded into the accelerator; for the 8-billion variant, it’s 4 billion.

Another novel technique is MatFormer (short for Matryoshka Transformer), which allows transformers to be nested so that a larger model, e.g. with 4B parameters, contains a smaller version of itself, e.g. with only 2B parameters. This approach enables what Google calls elastic inference and allows developers to choose either the full model or its faster but fully-functional sub-model. MatFormer also support a Mix-n-Match method to let developers create intermediate-sizes versions:

This technique allows you to precisely slice the E4B model’s parameters, primarily by adjusting the feed forward network hidden dimension per layer (from 8192 to 16384) and selectively skipping some layers.

In the future, Gemma 3n will fully support elastic inference, enabling dynamic switching between the full model and the sub-model on the fly, depending on the current task and device load.

Another new feature in Gemma 3n aimed at accelerating inference is KV cache sharing, which is designed to accelerate time-to-first-token, a key metric for streaming response applications. Using this technique, which according to Google is particularly efficient with long contexts:

The keys and values of the middle layer from local and global attention are directly shared with all the top layers, delivering a notable 2x improvement on prefill performance compared to Gemma 3 4B.

Gemma 3n also brings native multimodal capabilities, thanks to its audio and video encoders. On the audio front, it enables on-device automatic speech recognition and speech translation.

The encoder generates a token for every 160ms of audio (about 6 tokens per second), which are then integrated as input to the language model, providing a granular representation of the sound context.

Google says they have observed strong results translating between English and Spanish, French, Italian, and Portuguese. While Gemma 3n audio encoder can process arbitrarily long audios thanks to its streaming architecture, it will initially be limited to clips of up to 30 seconds at launch.

As a final note about Gemma 3n, it is worth highlighting that it supports resolutions of 256×256, 512×512, and 768×768 pixels and can process up to 60 frames per second on a Google Pixel device. In comparison with Gemma 3, it delivers a 13x speedup with quantization (6.5x without) and has a memory footprint that is four times smaller.

Recently open-sourced by Google, the Agent2Agent protocol is now part of the Linux Foundation, along with its accompanying SDKs and developer tools.

The Agent2Agent protocol will be the cornerstone of a wider Agent2Agent project formed by Google, AWS, Cisco, Microsoft, and others. The project aims to foster interoperability for AI agents and break down the silos that are limiting collaboration between them, says the company.

By providing a common language for AI agents to discover each other’s capabilities, securely exchange information, and coordinate complex tasks, the A2A protocol is paving the way for a new era of more powerful, collaborative, and innovative AI applications.

Using the Agent2Agent protocol, agents can discover each other’s capabilities, negotiate how to interact, and collaborate securely on long-running tasks. The protocol is particularly focused on preserving each agent’s internal state, including its prompt.

The protocol is based on JSON-RPC 2.0 over HTTP and uses server-sent events for real-time streaming between agents. Agents know about each other through “agent cards” that describe agent capabilities and provide connection info. In the future, agent cards will include also authorization schemes and optional credentials. Other areas of future development include client-initiated interactions and dynamic UX negotiation within tasks, such as adding audio/video formats after the initial negotiation phase, i.e. after the agents have started their conversation.

According to Google, the Agent2Agent protocol has seen significant adoption, with over 100 companies supporting it. Since its original announcement, the protocol has raised some controversy due to its overlap with Anthropic’s Model Context Protocol (MCP).

Reddit commenter Impressive-Owl3830 expressed concern that this overlap might prevent the two protocols from coexisting, with MCP already having “taken off”. Another redditor, Specialist_Apricot74, noted this announcement “puts to rest the threat of the triple E threat (Embrace, Extend, Extinguish)” and could help Agent2Agent to differentiate itself from MCP by reducing its overlap and specializing in at least one task that MCP cannot do.

Google says Agent2Agent is ideal when agents are developed and deployed independently, come from different teams, require dynamic discovery and composition, and need to support third-party integration or frequent changes, such as adding or removing agents at any time.

If you are interested in Agent2Agent, a great starting point is Google’s unofficial Python Notebook, which illustrates how you can set up a system with three agents, one searching the web for current trending topics, another performing deep analysis, and the last orchestrating the first two to provide insights.

Google has announced the integration of Gemini in Android Studio’s Agent Mode into the latest canary release of Android Studio, Android Studio Narwhal preview. According to Google, the new Agent Mode is designed to handle multi-step development tasks that span across several files.

Agent Mode takes Gemini integration in Android Studio a step forward by going beyond what was previously possible through simple chat interactions. Specifically, Agent Mode uses the entire project as context, not just the contents of the chat window. Its deeper integration with the IDE also allows it to directly modify the project, rather than merely suggesting code snippets to copy and paste. Most importantly, it can execute multi-step tasks.

With Agent Mode, you can describe a complex goal in natural language — from generating unit tests to complex refactors — and the agent formulates an execution plan that can span multiple files in your project and executes under your direction.

Examples of tasks you can ask Agent Mode to perform include building a project and fixing all errors, extracting hardcoded strings and migrating them to strings.xml, adding support for dark mode to an existing application, and more.

The agent carries out requested tasks step by step, allowing developers to review the changes. If they’re not satisfied, they can provide feedback and ask the agent to provide a new solution based on it until the result meets their expectations.

Agent Mode can interact with external tools via the Model Context Protocol (MCP), for example to create a pull request directly from Android Studio, or use any of the MCP servers currently available. The initial implementation of MCP support in this preview is partial, as only the stdio transport is available, with support for Streamable HTTP transport planned for a future release, along with external context resources, and prompt templates.

Agent Mode can be used with the Gemini’s free tier, which supports a limited context window though. Alternatively, you can use a one million tokens window by upgrading to Gemini 2.5 Pro.

Google highlights that Agent Mode is especially useful for routine, time-consuming tasks, helping free up developers’ time for more creative work. This could be seen as a subtle way to set expectation right about what this tool can bring to a developer’s workflow, or a hint at the most appropriate way to use it day to day.

As this is still a preview release, it’s expected that not all features will work flawlessly in every scenario. Early adopters of Gemini in Android Studio’s Agent Mode have reported several shortcomings, including issues seemingly related to the tool’s inability to run external tools to access source files or modify them. Google is aware of these issues and is actively working to address them.

The next major release of C++ reached an important milestone earlier this month, when the ISO C++ committee froze the feature set that will go into C++26. Notable additions include compile-time reflection, contracts, asynchronous execution, and many others.

Static reflection enables compile-time introspection on types and behavior, aiming to extend C++ metaprogramming capabilities. At a basic level, reflection allows to write code similar to the following to convert an enum to a string:

enum Color { red, green, blue };
static_assert(enum_to_string(Color::red) == "red");

While the user-facing syntax for using reflection is simple, in keeping with modern C++ philosophy, the implementation of the feature shown above is significantly more complex. Still it offers a glimpse of the capabilities reflection will unlock:

template 
    requires std::is_enum_v
constexpr std::string enum_to_string(E value) {
    template for (constexpr auto e : std::meta::members_of(^E)) {
        if (value == [:e:]) {
            return std::string(std::meta::name_of(e));
        }
    }
    return "";
}

Reflection will debut in C++ in an early form and is expected to evolve in future versions. Even so, it will already be powerful enough to enable advanced use cases, such as generating bindings for other languages like JavaScript and Python. This is how Herb Sutter describes the feature in its latest Trip Report:

Even with the first partial reflection capability we have today, we will already be able to reflect on C++ types and use that information plus plain old std::cout to generate arbitrary additional C++ source code that is based on that information and that we can compile and link into the same program as it’s being built.

In the future, using reflection it will be possible to generate C++ code within the same source file using token injection.

Other features accepted into the standard at the latest ISO C++ committee meeting include parallel algorithms for the Ranges library, async scopes, aimed at enabling RAII (resource allocation is initialization) style programming for resource management in asynchronous contexts, and a parallel scheduler providing an advanced execution context for thread pools.

Several major features accepted in previous committee meetings include contracts, asynchronous execution, and numerous others not covered here, such as enhancements to template metaprogramming with pack indexing, the #embed directive for including binary resources, bounds-checked iterators and null pointer validation, SIMD parallelism, and more.

Contracts aim to support design by contract through [[pre:]], [[post:]], and contract_assert. Originally planned for C++20, they were ultimately removed from the final draft, but are now making their way into the standard.

std::execution provides mechanisms to manage asynchronous execution on generic execution resources. Its core concepts are sender and receiver: a sender represents a unit of asynchronous work that sends its result to a receiver. The framework is completed by the concepts of state associated with an asynchronous operation and a light-weight scheduler.

Work on C++26 continues, but as Sutter highlights, it is closer to completion than many might expect, with GCC and Clang already supporting about two-thirds of the adopted language features. InfoQ will keep reporting on the evolution of the next C++ standard as it progresses.