Docs navigation
Core
Build Plugins
AI
Dev & Test
Build
Data Access
Database Migration
Errors
Messaging
Analytics
API
Cloud
Configuration
Languages
Misc
Reactive
Views
Validation
Configuration Reference
Langchain4j v2.4.1-SNAPSHOT
On this page

Langchain4j

1 Introduction

This module provides integration between Micronaut and Langchain4j.

Note
 This module is regarded as experimental and subject to change since the underlying technology (AI) is volatile and subject to change.

Various modules are provided that allow automatically configuring common Langchain4j types like ChatModel, ImageModel etc. Refer to the sections below for the supported Langchain4j extensions.

2 Quick Start

Add the following annotation processor dependency:

annotationProcessor("io.micronaut.langchain4j:micronaut-langchain4j-processor")

Then the core module:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-core")

You are now ready to configure one of the Chat Language Models, for the quick start we will use Ollama:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-ollama")

To test the integration add the test resources integration to your Maven build or Gradle build.

testResourcesService("io.micronaut.langchain4j:micronaut-langchain4j-ollama-testresource")
Note
 For Maven see Adding modules or libraries to the Test Resources service classpath.

Add the necessary configuration to configure the model name you want to use:

Configuring the Model Name
langchain4j.ollama.model-name=orca-mini

3 Response Streaming

It is possible to use response streaming. First, you need to configure a streaming chat model with langchain4j.*.streaming-chat-model.*. For example, with OpenAI:

Example Configuration
langchain4j.open-ai.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.streaming-chat-model.model-name=gpt-4o
langchain4j.open-ai.streaming-chat-model.log-requests=true
langchain4j.open-ai.streaming-chat-model.log-responses=true

Then, you will be able to inject a bean of type dev.langchain4j.model.chat.StreamingChatModel.

Asynchronous and Reactive AI Services

The methods of an AI Service can return asynchronous and reactive types:

imports
package example.micronaut.aiservice.reactive;

import dev.langchain4j.service.SystemMessage;
import io.micronaut.langchain4j.annotation.AiService;
import org.reactivestreams.Publisher;

import java.util.concurrent.CompletableFuture;
@AiService
public interface AsyncFriend {

    @SystemMessage("You are a good friend of mine. Answer using slang.")
    CompletableFuture<String> chat(String userMessage); // 

    @SystemMessage("You are a good friend of mine. Answer using slang.")
    Publisher<String> stream(String userMessage); // 
}

  1. A CompletableFuture (or CompletionStage) completes with the response of the chat model.

  2. A Reactive Streams Publisher<String> emits the partial responses of the streaming chat model.

The following return types are supported:

Return type Model Notes

TokenStream

Streaming chat model

LangChain4j’s callback API

Publisher<String> (Reactive Streams), Flux<String> (Reactor)

Streaming chat model

Works with every streaming chat model

Mono<T> (Reactor), CompletableFuture<T>, CompletionStage<T>

Chat model

Requires a chat model with a non-blocking implementation (doChatAsync)

java.util.concurrent.Flow.Publisher<String> or Flow.Publisher<AiServiceStreamingEvent>

Streaming chat model

Requires a streaming chat model with a non-blocking implementation

The AI service uses the chat model, the streaming chat model or both, depending on the return types of its methods.

The non-blocking implementations of LangChain4j 1.21 are those of the OpenAI and Anthropic models, which use the Micronaut HTTP client without blocking a thread. Asynchronous methods also run the tools without blocking when the @Tool methods return a CompletableFuture or a CompletionStage.

Tip
 In Kotlin, add the dev.langchain4j:langchain4j-kotlin dependency to return a kotlinx.coroutines.flow.Flow<String> from an AI service method.

4 AI Service

You can also define new AI services:

Defining @AiService interfaces
imports
package example.micronaut.aiservice;

import dev.langchain4j.service.SystemMessage;
import io.micronaut.langchain4j.annotation.AiService;
@AiService // 
public interface Friend {

    @SystemMessage("You are a good friend of mine. Answer using slang.") // 
    String chat(String userMessage);
}

  1. Define an interface annotated with @AiService

  2. Use Langchain4j annotations like @SystemMessage

Note
 In Python an AI service is an abstract class (abc.ABC) whose methods are declared with …​ bodies; the decorators mirror the LangChain4j annotations.
Note
 LangChain4j builds the service implementation reflectively from the Java interface generated for the Python class and reads the @SystemMessage/@UserMessage annotations from it. @AiService implies the @AllowsReflection hint (io.micronaut.core.annotation), so the Python compiler copies the annotations onto the generated interface without any @AllowsReflection or micronaut.introspection.allow-reflection option.

You can now inject the @AiService definition into any Micronaut component including tests:

Calling @AiService definitions
imports
package example.micronaut.aiservice;

import static org.junit.jupiter.api.Assertions.assertNotNull;

import dev.langchain4j.model.chat.ChatModel;
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.testcontainers.junit.jupiter.Testcontainers;
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class AiServiceTest implements OllamaTestPropertyProvider {
    @Test
    void testAiService(Friend friend, ChatModel languageModel) {
        String result = friend.chat("Hello");

        assertNotNull(result);
        assertNotNull(languageModel);
    }
}

Retrieval Augmented Generation

Retrieval is opt-in. An @AiService performs retrieval augmented generation (RAG) only when the application declares one of:

  • a dev.langchain4j.rag.RetrievalAugmentor bean, or

  • a dev.langchain4j.rag.content.retriever.ContentRetriever bean, which LangChain4j wraps in a DefaultRetrievalAugmentor.

A RetrievalAugmentor bean takes precedence over a ContentRetriever bean. For either type, a bean @Named after the service (for example @Named("expert") for @AiService("expert")) is preferred over the default bean. If several candidates exist and none is named after the service or marked @Primary, no retrieval is configured.

Defining an @AiService that returns Result<String>
imports
package example.micronaut.aiservice.rag;

import dev.langchain4j.service.Result;
import io.micronaut.context.annotation.Requires;
import io.micronaut.langchain4j.annotation.AiService;
@Requires(property = "spec.name", value = "RagTest")
@AiService // 
public interface Expert {
    Result<String> ask(String question); // 
}

  1. A plain @AiService; nothing on the interface refers to retrieval.

  2. Returning dev.langchain4j.service.Result exposes the retrieved sources alongside the answer.

Declaring a ContentRetriever bean
imports
package example.micronaut.aiservice.rag;

import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.rag.content.retriever.ContentRetriever;
import dev.langchain4j.rag.content.retriever.EmbeddingStoreContentRetriever;
import dev.langchain4j.store.embedding.EmbeddingStore;
import io.micronaut.context.annotation.Factory;
import io.micronaut.context.annotation.Requires;
import jakarta.inject.Singleton;
@Requires(property = "spec.name", value = "RagTest")
@Factory
class RetrievalFactory {

    @Singleton // 
    ContentRetriever contentRetriever(EmbeddingStore<TextSegment> embeddingStore, EmbeddingModel embeddingModel) { // 
        return EmbeddingStoreContentRetriever.builder()
            .embeddingStore(embeddingStore)
            .embeddingModel(embeddingModel)
            .maxResults(3)
            .build();
    }
}

  1. The bean is picked up by every @AiService that has no bean named after it. Add @Named to target one service.

  2. Build the retriever from the configured EmbeddingStore and EmbeddingModel.

Retrieved content reaches the model and the result
imports
package example.micronaut.aiservice.rag;

import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.service.Result;
import dev.langchain4j.store.embedding.EmbeddingStore;
import io.micronaut.context.annotation.Property;
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.testcontainers.junit.jupiter.Testcontainers;

import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
@Property(name = "spec.name", value = "RagTest")
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class RagTest implements OllamaTestPropertyProvider {

    @Test
    void retrievedContentIsPassedToTheModel(Expert expert, EmbeddingStore<TextSegment> embeddingStore, EmbeddingModel embeddingModel) {
        TextSegment fact = TextSegment.from("Micronaut LangChain4j integrates LangChain4j with the Micronaut framework.");
        embeddingStore.add(embeddingModel.embed(fact).content(), fact); // 

        Result<String> result = expert.ask("What does Micronaut LangChain4j do?"); // 

        assertNotNull(result.content());
        assertTrue(result.sources().stream().anyMatch(content -> content.textSegment().text().equals(fact.text()))); // 
    }
}

  1. Ingest a segment into the embedding store.

  2. Call the service as usual.

  3. The retrieved segment is available as a source of the result.

To configure a single service, declare an AiServiceCustomizer bean for it and call contentRetriever(…​) or retrievalAugmentor(…​) on AiServiceCreationContext.aiServices(), see Customizing AI Services.

Note
 Before 2.3.0 an EmbeddingStoreContentRetriever was attached to every @AiService automatically whenever an EmbeddingModel bean and an EmbeddingStore bean existed. Because provider modules expose an embedding model and the in-memory embedding store is enabled by default, that was the case in most applications, causing an embedding round trip on every call and failures against models without embedding support. Declare a ContentRetriever bean as shown above to restore the previous behaviour.

Configuring Retrieval

Content retrievers and the ingestion of documents can also be configured, without code.

A content retriever configured under langchain4j.rag.content-retrievers.<name> is an EmbeddingStoreContentRetriever bean qualified with <name>, so the @AiService with that name uses it:

Example Configuration
langchain4j.rag.content-retrievers.expert.max-results=3
langchain4j.rag.content-retrievers.expert.min-score=0.6

An ingestion configured under langchain4j.rag.ingestion.<name> loads the documents of its sources when the application starts, splits them into segments, embeds them and adds them to the embedding store. The sources are directories of the file system, or of the classpath with the classpath: prefix. The documents are parsed with the DocumentParser that LangChain4j finds on the classpath (for example dev.langchain4j:langchain4j-document-parser-apache-tika for PDF and Office documents), and as text otherwise.

Example Configuration
langchain4j.rag.ingestion.expert.sources[0]=classpath:docs
langchain4j.rag.ingestion.expert.sources[1]=/var/data/manuals
langchain4j.rag.ingestion.expert.glob=**.md
langchain4j.rag.ingestion.expert.max-segment-size=800
langchain4j.rag.ingestion.expert.max-overlap-size=80

Both use the default EmbeddingStore and EmbeddingModel beans unless embedding-store and embedding-model name other beans. Each ingestion also creates an EmbeddingStoreIngestor bean qualified with its name, to ingest other documents programmatically. Set ingest-on-startup to false to only create the ingestor.

Note
 The in-memory embedding store keeps the segments in memory only, so the documents are ingested again on every start. With a persistent embedding store, disable ingest-on-startup once the documents have been ingested, or ingest them with the EmbeddingStoreIngestor bean when they change.

A retrieval augmentor configured under langchain4j.rag.retrieval-augmentors.<name> retrieves content from one or more content retrievers and re-ranks it with a scoring model, so that only the most relevant content reaches the model. It is a RetrievalAugmentor bean qualified with <name>, which takes precedence over a content retriever with the same name:

Example Configuration
langchain4j.cohere.api-key=${COHERE_API_KEY}
langchain4j.cohere.scoring-model.model-name=rerank-v3.5
langchain4j.rag.content-retrievers.expert.max-results=10
langchain4j.rag.retrieval-augmentors.expert.re-rank=true
langchain4j.rag.retrieval-augmentors.expert.min-score=0.7
langchain4j.rag.retrieval-augmentors.expert.max-results=3

The content retriever retrieves 10 segments by embedding similarity, and the scoring model passes the 3 most relevant ones with a score of at least 0.7 to the model.

Multimodal Parameters

An AI service can send images, PDF files, audio and video to the models that support them, together with the user message. Annotate a String, URI or URL parameter, or a collection of them, with @ImageUrl, @PdfUrl, @AudioUrl or @VideoUrl: the argument is sent as the LangChain4j content of that type.

imports
package example.micronaut.aiservice.multimodal;

import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import io.micronaut.langchain4j.annotation.AiService;
import io.micronaut.langchain4j.annotation.ImageUrl;
import io.micronaut.langchain4j.annotation.PdfUrl;

import java.net.URI;
@AiService
public interface PictureDescriber {

    @SystemMessage("You describe pictures for visually impaired people")
    String describe(@UserMessage String question,
                    @UserMessage @ImageUrl String picture); // 

    @UserMessage("Summarize the {{topic}} of this document")
    String summarize(@V("topic") String topic,
                     @UserMessage @PdfUrl URI document); // 
}

  1. The picture at the URL is sent after the text of the user message. Like any content parameter of a LangChain4j AI service, the parameter is also annotated with @UserMessage.

  2. The content parameter can be combined with a user message template.

Parameters of the LangChain4j content types (ImageContent, PdfFileContent, AudioContent, VideoContent, or a List of Content) annotated with @UserMessage are also supported, for example to send an image read from a file.

A test can check the contents that the model receives:

imports
package example.micronaut.aiservice.multimodal;

import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.data.message.ImageContent;
import dev.langchain4j.data.message.PdfFileContent;
import dev.langchain4j.data.message.TextContent;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.response.ChatResponse;
import io.micronaut.context.annotation.Primary;
import io.micronaut.context.annotation.Property;
import io.micronaut.context.annotation.Requires;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Singleton;
import org.junit.jupiter.api.Test;

import java.net.URI;
import java.util.stream.Collectors;

import static org.junit.jupiter.api.Assertions.assertEquals;
@Property(name = "spec.name", value = "PictureDescriberTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false)
class PictureDescriberTest {

    @Test
    void sendsThePicture(PictureDescriber describer) {
        assertEquals("text:What is it?|image:https://example.com/cat.png",
            describer.describe("What is it?", "https://example.com/cat.png"));
    }

    @Test
    void sendsTheDocument(PictureDescriber describer) {
        assertEquals("text:Summarize the risks of this document|pdf:https://example.com/report.pdf",
            describer.summarize("risks", URI.create("https://example.com/report.pdf")));
    }

    /**
     * Answers with the contents of the user message.
     */
    @Singleton
    @Primary
    @Requires(property = "spec.name", value = "PictureDescriberTest")
    static class EchoChatModel implements ChatModel {
        @Override
        public ChatResponse doChat(ChatRequest request) {
            UserMessage message = (UserMessage) request.messages().getLast();
            String described = message.contents().stream().map(content -> switch (content) {
                case TextContent text -> "text:" + text.text();
                case ImageContent image -> "image:" + image.image().url();
                case PdfFileContent pdf -> "pdf:" + pdf.pdfFile().url();
                default -> content.type().name();
            }).collect(Collectors.joining("|"));
            return ChatResponse.builder().aiMessage(AiMessage.from(described)).build();
        }
    }
}

Customizing AI Services

An AiServiceCustomizer bean customizes the LangChain4j AiServices builder of an AI service before the service is built, for any setting that the annotations and the configuration do not cover:

imports
package example.micronaut.aiservice.customizer;

import dev.langchain4j.service.SystemMessage;
import io.micronaut.langchain4j.annotation.AiService;
@AiService
public interface Concierge {
    @SystemMessage("You are the concierge of a hotel.")
    String ask(String question);
}
imports
package example.micronaut.aiservice.customizer;

import io.micronaut.langchain4j.aiservices.AiServiceCreationContext;
import io.micronaut.langchain4j.aiservices.AiServiceCustomizer;
import jakarta.inject.Singleton;

import java.util.function.UnaryOperator;
@Singleton
public class ConciergeCustomizer implements AiServiceCustomizer<Concierge> { // 

    @Override
    public void customize(AiServiceCreationContext<Concierge> creationContext) {
        UnaryOperator<String> transformer = systemMessage -> systemMessage + " Keep your answers short.";
        creationContext.aiServices().systemMessageTransformer(transformer); // 
    }
}

  1. The customizer is a bean whose type argument is the AI service it customizes. A customizer qualified with the name of the service (@Named), or selected with the customizer member of @AiService, takes precedence.

  2. The AiServices builder of the service: here, a transformer that appends an instruction to the system message.

The builder of the model beans (for example OpenAiChatModel.OpenAiChatModelBuilder) can be customized the same way with a BeanCreatedEventListener.

Moderation

The methods of an AI service annotated with LangChain4j’s @Moderate check the user message with the ModerationModel bean, in parallel with the call to the chat model, and throw a ModerationException when it is flagged:

imports
package example.micronaut.aiservice.moderation;

import dev.langchain4j.service.Moderate;
import io.micronaut.langchain4j.annotation.AiService;
@AiService
public interface ModeratedAssistant {

    @Moderate // 
    String chat(String userMessage);
}

  1. The user message is moderated.

The moderation model of the AI service is the ModerationModel bean qualified with the name of the service, otherwise the default one, for example the OpenAI moderation model:

Example Configuration
langchain4j.open-ai.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.moderation-model.model-name=omni-moderation-latest

5 Agentic Service

Agentic Service

Micronaut lets you declare LangChain4j Agentic services and have concrete agents generated at runtime.

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-agentic")

Annotate an interface with AgenticService and declare methods using LangChain4j Agentic annotations.

Note
 In Python an agentic service is an abstract class (abc.ABC) whose methods are declared with …​ bodies; the decorators mirror the LangChain4j annotations.
Note
 LangChain4j builds the agents reflectively from the Java interface generated for the Python class and reads the @Agent, @UserMessage, @SequenceAgent, …​ annotations from it. @AgenticService and the LangChain4j annotations imply the @AllowsReflection hint, so the Python compiler copies them onto the generated interfaces, including those of sub-agents and workflows declared without @AgenticService: no @AllowsReflection or micronaut.introspection.allow-reflection option is needed. The hint for the LangChain4j annotations is added by micronaut-langchain4j-processor, which must be on the compile classpath.

The integration interprets your annotations and uses Micronaut DI to wire the underlying LangChain4j builders. It supports:

  • Typed agents built via AgenticServices.agentBuilder(Class) with @Agent methods.

  • Declarative workflow agents via AgenticServices.createAgenticSystem(…​).

  • Micronaut model, memory, RAG, tool, and lifecycle integration for each generated agent builder.

Quick start

imports
package example.micronaut.agentic;

import dev.langchain4j.agentic.Agent;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
/**
 * Minimal typed Agentic service used by the test-suite to validate Micronaut integration.
 */
@AgenticService
public interface GreeterAgent {

    @UserMessage("Say hello to {{name}}")
    @Agent(description = "Greets a person by name")
    String greet(@V("name") String name);
}

Usage in tests

imports
package example.micronaut.agentic;

import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;

import static org.junit.jupiter.api.Assertions.assertNotNull;
@MicronautTest(startApplication = false, environments = "agentic-test")
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class AgenticServiceTest {

    @Test
    void testAgenticGreeter(GreeterAgent agent) {
        String result = agent.greet("John");
        assertNotNull(result);
    }
}

Declarative workflows

You can build workflows using LangChain4j’s declarative annotations. The integration routes construction through Micronaut DI so that supported workflow builders are Micronaut-managed beans (allowing listeners and customization).

Supported patterns:

The Micronaut-managed workflow builder lifecycle applies to the workflow types listed above. Supervisor and planner agents (SupervisorAgent, PlannerAgent) are supported too: see Supervisor agents.

Example: an evening planner (Sequence)

imports
package example.micronaut.agentic;

import dev.langchain4j.agentic.declarative.SequenceAgent;
import dev.langchain4j.service.V;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
/**
 * Declarative sequence workflow inspired by the "EveningPlannerAgent" example.
 * This agent coordinates sub-agents and produces a final "plan" output.
 */
@AgenticService(outputKey = "plan")
public interface EveningPlannerAgent {

    // Declarative sequence workflow definition (no method body needed)
    @SequenceAgent(
        subAgents = {
            TravelRecommenderAgent.class,
            RecipeAdvisorAgent.class,
            PlanSynthesizerAgent.class
        },
        outputKey = "plan",
        name = "planEvening"
    )
    String plan(@V("topic") String topic);
}

Parallel workflow example

imports
package example.micronaut.agentic;

import dev.langchain4j.agentic.Agent;
import dev.langchain4j.agentic.declarative.Output;
import dev.langchain4j.agentic.declarative.ParallelAgent;
import dev.langchain4j.agentic.declarative.ParallelExecutor;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;

import java.util.concurrent.Executor;
import java.util.concurrent.ForkJoinPool;

import static org.junit.jupiter.api.Assertions.assertFalse;
/**
 * Validates declarative ParallelAgent workflow wiring through @AgenticService.
 * Ensures Micronaut DI correctly builds the agentic system and executes the parallel plan.
 */
@MicronautTest(startApplication = false, environments = "agentic-test")
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class ParallelPlanningAgentTest {

    @Test
    void testDeclarativeParallel(EveningPlanner agent) {
        String plan = agent.plan("jazz", "romantic");
        System.out.println("parallel plan = " + plan);
        assertFalse(plan.isEmpty());
    }

    @AgenticService
    public interface MusicPlanner {
        @UserMessage("""
            Choose a band which plays music in the {{style}} style.
            Answer with the name of the band only: no details, no explanation.
            """)
        @Agent(outputKey = "band")
        String suggestBand(@V("style") String style);
    }

    @AgenticService
    public interface DinnerPlanner {
        @UserMessage("""
            Choose a menu for dinner for the following mood: {{mood}}
            Answer with the menu only: no details, no explanations.
            """)
        @Agent(outputKey = "menu")
        String suggestMenu(@V("mood") String mood);
    }

    /**
     * Demonstrates declarative ParallelAgent orchestration using Micronaut DI.
     */
    @AgenticService(outputKey = "plan")
    public interface EveningPlanner {

        @ParallelAgent(
            subAgents = {
                MusicPlanner.class,
                DinnerPlanner.class
            },
            outputKey = "plan",
            name = "planEvening"
        )
        String plan(@V("style") String style, @V("mood") String mood);

        // Use a shared executor for parallel execution
        @ParallelExecutor
        static Executor executor() {
            return ForkJoinPool.commonPool();
        }

        // Aggregate the parallel outputs into a single "plan" string
        @Output
        static String aggregate(@V("band") String band, @V("menu") String menu) {
            String c = band == null ? "" : band.trim();
            String r = menu == null ? "" : menu.trim();
            if (c.isEmpty() && r.isEmpty()) {
                return "";
            }
            if (c.isEmpty()) {
                return "Play: " + r;
            }
            if (r.isEmpty()) {
                return "Menu: " + c;
            }
            return "Play: " + c + " | Menu: " + r;
        }
    }
}

Loop example with customization via a listener

imports
package example.micronaut.agentic;

import dev.langchain4j.agentic.Agent;
import dev.langchain4j.agentic.declarative.LoopAgent;
import dev.langchain4j.agentic.workflow.LoopAgentService;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import io.micronaut.context.annotation.Requires;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import io.micronaut.core.annotation.NonNull;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
import jakarta.inject.Singleton;
/**
 * Demonstrates declarative LoopAgent orchestration using Micronaut DI.
 */
@AgenticService(outputKey = "translation")
public interface LoopingPlannerAgent {

    @LoopAgent(
        subAgents = {
            TranslatorAgent.class
        },
        outputKey = "text",
        maxIterations = 3
    )
    String translatesInLoop(@V("text") String text);

    interface TranslatorAgent {
        @UserMessage("""
            You are a translator.
            If the text is in English, translate to French.
            If the text is in French, translate to German.
            If the text is in German, translate to Spanish.
            Translate this: "{{text}}". Answer with the translation only, no explanations, no details.
            """)
        @Agent(outputKey = "text")
        String translate(@V("text") String text);
    }

    @Singleton
    @Requires(property = "spec.name", value = "LoopingPlannerAgentTest")
    class LoopBuilderListener implements BeanCreatedEventListener<LoopAgentService<?>> {

        @Override
        public LoopAgentService<?> onCreated(@NonNull BeanCreatedEvent<LoopAgentService<?>> event) {
            LoopAgentService<?> builder = event.getBean();
            builder.exitCondition((scope, idx) -> idx == 3);
            return builder;
        }
    }
}

Configuration

You can select which ChatModel bean to use per agent via configuration. The agent id is derived from the interface name by:

  • stripping a trailing "Agent" suffix

  • converting UpperCamel to lower-kebab (e.g. CreativeWriterAgent → creative-writer)

langchain4j.agentic.agents.<agent-id>.chat-model

Examples

langchain4j.agentic.agents.greeter.chat-model=friendly-chat-model
langchain4j.agentic.agents.creative-writer.chat-model=creative-chat-model

If the property is not set and there is exactly one ChatModel bean in the context, it will be used automatically.

Memory

By default, agentic services reuse the core MessageWindowChatMemory built from the configured ChatMemoryStore. The core module publishes a MessageWindowChatMemory.Builder per available ChatMemoryStore (via @EachBean(ChatMemoryStore)), so agents automatically use the default builder when a single memory store is configured.

You can optionally override memory per agent:

langchain4j.agentic.agents.<agent-id>.memory.store

langchain4j.agentic.agents.<agent-id>.memory.max-messages

Notes: - If memory.store is not set, the default MessageWindowChatMemory.Builder is used (as resolved by Micronaut DI). - If multiple memory stores are available, set memory.store to select the store for an agent. - If memory.max-messages is not set, the global core setting langchain4j.chat-memory-store.message-window.max-messages applies.

Examples

langchain4j.agentic.agents.greeter.memory.store=redis
langchain4j.agentic.agents.creative-writer.memory.max-messages=50

Retrieval

Retrieval is opt-in. An agent performs retrieval augmented generation only when the application declares a dev.langchain4j.rag.RetrievalAugmentor bean or a dev.langchain4j.rag.content.retriever.ContentRetriever bean. A RetrievalAugmentor takes precedence over a ContentRetriever. For either type, a bean @Named after the agent id (for example @Named("greeter") for GreeterAgent) is preferred over the default bean. Embedding model and embedding store beans are never paired automatically.

Tools

You can request specific tool beans to be registered with the agent builder using the tools attribute. Tool classes should be Micronaut beans containing methods annotated with dev.langchain4j.agent.tool.Tool.

Customization

You can customize builders using Micronaut lifecycle listeners. Because workflow builders are created as Micronaut beans during declarative system construction, your listeners can adjust names, exit conditions, parallelism, etc., before the system executes.

Typed agents can also be customized via BeanCreatedEventListener<AgentBuilder<?, ?>>; the LoopingPlannerAgent snippet above shows the same Micronaut lifecycle technique applied to LoopAgentService.

For example, this listener customizes every generated declarative agent builder before LangChain4j builds the agentic system:

imports
package example.micronaut.agentic;

import dev.langchain4j.agentic.agent.AgentBuilder;
import io.micronaut.context.annotation.Requires;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import io.micronaut.core.annotation.NonNull;
import jakarta.inject.Singleton;
@Singleton
@Requires(env = "agentic-docs")
final class SupportAgentBuilderListener implements BeanCreatedEventListener<AgentBuilder<?, ?>> {

    @Override
    public AgentBuilder<?, ?> onCreated(@NonNull BeanCreatedEvent<AgentBuilder<?, ?>> event) {
        AgentBuilder<?, ?> builder = event.getBean();
        builder.name("customer-support-agent");
        builder.outputKey("supportResponse");
        return builder;
    }
}

Workflow builders can be customized the same way. This example changes the loop exit condition for all LoopAgentService builders created by the agentic integration:

imports
package example.micronaut.agentic;

import dev.langchain4j.agentic.workflow.LoopAgentService;
import io.micronaut.context.annotation.Requires;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import io.micronaut.core.annotation.NonNull;
import jakarta.inject.Singleton;
@Singleton
@Requires(env = "agentic-docs")
final class LoopAgentServiceListener implements BeanCreatedEventListener<LoopAgentService<?>> {

    @Override
    public LoopAgentService<?> onCreated(@NonNull BeanCreatedEvent<LoopAgentService<?>> event) {
        LoopAgentService<?> builder = event.getBean();
        builder.exitCondition((scope, iteration) -> iteration >= 3);
        return builder;
    }
}

The same approach applies to Micronaut-managed workflow services: SequentialAgentService, ParallelAgentService, ParallelMapperService, ConditionalAgentService, and LoopAgentService.

Supervisor agents

A supervisor agent lets the chat model decide which of its sub-agents to invoke, and with which arguments, until the request is addressed. Declare it with LangChain4j’s @SupervisorAgent on a method of an @AgenticService:

imports
package example.micronaut.agentic;

import dev.langchain4j.agentic.declarative.SupervisorAgent;
import dev.langchain4j.agentic.supervisor.SupervisorResponseStrategy;
import dev.langchain4j.service.V;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
/**
 * A supervisor agent: the chat model plans which sub-agent to invoke next until the request is addressed.
 */
@AgenticService
public interface TravelSupervisorAgent {

    @SupervisorAgent(
        subAgents = {
            TravelRecommenderAgent.class,
            RecipeAdvisorAgent.class
        },
        responseStrategy = SupervisorResponseStrategy.SUMMARY,
        maxAgentsInvocations = 5
    )
    String organize(@V("request") String request);
}

The supervisor plans with the chat model of the agentic service (see Configuration), and its sub-agents receive the Micronaut model, memory, retrieval and tool configuration of their agent id like any other agent. responseStrategy selects whether the supervisor answers with the last response of a sub-agent (LAST, the default), with the summary written by the planner (SUMMARY) or with the better of both (SCORED).

Tests can script the planner with a chat model bean that answers with the agent invocations:

imports
package example.micronaut.agentic;

import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.data.message.SystemMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.response.ChatResponse;
import io.micronaut.context.annotation.Property;
import io.micronaut.context.annotation.Primary;
import io.micronaut.context.annotation.Requires;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Singleton;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertEquals;
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false, environments = "supervisor-test")
class TravelSupervisorAgentTest {

    @Test
    void supervisorInvokesTheSubAgents(TravelSupervisorAgent supervisor) {
        assertEquals("Visit Italy and cook a pizza", supervisor.organize("I love jazz, plan my holidays"));
    }

    /**
     * Plays the planner of the supervisor: recommends a country, then a recipe, then answers.
     */
    @Singleton
    @Primary
    @Requires(env = "supervisor-test")
    static class ScriptedPlannerChatModel implements ChatModel {

        @Override
        public ChatResponse doChat(ChatRequest request) {
            String user = ((UserMessage) request.messages().getLast()).singleText();
            boolean planner = request.messages().stream()
                .anyMatch(message -> message instanceof SystemMessage system && system.text().contains("planner"));
            if (!planner) {
                return answer(user.contains("travel expert") ? "Italy" : "Italy - Pizza");
            }
            if (user.contains("'Italy - Pizza'")) {
                return answer("{\"agentName\": \"done\", \"arguments\": {\"response\": \"Visit Italy and cook a pizza\"}}");
            }
            if (user.contains("'Italy'")) {
                return answer("{\"agentName\": \"suggestRecipe\", \"arguments\": {\"country\": \"Italy\"}}");
            }
            return answer("{\"agentName\": \"recommendCountry\", \"arguments\": {\"topic\": \"jazz\"}}");
        }

        private static ChatResponse answer(String text) {
            return ChatResponse.builder().aiMessage(AiMessage.from(text)).build();
        }
    }
}

Supplier methods

LangChain4j agents can declare static supplier methods, such as @ChatModelSupplier, @ToolsSupplier, @ToolProviderSupplier, @ContentRetrieverSupplier, @ChatMemoryProviderSupplier, @ParallelExecutor or @McpClientSupplier. With Micronaut, the parameters of these methods are beans: annotate a parameter with @Named to select a bean by name.

Note
 LangChain4j calls a @ChatModelSupplier or @StreamingChatModelSupplier method that has parameters for each invocation of the agent within a workflow: a standalone typed agent ignores it.

MCP client agents

An MCP client agent calls a tool of an MCP server as a step of a workflow. Add the LangChain4j agentic MCP dependency, together with an MCP client, for example the one of Micronaut MCP:

implementation("dev.langchain4j:langchain4j-agentic-mcp")
implementation("io.micronaut.mcp:micronaut-mcp-client-langchain4j")

The @McpClientSupplier method receives the McpClient bean of the server:

imports
package example.micronaut.agentic;

import dev.langchain4j.agentic.declarative.McpClientAgent;
import dev.langchain4j.agentic.declarative.McpClientSupplier;
import dev.langchain4j.agentic.declarative.SequenceAgent;
import dev.langchain4j.mcp.client.McpClient;
import dev.langchain4j.service.V;
import io.micronaut.context.annotation.Requires;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
import jakarta.inject.Named;
/**
 * A workflow whose sub-agent calls a tool of an MCP server through a Micronaut MCP client bean.
 */
@Requires(beans = McpClient.class)
@AgenticService
public interface WorldClockAgent {

    @SequenceAgent(outputKey = "time", subAgents = CurrentTimeAgent.class)
    String time(@V("zone") String zone);

    interface CurrentTimeAgent {
        @McpClientAgent(toolName = "current-time", outputKey = "time") // 
        String currentTime(@V("zone") String zone);

        @McpClientSupplier
        static McpClient mcpClient(@Named("time-server") McpClient mcpClient) { // 
            return mcpClient;
        }
    }
}

  1. The tool of the MCP server to call. The arguments of the tool are the parameters of the method, read from the agentic scope.

  2. The Micronaut MCP client bean named time-server, configured under micronaut.mcp.client.http.time-server (see Tools for an MCP client configuration).

Note
 In Python, a class with a static function compiles to a Java interface only when it is an introduction type, so the MCP client agent above is an @AgenticService too.

A2A client agents

LangChain4j’s @A2AClientAgent invokes a remote agent with the Agent2Agent (A2A) protocol. It requires the dev.langchain4j:langchain4j-agentic-a2a dependency, and the URL of the remote agent server is set with the a2aServerUrl attribute or returned by a static @A2AServerUrlSupplier method without parameters. A static @A2AClientCustomizer method can customize the A2A client.

6 Structured Outputs

When an @AiService or @AgenticService method returns a POJO (or a List or Set of POJOs), LangChain4j asks the model for a JSON answer: it derives a JSON schema from the return type reflectively and sends it as the response format, or appends format instructions to the user message when the model does not support JSON schemas.

With Micronaut JSON Schema, the schema is generated at compile time instead, and the generated schema is the one the model receives. Compared with the schema LangChain4j derives:

  • the descriptions come from the Javadoc (or KDoc, or Python docstrings) of the type and its properties, and the validation constraints (format, pattern, minimum, …​) are passed on to the model in the descriptions;

  • no reflection is needed to build the schema, which suits GraalVM native images;

  • the schema only contains the properties of the type, whereas the reflective schema of a class generated by a compiler (for example, the Java class of a Python class) can contain synthetic or internal fields.

Add the JSON Schema annotation processor and the runtime support:

annotationProcessor("io.micronaut.jsonschema:micronaut-json-schema-processor")
implementation("io.micronaut.jsonschema:micronaut-json-schema-utils")

Then annotate the types returned by your services with io.micronaut.jsonschema.JsonSchema:

A structured output type
imports
package example.structured;

import io.micronaut.jsonschema.JsonSchema;
/**
 * A conference talk recommended to an attendee.
 *
 * @param title The title of the talk
 * @param reason Why the talk matches the interests of the attendee
 * @param score How well the talk matches the interests, from 1 to 5
 */
@JsonSchema // 
public record TalkRecommendation(String title, String reason, int score) {
}

  1. Micronaut JSON Schema generates the schema of the type, with its documentation as descriptions.

Note
 The Groovy compiler does not keep the Groovydoc of the classes, so set the description with the description member of @JsonSchema.
Note
 The class is declared @Introspected so that LangChain4j, which parses the answer of the model with Jackson, can construct it. In Python, do not name the @JsonSchema types in the micronaut.introspection.allow-reflection compiler option (the schema would be generated twice, failing the compilation): the integration does not need the annotation at runtime.
Note
 Micronaut JSON Schema describes the properties of a type: the components of a record, the accessors of a bean, or the fields of a class annotated with @Introspected(accessKind = Introspected.AccessKind.FIELD). LangChain4j also reads and writes plain fields, which are not properties for Micronaut. For a class whose state is only held in such fields, the generated schema has no properties: a warning is logged, and the schema LangChain4j derives is sent instead.

No other change is needed: the AI services and agents returning these types send the generated schemas.

An @AiService returning structured outputs
imports
package example.micronaut.aiservice.structured;

import dev.langchain4j.service.SystemMessage;
import example.structured.TalkRecommendation;
import io.micronaut.langchain4j.annotation.AiService;

import java.util.List;
@AiService
public interface TalkAdvisor {

    @SystemMessage("You recommend conference talks to attendees.")
    TalkRecommendation recommend(String interests); // 

    @SystemMessage("You recommend conference talks to attendees.")
    List<TalkRecommendation> shortlist(String interests); // 
}

  1. The schema generated for TalkRecommendation is sent as the response format.

  2. For a list, LangChain4j asks for an object whose values property holds the elements: the generated schema describes the elements.

Checking the schema received by the model
imports
package example.micronaut.aiservice.structured;

import dev.langchain4j.model.chat.request.json.JsonArraySchema;
import dev.langchain4j.model.chat.request.json.JsonObjectSchema;
import dev.langchain4j.model.chat.request.json.JsonSchema;
import example.structured.TalkRecommendation;
import io.micronaut.langchain4j.jsonschema.StructuredOutputSchemaProvider;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Inject;
import org.junit.jupiter.api.Test;

import java.util.List;
import java.util.Set;

import static org.junit.jupiter.api.Assertions.assertEquals;
@MicronautTest(startApplication = false, environments = "structured-output")
class StructuredOutputTest {

    @Inject
    TalkAdvisor talkAdvisor;

    @Inject
    StructuredOutputChatModel chatModel;

    @Inject
    StructuredOutputSchemaProvider schemaProvider;

    @Test
    void sendsTheGeneratedSchema() {
        TalkRecommendation recommendation = talkAdvisor.recommend("LLMs and Java"); // 

        assertEquals(new TalkRecommendation("Structured outputs", "You like LLMs", 5), recommendation);
        JsonSchema schema = chatModel.lastRequest().responseFormat().jsonSchema(); // 
        assertEquals("TalkRecommendation", schema.name());
        assertEquals(schemaProvider.findSchema(TalkRecommendation.class).orElseThrow(), schema.rootElement()); // 
        JsonObjectSchema root = (JsonObjectSchema) schema.rootElement();
        assertEquals("A conference talk recommended to an attendee.", root.description());
        assertEquals(Set.of("title", "reason", "score"), root.properties().keySet());
        assertEquals("Why the talk matches the interests of the attendee", root.properties().get("reason").description());
    }

    @Test
    void sendsTheGeneratedSchemaOfListElements() {
        List<TalkRecommendation> shortlist = talkAdvisor.shortlist("LLMs and Java");

        assertEquals(1, shortlist.size());
        JsonSchema schema = chatModel.lastRequest().responseFormat().jsonSchema();
        assertEquals("List_of_TalkRecommendation", schema.name());
        JsonArraySchema values = (JsonArraySchema) ((JsonObjectSchema) schema.rootElement()).properties().get("values");
        assertEquals(schemaProvider.findSchema(TalkRecommendation.class).orElseThrow(), values.items());
    }
}

  1. Call the service as usual.

  2. The test chat model records the request it receives.

  3. The response format carries the schema generated at compile time.

The schema of a type returned directly, in a List or a Set, or wrapped in Result, CompletableFuture or CompletionStage is replaced. References to the schemas of other @JsonSchema types are resolved and inlined, except recursive ones, which are sent as definitions ($defs). LangChain4j still parses the answer and decides whether the response format is sent; streaming methods are unaffected.

To supply the schemas from another source, register a bean of type StructuredOutputSchemaProvider.

Models without JSON schema support

LangChain4j only sends the schema to the chat models that declare the RESPONSE_FORMAT_JSON_SCHEMA capability. For the other models, it appends format instructions derived from the type to the user message, and the generated schema is not used.

Some models accept a JSON schema on each request but do not always declare the capability. Google AI Gemini declares it only when a JSON response format is configured on the model, yet it honors the response format of each request: with the micronaut-langchain4j-googleai-gemini module, the services returning a type with a generated schema send it to Gemini without further configuration. Providers declare such models with a bean of type JsonSchemaResponseFormatSupport.

For other models, set langchain4j.structured-output.force-json-schema-response-format to true to send the schema anyway, or to false to never send it to the models that do not declare the capability:

langchain4j.structured-output.force-json-schema-response-format=true

Only use it with models that support a JSON schema response format: a model that does not may ignore it or reject the request.

To use the schemas LangChain4j derives instead of the generated ones, set langchain4j.structured-output.enabled to false.

7 Testing

Micronaut LangChain4j includes a small evaluation API for asserting AI responses in tests.

The EvaluationRequest record captures the original user text, optional grounding context, and generated response. An Evaluator consumes that request and returns an EvaluationResult.

Built-in evaluators include:

  • RelevancyEvaluator for checking whether the response answers the user request.

  • FactCheckingEvaluator for checking whether the response is grounded in the supplied context.

When an AI service returns dev.langchain4j.service.Result<T>, you can reuse retrieved sources as evaluation context:

Defining an @AiService that returns Result<String>
imports
package example.micronaut.aiservice.evaluation;

import dev.langchain4j.service.Result;
import dev.langchain4j.service.SystemMessage;
import io.micronaut.context.annotation.Requires;
import io.micronaut.langchain4j.annotation.AiService;
@Requires(property = "spec.name", value = "AiServiceEvaluationExample")
@AiService
public interface EvaluatingFriend {
    @SystemMessage("You are a good friend of mine. Answer using slang.")
    Result<String> chat(String userMessage);
}
Evaluating an AI service response
imports
package example.micronaut.aiservice.evaluation;

import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.service.Result;
import io.micronaut.context.annotation.Property;
import io.micronaut.langchain4j.evaluation.EvaluationRequest;
import io.micronaut.langchain4j.evaluation.EvaluationResult;
import io.micronaut.langchain4j.evaluation.RelevancyEvaluator;
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.testcontainers.junit.jupiter.Testcontainers;

import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
@Property(name = "spec.name", value = "AiServiceEvaluationExample")
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class AiServiceEvaluationExample implements OllamaTestPropertyProvider {

    @Test
    void evaluatesAiServiceResponse(EvaluatingFriend friend, ChatModel chatModel) {
        String userText = "Reply with exactly: Micronaut is a JVM framework.";
        Result<String> response = friend.chat(userText);

        RelevancyEvaluator evaluator = new RelevancyEvaluator(chatModel);
        EvaluationResult evaluation = evaluator.evaluate(EvaluationRequest.from(userText, response));

        assertNotNull(evaluation);
        assertFalse(evaluation.feedback().isBlank());
    }
}

FactCheckingEvaluator requires non-empty context, which makes it a good fit for RAG-style responses backed by retrieved sources.

Evaluation samples

The Micronaut LangChain4j test module runs evaluations from sample files and provides a scripted chat model for tests. Add it to the test classpath:

testImplementation("io.micronaut.langchain4j:micronaut-langchain4j-test")

A sample file lists the inputs to send to the AI service, with the optional context that the answers must be grounded in, for example src/test/resources/samples/support.yml:

samples:
  - name: framework
    input: What is Micronaut?
    context: Micronaut is a modern JVM framework for building microservices and serverless applications.
  - name: native
    input: Can a Micronaut application be compiled to a native executable?
    context: Micronaut applications can be compiled to native executables with GraalVM.

A JUnit parameterized test then evaluates the answer to each sample:

imports
package example.micronaut.testing;

import dev.langchain4j.model.chat.ChatModel;
import io.micronaut.context.annotation.Bean;
import io.micronaut.context.annotation.Factory;
import io.micronaut.context.annotation.Primary;
import io.micronaut.context.annotation.Property;
import io.micronaut.context.annotation.Requires;
import io.micronaut.langchain4j.evaluation.RelevancyEvaluator;
import io.micronaut.langchain4j.test.EvaluationSample;
import io.micronaut.langchain4j.test.EvaluationSamplesSource;
import io.micronaut.langchain4j.test.ScriptedChatModel;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Inject;
import jakarta.inject.Named;
import org.junit.jupiter.params.ParameterizedTest;

import static io.micronaut.langchain4j.test.EvaluationAssertions.assertPasses;
@Property(name = "spec.name", value = "SupportAssistantEvaluationTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@Property(name = "langchain4j.evaluation.chat-model", value = "judge") // 
@MicronautTest(startApplication = false)
class SupportAssistantEvaluationTest {

    @Inject
    SupportAssistant assistant;

    @Inject
    RelevancyEvaluator relevancy; // 

    @ParameterizedTest
    @EvaluationSamplesSource("samples/support.yml") // 
    void answersEachSample(EvaluationSample sample) {
        String answer = assistant.answer(sample.input());
        assertPasses(sample, relevancy.evaluate(sample.request(answer))); // 
    }

    @Factory
    @Requires(property = "spec.name", value = "SupportAssistantEvaluationTest")
    static class Models {

        @Bean
        @Primary
        ChatModel assistantModel() { // 
            return ScriptedChatModel.builder()
                .whenUserMessageContains("native").respond("Yes, with GraalVM.")
                .otherwise("Micronaut is a JVM framework for microservices.")
                .build();
        }

        @Bean
        @Named("judge")
        ChatModel judge() {
            return ScriptedChatModel.respondingWith("PASS\nThe answer is relevant.");
        }
    }
}

  1. The module exposes RelevancyEvaluator and FactCheckingEvaluator beans, judged by the ChatModel bean named by langchain4j.evaluation.chat-model, otherwise by the default ChatModel. In a real evaluation the judge is a capable model.

  2. The evaluator is injected like any bean.

  3. @EvaluationSamplesSource runs the test once per sample of the file, which EvaluationSamples.load(…​) also reads.

  4. EvaluationAssertions.assertPasses fails the test with the feedback of the evaluator.

  5. ScriptedChatModel answers with the first rule that matches the request and records the requests, so a test does not need a real model.

Note
 In Python, the test loops over the samples, since JUnit parameterized tests are not available.

ScriptedChatModel can also request tool calls and stream:

  • whenUserMessageContains(text), whenToolResult() and when(predicate) select the requests, and respond(…​) or callTool(name, argumentsJson) answer them, in order. otherwise(…​) answers the other requests.

  • ScriptedChatModel.respondingWith(responses…​) answers the requests with the responses, in order.

  • streaming() returns a StreamingChatModel with the same rules, which streams the text of the responses word by word.

  • requests() returns the requests received, for example to check the messages sent to the model.

8 Tools

Tools allow AI models to request specific actions that extends beyond their built-in capabilities.

imports
package example.micronaut.aiservice.tools;

import dev.langchain4j.agent.tool.Tool;
import jakarta.inject.Singleton;

import java.time.LocalDate;
@Singleton // 
public class LegalDocumentTools {
    @Tool("Returns the last time the PRIVACY document was updated") // 
    public LocalDate lastUpdatePrivacy() {
        return LocalDate.of(2013, 3, 9); // 
    }
}

  1. You should annotate the classes with @Tool methods with @Singleton.

  2. The @Tool value specifies the description of the tool. The annotated method returns the date when a company PRIVACY document was last updated.

A model would have no way to know the last date when the PRIVACY document was updated. Hence, it is a good candidate for a tool. <3> The dates are harcoded for the purpose of this example, but they could have been retrieved from a database or external API.

You can supply the tools to use to an @AiService:

imports
package example.micronaut.aiservice.tools;

import io.micronaut.langchain4j.annotation.AiService;
@AiService(tools = LegalDocumentTools.class)
public interface CompanyBot {
    String ask(String question);
}
Note
 The tool is registered under the method name, which the model sees, so the Python example keeps the Java name lastUpdatePrivacy.

The tools are not discovered by LangChain4j scanning the tool class: Micronaut builds the ToolSpecification of each @Tool method from its compile-time metadata (the method and parameter names, @Tool, @P, @ToolMemoryId) and invokes the method through its Micronaut ExecutableMethod. Tool classes therefore need no reflection, neither in a GraalVM native image nor for classes written in Python. The tools of a service are also available programmatically, as LangChain4j AiServiceTool instances, from tools.ToolRegistry#getAiServiceTools(java.util.Set).

Note
 LangChain4j’s @CompensateFor compensating actions are not supported, since LangChain4j only registers them for the tools it discovers reflectively. A warning is logged when a tool bean used by a service declares one, because the action never runs, even with compensateOnToolErrors.

Tool Providers

Tools can also be provided dynamically by a LangChain4j ToolProvider bean, for example the tool provider of the Micronaut MCP LangChain4j client, which provides the tools of the configured MCP servers.

imports
package example.micronaut.aiservice.tools;

import dev.langchain4j.agent.tool.ToolSpecification;
import dev.langchain4j.service.tool.ToolProvider;
import dev.langchain4j.service.tool.ToolProviderRequest;
import dev.langchain4j.service.tool.ToolProviderResult;
import jakarta.inject.Named;
import jakarta.inject.Singleton;
@Singleton
@Named("weather") // 
public class WeatherToolProvider implements ToolProvider {

    private final ToolSpecification forecast = ToolSpecification.builder()
        .name("forecast")
        .description("Returns the weather forecast")
        .build();

    @Override
    public ToolProviderResult provideTools(ToolProviderRequest request) { // 
        return ToolProviderResult.builder()
            .add(forecast, (toolRequest, memoryId) -> "sunny")
            .build();
    }
}

  1. The tool provider is a bean qualified with a name.

  2. It returns the specifications of the tools and the executors that invoke them.

Name the tool providers of an AI service with the toolProviders member of @AiService:

imports
package example.micronaut.aiservice.tools;

import io.micronaut.langchain4j.annotation.AiService;
@AiService(toolProviders = "weather") // 
public interface WeatherAssistant {
    String ask(String question);
}

  1. The AI service uses the tools of the ToolProvider bean named weather.

An AI service uses the following tool providers:

  • The ToolProvider beans named by the toolProviders member of @AiService. Set the member to an empty array to use no tool provider.

  • Otherwise, the ToolProvider bean qualified with the name of the service, given by the named member of @AiService.

  • Otherwise, the default ToolProvider bean: the only ToolProvider bean without a qualifier (or annotated with @Primary). Tool providers qualified with a name are only used by the services that select them, and when there are several unqualified tool providers none of them is used implicitly, so that tools are never exposed to a service by accident.

The following LangChain4j beans are also applied to AI services, preferring a bean qualified with the name of the service over the default bean:

  • ToolExecutionErrorHandler - converts an exception thrown by a tool into the result sent to the model.

  • ToolArgumentsErrorHandler - handles arguments the model sent that cannot be parsed.

  • ToolSearchStrategy - selects the tools sent to the model when there are many tools.

Tool Calling Configuration

The tool calling loop of the AI services can be configured under langchain4j.ai-services. The default entry applies to every AI service, an entry with the name of a service replaces it for that service:

Example Configuration
langchain4j.ai-services.default.max-tool-calling-round-trips=10
langchain4j.ai-services.weather.execute-tools-concurrently=true
langchain4j.ai-services.weather.tool-executor=virtual

MCP Tools

The tools of Model Context Protocol (MCP) servers can be provided to AI services with the Micronaut MCP LangChain4j client:

implementation("io.micronaut.mcp:micronaut-mcp-client-langchain4j")

The client creates a LangChain4j McpClient bean for each configured MCP server, named after its configuration, and a ToolProvider bean that provides the tools of all of them.

Example Configuration
micronaut.mcp.client.http.github.url=https://example.com/mcp

This ToolProvider is unqualified, so when it is the only unqualified one it is the default tool provider of the AI services that do not select tool providers (see the previous section): they receive the tools of every configured MCP server. To restrict a service to some MCP servers, name their clients with the mcpClients member of @AiService:

imports
package example.micronaut.aiservice.mcp;

import io.micronaut.langchain4j.annotation.AiService;
@AiService(mcpClients = "github") // 
public interface GitHubAssistant {
    String ask(String question);
}

  1. The AI service only receives the tools of the MCP server configured under micronaut.mcp.client.http.github.

When mcpClients is set, no ToolProvider bean is applied implicitly. Combine it with toolProviders to add other tool providers. The mcpClients member requires dev.langchain4j:langchain4j-mcp on the classpath, which the Micronaut MCP LangChain4j client provides.

To expose the tools of a Micronaut application as an MCP server, see the Micronaut MCP server documentation.

9 Guardrails

Guardrails validate the input of an AI service before it is sent to the model, and the output of the model before it is returned. They are declared with LangChain4j’s @InputGuardrails and @OutputGuardrails annotations, on an AI service method or on the AI service to apply to all its methods.

The guardrail classes are resolved as Micronaut beans, so they can inject other beans:

imports
package example.micronaut.aiservice.guardrails;

import dev.langchain4j.guardrail.InputGuardrail;
import dev.langchain4j.guardrail.InputGuardrailRequest;
import dev.langchain4j.guardrail.InputGuardrailResult;
import jakarta.inject.Singleton;
@Singleton // 
public class PromptInjectionGuard implements InputGuardrail {

    @Override
    public InputGuardrailResult validate(InputGuardrailRequest request) { // 
        if (request.userMessage().singleText().toLowerCase().contains("ignore previous instructions")) {
            return fatal("Possible prompt injection"); // 
        }
        return success();
    }
}

  1. The guardrail is a bean.

  2. An input guardrail validates the user message, and can read the rest of the request (the chat memory, the augmentation result…​).

  3. A fatal result stops the invocation: the AI service throws an InputGuardrailException.

imports
package example.micronaut.aiservice.guardrails;

import dev.langchain4j.service.guardrail.InputGuardrails;
import io.micronaut.langchain4j.annotation.AiService;
@AiService
public interface GuardedAssistant {

    @InputGuardrails(PromptInjectionGuard.class) // 
    String chat(String userMessage);
}

  1. The guardrail validates the input of this method.

Output guardrails are declared the same way with @OutputGuardrails. They can ask the model to retry or reprompt it, up to the maxRetries of the annotation.

A guardrail class that is not a bean is instantiated with its public no-argument constructor, which requires reflection in a GraalVM native image. The guardrails provided by LangChain4j’s dev.langchain4j:langchain4j-guardrails module (for example the prompt injection and moderation guardrails) can be used this way, or registered as beans with a factory to configure them.

The guardrails also produce events, see AI Service Events.

10 Reflection and GraalVM Native Image

LangChain4j implements AI services and agents with java.lang.reflect.Proxy and reads their annotations (@SystemMessage, @UserMessage, @V, @Agent, @SequenceAgent, @Output, …​), parameters and return types reflectively. Micronaut LangChain4j takes care of what this requires, so applications declare no reflection configuration:

  • Tools are not discovered reflectively. The ToolSpecification of each @Tool method is built from the compile-time metadata of the method, and the method is invoked through its Micronaut ExecutableMethod, so tool classes need no reflection at all.

  • GraalVM Native Image: micronaut-langchain4j-processor writes the native image metadata (META-INF/native-image/io.micronaut.langchain4j/<type>/reachability-metadata.json) of every @AiService, every @AgenticService and every type declaring @Agent, workflow (@SequenceAgent, @ParallelAgent, @LoopAgent, …​) or other declarative agent (@HumanInTheLoop, @A2AClientAgent, @McpClientAgent, @RegistryAgent) methods: the reflective access to their methods, and the proxy classes LangChain4j creates for them.

  • Python: @AiService, @AgenticService and the LangChain4j annotations imply the @AllowsReflection hint (io.micronaut.core.annotation), so the Python compiler copies the annotations onto the Java types it generates. Neither @AllowsReflection nor the micronaut.introspection.allow-reflection compiler option is needed. The hint for the LangChain4j annotations is added by micronaut-langchain4j-processor, which must be on the compile classpath of the Python sources.

  • Structured outputs and tool types: LangChain4j derives the JSON schema of the structured outputs of AI services and agents (the records or POJOs their methods return), and of the parameters and results of tools, from their fields, and maps them with Jackson, reflectively. The processor also writes the native image metadata of these types and of the types of their fields, so they need neither @ReflectiveAccess nor hand-written metadata. With micronaut-langchain4j-serde they are mapped with Micronaut Serialization instead of Jackson, see Micronaut Serialization.

Note
 The processor must be on the annotation processor path of the classes declaring the services and agents (annotationProcessor("io.micronaut.langchain4j:micronaut-langchain4j-processor") in Gradle) for the native image metadata to be generated.

Native Image Support by Module

Several LangChain4j providers map their REST APIs with Jackson, reflectively, and do not ship native image metadata. The Micronaut modules of these providers (Ollama, Mistral AI, Google AI Gemini, Hugging Face, Cohere, Jina and Voyage AI) ship the reachability metadata of their request and response types, so applications need no reflection configuration for them either. The test-suite-graal project verifies the following in a native image:

Module Status

micronaut-langchain4j-openai

Tested: chat, streaming chat, embeddings

micronaut-langchain4j-anthropic

Tested: chat

micronaut-langchain4j-mistralai

Tested: chat, streaming chat, embeddings

micronaut-langchain4j-ollama

Tested: chat, streaming chat, embeddings, tool calling through an AI service

micronaut-langchain4j-googleai-gemini

Tested: chat, streaming chat, embeddings

micronaut-langchain4j-bedrock

Tested: model creation. The AWS SDK provides its own native image metadata.

micronaut-langchain4j-cohere

Tested: scoring, embeddings

micronaut-langchain4j-jina

Tested: scoring

micronaut-langchain4j-voyageai

Tested: scoring, embeddings

micronaut-langchain4j-hugging-face

Reachability metadata provided, not tested

micronaut-langchain4j-azure

Not tested. Known issue: the Azure SDK (azure-core) initializes SLF4J and Logback at image build time, which fails the image build when another class with a static logger is also initialized at build time (seen together with micronaut-langchain4j-agentic)

micronaut-langchain4j-vertexai, micronaut-langchain4j-vertexai-gemini, micronaut-langchain4j-oci-genai

Not tested: they rely on the native image support of the Google Cloud and OCI SDKs

micronaut-langchain4j-jlama

Not supported: Jlama runs the model in the JVM with the Vector API

In-memory embedding store (core)

Tested

Other embedding stores

Not tested: they rely on the native image support of their client libraries

11 Micronaut Serialization

LangChain4j maps the structured outputs of AI services and agents (the records or POJOs their methods return), the arguments and results of tools and the arguments of agents to and from JSON with Jackson, reflectively. With the following dependency, it maps them with Micronaut Serialization instead:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-serde")

Micronaut Serialization maps a type with the introspection generated at compile time for it, without reflection, when the type is annotated with @Serdeable or imported with @SerdeImport. With micronaut-serde-processor and micronaut-langchain4j-processor on the annotation processor path, the public records returned by AI services and agents, and the public records used as parameters or results of tools, are imported automatically, together with the records they are made of.

annotationProcessor("io.micronaut.serde:micronaut-serde-processor")

The other types are still mapped by Micronaut Serialization: with their introspection when they are @Introspected, and otherwise with an introspection that micronaut-reflection builds reflectively from their fields, of any visibility, and their accessors, as LangChain4j’s Jackson mapping does. Enable the DEBUG level of the io.micronaut.langchain4j.serde logger to see the types mapped reflectively. Jackson annotations (@JsonProperty, @JsonCreator, …​) are only honoured on the types with a compile-time introspection: Micronaut Serialization maps them onto the introspection when it is generated.

The mapping follows the defaults of LangChain4j’s codec rather than the configuration of the application, whose JSON mapping is left unchanged: unknown properties are rejected (LangChain4j relies on it to find the JSON in the answers of the models), enums are read case insensitively, null values are written, and LocalDate, LocalTime and LocalDateTime are also read in the object shape of the JSON schema LangChain4j derives for them ({"year": 2001, "month": 2, "day": 3}).

Note
 Micronaut Serialization replaces Jackson for these mappings only. Jackson remains a dependency of LangChain4j, which also uses it for the requests and responses of the model providers, the persistence of chat memories and agentic scopes, and to read the Jackson annotations of the types it derives JSON schemas from.

12 Observability

Every chat model, streaming chat model and embedding model whose LangChain4j builder accepts listeners receives the ChatModelListener (or embedding model listener) beans of the application. Define a listener as a bean to observe the requests, responses and errors of all models:

imports
package example.micronaut.observability;

import dev.langchain4j.model.chat.listener.ChatModelListener;
import dev.langchain4j.model.chat.listener.ChatModelResponseContext;
import dev.langchain4j.model.output.TokenUsage;
import jakarta.inject.Singleton;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Singleton // 
public class TokenUsageListener implements ChatModelListener {

    private static final Logger LOG = LoggerFactory.getLogger(TokenUsageListener.class);

    @Override
    public void onResponse(ChatModelResponseContext responseContext) { // 
        TokenUsage tokenUsage = responseContext.chatResponse().tokenUsage();
        LOG.info("Tokens used: {}", tokenUsage);
    }
}

  1. The listener is a bean, so it is injected into every model.

  2. It is notified of every chat model response, here to log the token usage.

Micrometer

To record the OpenTelemetry GenAI metrics of the chat models with Micronaut Micrometer, add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-micrometer")

When there is a MeterRegistry bean, LangChain4j’s MicrometerMetricsChatModelListener is registered. It records:

  • gen_ai.client.token.usage - the input and output tokens, tagged with the provider, the operation, the request and response model names and the token type.

  • gen_ai.client.operation.duration - the duration of the chat model requests.

When there is also an ObservationRegistry bean (for example with micronaut-micrometer-observation) and dev.langchain4j:langchain4j-observation is on the classpath, LangChain4j’s ObservationChatModelListener is registered instead. It records each chat model request as a Micrometer observation, which produces the same metrics and, with a tracing bridge, spans.

Set langchain4j.micrometer.enabled to false to disable these listeners.

AI Service Events

The AI services publish LangChain4j’s AI service events as Micronaut application events. Observe them with @EventListener methods or ApplicationEventListener beans:

imports
package example.micronaut.aiservice.events;

import dev.langchain4j.observability.api.event.AiServiceCompletedEvent;
import dev.langchain4j.observability.api.event.ToolExecutedEvent;
import io.micronaut.runtime.event.annotation.EventListener;
import jakarta.inject.Singleton;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Singleton
public class AiServiceAuditor {

    private static final Logger LOG = LoggerFactory.getLogger(AiServiceAuditor.class);

    @EventListener // 
    void onCompleted(AiServiceCompletedEvent event) {
        LOG.info("AI service {} completed invocation {}",
            event.invocationContext().interfaceName(),
            event.invocationContext().invocationId()); // 
    }

    @EventListener
    void onToolExecuted(ToolExecutedEvent event) { // 
        LOG.info("Tool {} returned {}", event.request().name(), event.resultText());
    }
}

  1. An event listener method receives the events of every AI service.

  2. The invocation context identifies the AI service, the method and the invocation, which correlates the events of one invocation.

  3. Each tool execution produces an event with the request and the result of the tool.

The following events are published: AiServiceStartedEvent, AiServiceRequestIssuedEvent, AiServiceResponseReceivedEvent, AiServiceCompletedEvent, AiServiceErrorEvent, InputGuardrailExecutedEvent, OutputGuardrailExecutedEvent, ToolExecutedEvent and ToolCompensatedEvent (all in the dev.langchain4j.observability.api.event package). A listener of a super type, such as AiServiceEvent, receives all of them. An event type is only published when there is a listener for it.

Beans implementing LangChain4j’s AiServiceListener (for example AiServiceCompletedListener) are also registered with every AI service.

13 HTTP Client

The OpenAI, Anthropic, Ollama, Mistral AI, Google AI Gemini, Hugging Face, Cohere, Jina and Voyage AI models send their requests with the Micronaut HTTP client instead of the HTTP client of LangChain4j. These modules exclude LangChain4j’s JDK HTTP client, so add a Micronaut HTTP client implementation to your application:

runtimeOnly("io.micronaut:micronaut-http-client")

Because the requests go through the Micronaut HTTP client:

  • The micronaut.http.client.* configuration applies to them, for example the proxy, the SSL configuration, the connection pool and the event loop.

  • The timeout of a model sets the read timeout of its requests.

  • HTTP client filters apply to them, for example to add headers or to record requests:

imports
package example.micronaut.http;

import io.micronaut.http.MutableHttpRequest;
import io.micronaut.http.annotation.ClientFilter;
import io.micronaut.http.annotation.RequestFilter;
@ClientFilter("/v1/**") // 
public class RequestSourceFilter {

    @RequestFilter
    public void addSource(MutableHttpRequest<?> request) {
        request.header("X-Request-Source", "my-service"); // 
    }
}

  1. The filter matches the request paths of the provider, here the /v1/ paths of the OpenAI API.

  2. A header is added to every request.

  • HTTP client observability, such as Micrometer metrics and the tracing of Micronaut Tracing, records the requests.

Model Credentials

The credentials of a model are usually its configured API key. To send other credentials with each request, for example a short-lived token or the token of the current user, declare a ModelAuthProvider bean. The providers are asked in order for each request of the models that use the Micronaut HTTP client: the first value that is not null replaces the Authorization header.

imports
package example.micronaut.http;

import io.micronaut.context.annotation.Requires;
import io.micronaut.context.annotation.Value;
import io.micronaut.langchain4j.http.client.ModelAuthProvider;
import io.micronaut.langchain4j.http.client.ModelAuthRequest;
import jakarta.inject.Singleton;
import org.jspecify.annotations.Nullable;
@Singleton
@Requires(property = "ai-gateway.host") // 
public class GatewayAuthProvider implements ModelAuthProvider {

    private final String gatewayHost;

    public GatewayAuthProvider(@Value("${ai-gateway.host}") String gatewayHost) {
        this.gatewayHost = gatewayHost;
    }

    @Override
    public @Nullable String authorization(ModelAuthRequest request) {
        if (!gatewayHost.equals(request.uri().getHost())) {
            return null; // 
        }
        return "Bearer " + currentToken(); // 
    }

    private String currentToken() {
        // for example a short-lived token of a secret manager, refreshed when it expires
        return System.getenv().getOrDefault("AI_GATEWAY_TOKEN", "");
    }
}

  1. The provider is only enabled when the gateway is configured.

  2. Returning null keeps the credentials of the model configuration, here for the models that are not served by the gateway.

  3. The value of the Authorization header.

The provider is called on the thread that sends the request, also for the streaming and asynchronous requests, so it can read the context of the current HTTP request.

Token Propagation

When the application is secured with Micronaut Security, the bearer token of the user that calls the application can be sent to the models in place of their API key, for example to an OpenAI-compatible gateway that authorizes each user:

langchain4j.token-propagation.enabled=true
langchain4j.token-propagation.uri-regex=https://ai-gateway[.]example[.]com/.*

uri-regex limits the propagation to the models whose URLs match. The token is the one that Micronaut Security authenticated for the current request.

Note
 The token propagation of Micronaut Security itself (micronaut.security.token.propagation) does not apply to the models: it never replaces the Authorization header that a request already has, and the models send their API key.

Streaming and asynchronous requests

The streaming responses (server-sent events) and the non-blocking requests of the asynchronous AI services are also handled by the Micronaut HTTP client.

The other providers (Azure OpenAI, Bedrock, Vertex AI, OCI GenAI and Google Gen AI) use the HTTP client of their cloud SDK.

14 Image Models

Image models generate images from a text prompt. The OpenAI, Azure OpenAI, Vertex AI and Google Gen AI modules configure a dev.langchain4j.model.image.ImageModel bean, under langchain4j.<provider>.image-model (and named ones under langchain4j.<provider>.image-models.<name>):

Example Configuration
langchain4j.open-ai.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.image-model.model-name=gpt-image-1
langchain4j.open-ai.image-model.size=1024x1024

Inject the image model where images are generated:

imports
package example.micronaut.image;

import dev.langchain4j.data.image.Image;
import dev.langchain4j.model.image.ImageModel;
import jakarta.inject.Singleton;
@Singleton
public class ImageGenerator {

    private final ImageModel imageModel;

    public ImageGenerator(ImageModel imageModel) { // 
        this.imageModel = imageModel;
    }

    public Image generate(String prompt) {
        return imageModel.generate(prompt).content(); // 
    }
}

  1. The ImageModel bean is injected.

  2. The model generates an image. Depending on the model, the Image holds a URL (url()) or base64 data (base64Data() with its mimeType()): gpt-image-1, for example, only returns base64 data.

15 Test Resources

Micronaut Test Resources starts the services the tests need in containers. Micronaut LangChain4j provides test resources for Ollama and Qdrant, and the embedding and chat memory stores backed by a database (PGVector, Oracle, Redis, Neo4j, Cassandra, MongoDB, JDBC) can use the database test resources of Micronaut.

Add the test resources of Micronaut LangChain4j to the test resources service classpath of your Gradle build or Maven build.

Ollama

testResourcesService("io.micronaut.langchain4j:micronaut-langchain4j-ollama-testresource")

When langchain4j.ollama.base-url is not configured, an Ollama container with the configured model is started, and the base URL is resolved from it. The image is langchain4j/ollama-<model-name>, which bundles the model:

Example Configuration
langchain4j.ollama.model-name=orca-mini

Qdrant

testResourcesService("io.micronaut.langchain4j:micronaut-langchain4j-qdrant-testresource")

When the host and port of the Qdrant embedding store are not configured, a Qdrant container is started with a collection of 384 dimensions (cosine distance) named after collection-name, and langchain4j.qdrant.embedding-store.host and port are resolved from it:

Example Configuration
langchain4j.qdrant.embedding-store.collection-name=test

16 Experimental Features

The features of this section integrate experimental LangChain4j APIs: they are annotated with @Experimental and can change in a minor release, following LangChain4j.

Decision Services

A decision service answers questions with a LangChain4j DecisionModel, a model that returns the probability of each possible answer instead of generated text: yes/no questions, a choice between options, or a position on a scale. Annotate an interface with @DecisionService and its methods with LangChain4j’s @Decide:

imports
package example.micronaut.decision;

import dev.langchain4j.service.V;
import dev.langchain4j.service.decision.Choice;
import dev.langchain4j.service.decision.Decide;
import io.micronaut.langchain4j.annotation.DecisionService;
@DecisionService // 
public interface TicketTriage {

    @Decide("Is this message spam?")
    boolean isSpam(@V("message") String message); // 

    @Decide("Which team should handle this ticket?")
    Team route(@V("ticket") String ticket); // 

    @Decide("Which team should handle this ticket?")
    Choice<Team> routeWithProbabilities(@V("ticket") String ticket); // 
}

  1. The interface is implemented with LangChain4j’s DecisionServices. It uses the DecisionModel bean named after the service (@DecisionService("support")), otherwise the default DecisionModel bean, and the ThresholdProvider bean selected the same way.

  2. A boolean method asks a yes/no question: the answer is true when the probability of "yes" reaches the threshold.

  3. An enum method chooses one of the constants.

  4. Choice, Scale, YesNoAnswer and DecisionResult also return the probabilities. A record whose fields are annotated with @Decide asks several questions in a single call.

The parameters are sent to the model as the input, by name, so name them with @V unless the code is compiled with -parameters.

The constants of the enum can be described with LangChain4j’s @Description:

imports
package example.micronaut.decision;

import dev.langchain4j.model.output.structured.Description;
public enum Team {
    @Description("Payments, invoices and refunds") // 
    BILLING,
    @Description("Problems using the product, crashes and errors")
    SUPPORT
}

  1. The description helps the model choose.

Note
 A Python Enum compiles to a Java enum whose constants have no @Description: give them self-explanatory names.

A test can provide its own DecisionModel bean:

imports
package example.micronaut.decision;

import dev.langchain4j.model.decision.DecisionModel;
import dev.langchain4j.model.decision.request.ChoiceQuestion;
import dev.langchain4j.model.decision.request.DecisionRequest;
import dev.langchain4j.model.decision.response.ChoiceAnswer;
import dev.langchain4j.model.decision.response.DecisionResponse;
import dev.langchain4j.model.decision.response.YesNoAnswer;
import io.micronaut.context.annotation.Property;
import io.micronaut.context.annotation.Requires;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Singleton;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
@Property(name = "spec.name", value = "TicketTriageTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false)
class TicketTriageTest {

    @Test
    void triagesTickets(TicketTriage triage) {
        assertFalse(triage.isSpam("I was charged twice"));
        assertEquals(Team.BILLING, triage.route("I was charged twice"));
        assertEquals(0.9, triage.routeWithProbabilities("I was charged twice").probabilityOf(Team.BILLING));
    }

    /**
     * Answers no to the yes/no questions and chooses the first option of the other questions.
     */
    @Singleton
    @Requires(property = "spec.name", value = "TicketTriageTest")
    static class FirstOptionDecisionModel implements DecisionModel {
        @Override
        public DecisionResponse doDecide(DecisionRequest request) {
            DecisionResponse.Builder response = DecisionResponse.builder();
            request.questions().forEach((name, question) -> {
                if (question instanceof ChoiceQuestion choice) {
                    String first = choice.options().keySet().iterator().next();
                    ChoiceAnswer.Builder answer = ChoiceAnswer.builder().value(first).options(choice.options().keySet());
                    choice.options().keySet().forEach(option -> answer.probability(option, option.equals(first) ? 0.9 : 0.1));
                    response.answer(name, answer.build());
                } else {
                    response.answer(name, YesNoAnswer.of(0.1));
                }
            });
            return response.build();
        }
    }
}

Routing Chat Models

A routing chat model is a ChatModel that sends each request to one of several chat models, for example a small model for simple requests and a large one for the others. Configure it under langchain4j.routing-chat-models.<name>: the model is a ChatModel bean of that name, which an @AiService with the same name uses.

langchain4j.routing-chat-models.assistant.routes.fast=small
langchain4j.routing-chat-models.assistant.routes.expert=large
langchain4j.routing-chat-models.assistant.descriptions.fast=Greetings, small talk and simple questions
langchain4j.routing-chat-models.assistant.descriptions.expert=Mathematics, reasoning and code
langchain4j.routing-chat-models.assistant.default-route=fast

routes maps each route to the name of a ChatModel bean, and descriptions describes the requests of each route. By default, the route is chosen by asking a DecisionModel which description matches the request: the DecisionModel bean named by decision-model, otherwise the default one. question, min-probability and fallback-strategy (DEFAULT_ROUTE or FAIL) tune that choice. Set router to the name of a ChatModelRouter bean to choose the route in another way.

Agent Skills

Agent Skills are folders of instructions and resources that a model loads when a request needs them, instead of keeping them all in its context. Add the LangChain4j Skills dependency:

implementation("dev.langchain4j:langchain4j-skills")

Each skill is a directory with a SKILL.md file, whose front matter names and describes it, for example src/main/resources/skills/packing/SKILL.md:

---
name: packing
description: Prepares a packing list for a trip
---

List the clothes for the weather of the destination, then the travel documents.

Configure a set of skills, loaded from a classpath directory and/or a file system directory, under langchain4j.skills.<name>:

langchain4j.skills.travel.classpath=skills

Then select it with the skills member of @AiService:

imports
package example.micronaut.aiservice.skills;

import dev.langchain4j.service.SystemMessage;
import io.micronaut.langchain4j.annotation.AiService;
@AiService(skills = "travel") // 
public interface TravelAgent {

    @SystemMessage("You are a travel agent") // 
    String chat(String message);
}

  1. The service receives the activate_skill tool, which returns the instructions of a skill to the model.

  2. The available skills are listed after the system message.

17 Chat Language Models

The following modules provide integration with Langchain4j Language Models.

Each module configures one or more ChatModel beans, making them available for dependency injection based on configuration.

17.1 ChatModel Example

This example, asks a chat model to generate the list of the top 3 albums of a Jazz musician.

imports
package example.micronaut;

import dev.langchain4j.data.message.ChatMessage;
import dev.langchain4j.data.message.SystemMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.response.ChatResponse;
import jakarta.inject.Singleton;

import java.util.List;
@Singleton
public class MusicianAssistant {
    private static final SystemMessage SYSTEM_MSG = SystemMessage.from("""
      You are an expert in Jazz music.
      Reply with only the names of the artists, albums, etc.
      Be very concise.
      If a list is given, separate the items with commas.""");

    private final ChatModel model;

    public MusicianAssistant(ChatModel model) { // 
        this.model = model;
    }

    public Musician generateTopThreeAlbums(String name) {
        List<ChatMessage> messages = generateTopThreeAlbumsMessages(name);
        ChatResponse albums = model.chat(messages);
        String topThreeAlbums = albums.aiMessage().text();
        return new Musician(name, topThreeAlbums);
    }

    private static List<ChatMessage> generateTopThreeAlbumsMessages(String name) {
        return List.of(SYSTEM_MSG, UserMessage.from(
            String.format("Only list the top 3 albums of %s", name)
        ));
    }
}

  1. Inject via constructor injection a bean of type ChatModel.

imports
package example.micronaut;
public record Musician(String name, String albums) {
}

You can configure the chat model via configuration.

For example, you may want to configure OpenAI in the main classpath:

src/main/resources/application.properties
micronaut.application.name=micronaut-guide
langchain4j.open-ai.chat-model.log-requests=true
langchain4j.open-ai.chat-model.log-responses=true
langchain4j.open-ai.chat-model.timeout=60s
langchain4j.open-ai.chat-model.temperature=0.3
langchain4j.open-ai.chat-model.model-name=gpt-4.1

And a local SLM (Small Language Model) such as Ollama in the test classpath:

src/test/resources/application-test.properties
langchain4j.open-ai.enabled=false
langchain4j.ollama.model-name=tinyllama
langchain4j.ollama.chat-model.timeout=5m
# Ollama 0.32.10 changed the default from 1.1 to 1.0; without the penalty tinyllama ignores the chat history
langchain4j.ollama.chat-model.repeat-penalty=1.1
langchain4j.ollama.chat-model.log-requests=true
langchain4j.ollama.chat-model.log-responses=true

Moreover, you can also register a bean of type BeanCreatedEventListener to configure the Chat Model builder programmatically if configuration is not enough.

imports
package example.micronaut;

import dev.langchain4j.model.ollama.OllamaChatModel;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import org.jspecify.annotations.NonNull;
import jakarta.inject.Singleton;
@Singleton
class OllamaChatModelBuilderListener
    implements BeanCreatedEventListener<OllamaChatModel.OllamaChatModelBuilder> {
    @Override
    public OllamaChatModel.OllamaChatModelBuilder onCreated(
        @NonNull BeanCreatedEvent<OllamaChatModel.OllamaChatModelBuilder> event) {
        OllamaChatModel.OllamaChatModelBuilder builder = event.getBean();
        builder.temperature(0.0);
        return builder;
    }
}

17.2 Chat Memory

Models are stateless by design. Chat memory serves as container for previous messages, helping you maintain context in a conversation, but the model itself is not aware of this memory; it relies on you to include the relevant messages in each request for coherent and contextually relevant responses.

Langchain4J provides an API ChatMemory to help you manage chat memory. You can provide your own implementation or use one of the provided implementations.

The default implementation of ChatMemory, dev.langchain4j.store.memory.chat.InMemoryChatMemoryStore, stores ChatMessage instances in memory.

To use the Redis implementation dev.langchain4j.community.store.memory.chat.redis.RedisChatMemoryStore, add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-redis")

To use the Neo4J implementation dev.langchain4j.community.store.memory.chat.neo4j.Neo4jChatMemoryStore, add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-neo4j")

To use the Cassandra implementation dev.langchain4j.store.memory.chat.cassandra.CassandraChatMemoryStore, add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-cassandra")

To use the Oracle implementation dev.langchain4j.store.memory.chat.oracle.OracleChatMemoryStore, add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-oracle")

Then configure a JDBC datasource and the chat memory store properties, for example:

datasources.default.dialect=oracle
langchain4j.chat-memory-store.oracle.default.enabled=true
langchain4j.chat-memory-store.oracle.default.table-name=CHAT_MEMORY
langchain4j.chat-memory-store.oracle.default.memory-id-column-name=MEMORY_ID
langchain4j.chat-memory-store.oracle.default.content-column-name=CONTENT

The table must already exist. By default, the expected schema is CHAT_MEMORY(MEMORY_ID, CONTENT).

The segment under oracle (for example default) maps to the datasource name. For multiple datasources, configure multiple entries such as langchain4j.chat-memory-store.oracle.reporting.*.

To use the JDBC implementation dev.langchain4j.community.store.memory.chat.sql.SQLChatMemoryStore (PostgreSQL, MySQL, MariaDB or H2), add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-jdbc")

Then configure the chat memory store of a datasource. The SQL dialect is detected from the database, and auto-create-table creates the table if it does not exist:

langchain4j.chat-memory-store.jdbc.default.table-name=chat_memory
langchain4j.chat-memory-store.jdbc.default.auto-create-table=true

The segment under jdbc (for example default) is the name of the datasource.

Note
 For Oracle Database, use the dedicated Oracle chat memory store of micronaut-langchain4j-store-oracle described above rather than the JDBC store.

To use the MongoDB implementation dev.langchain4j.community.store.memory.chat.mongodb.MongoDbChatMemoryStore, add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-mongodb")

Then configure the database and the collection, and optionally expire-after-seconds to delete conversations after some time. The store uses the default MongoDB client, or the one of the server named:

langchain4j.chat-memory-store.mongodb.default.database-name=chat
langchain4j.chat-memory-store.mongodb.default.collection-name=memory

Chat Memory of the AI Services

The AI services whose methods have a @MemoryId parameter keep a memory per memory ID. Their default ChatMemoryProvider keeps the conversations in the chat memory store named by langchain4j.chat-memory.store, otherwise in the persistent chat memory store if the application has exactly one (Redis, Neo4j, Cassandra, Oracle, JDBC or MongoDB), otherwise in memory. A ChatMemoryProvider bean named after the service, or annotated with @Primary, replaces it.

By default the memory keeps the last 20 messages. A token window memory keeps the last messages that fit in a number of tokens instead, counted with the TokenCountEstimator bean of the application (for example OpenAiTokenCountEstimator):

langchain4j.chat-memory.type=token-window
langchain4j.chat-memory.max-tokens=4000

The following example shows how to use the ChatMemory:

imports
package example.micronaut;

import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.memory.ChatMemory;
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.response.ChatResponse;
import jakarta.inject.Singleton;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
@Singleton
public class AssistantWithMemory {
    private final Map<String, ChatMemory> conversations = new ConcurrentHashMap<>();
    private final ChatModel model;
    private final MessageWindowChatMemory.Builder messageWindowChatMemoryBuilder;

    public AssistantWithMemory(MessageWindowChatMemory.Builder messageWindowChatMemoryBuilder,
                               ChatModel model) {
        this.messageWindowChatMemoryBuilder = messageWindowChatMemoryBuilder;
        this.model = model;
    }

    public MemoryIdAndResponse chat(String conversationId, String message) {
        ChatMemory chatMemory = conversations.get(conversationId);
        if (chatMemory == null) {
            throw new IllegalArgumentException("Unknown conversation: " + conversationId);
        }
        chatMemory.add(UserMessage.from(message));
        ChatResponse chatResponse = model.chat(chatMemory.messages());
        AiMessage answer = chatResponse.aiMessage();
        chatMemory.add(answer);
        return new MemoryIdAndResponse(conversationId, answer.text());
    }

    public MemoryIdAndResponse chat(String message) {
        String conversationId = startConversation();
        return chat(conversationId, message);
    }

    private String startConversation() {
        String memoryId = generateChatMemoryId();
        ChatMemory chatMemory = generateChatMemory(memoryId);
        conversations.putIfAbsent(memoryId, chatMemory);
        return memoryId;
    }

    private String generateChatMemoryId() {
        return UUID.randomUUID().toString();
    }

    private ChatMemory generateChatMemory(String memoryId) {
        return messageWindowChatMemoryBuilder
            .id(memoryId)
            .build();
    }
}

You could invoke the previous class as illustrated in the following test:

@Test
void chatWithMemory(AssistantWithMemory assistant) {
    MemoryIdAndResponse johnConversation = assistant.chat("Let me introduce myself. My name is John");
    String johnConversationId = johnConversation.memoryId();
    assertNotNull(johnConversationId);
    MemoryIdAndResponse aegonConversation = assistant.chat("Let me introduce myself. My name is Dan");
    String aegonConversationId = aegonConversation.memoryId();
    assertNotNull(aegonConversationId);
    MemoryIdAndResponse answer = assistant.chat(johnConversationId, "What's my name?");
    assertTrue(answer.response().toLowerCase().contains("john"), answer.response());
    answer = assistant.chat(aegonConversationId, "What's my name?");
    assertTrue(answer.response().toLowerCase().contains("dan"), answer.response());
}

17.3 Anthropic

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-anthropic")

Then add the necessary configuration.

Example Configuration
langchain4j.anthropic.api-key=YOUR_KEY

Extended thinking and prompt caching are configured on the models:

Example Configuration
langchain4j.anthropic.api-key=${ANTHROPIC_API_KEY}
langchain4j.anthropic.chat-model.model-name=claude-sonnet-4-5
langchain4j.anthropic.chat-model.thinking-type=enabled
langchain4j.anthropic.chat-model.thinking-budget-tokens=2048
langchain4j.anthropic.chat-model.return-thinking=true
langchain4j.anthropic.chat-model.cache-system-messages=true
langchain4j.anthropic.chat-model.cache-tools=true

17.4 Azure

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-azure")

Then add the necessary configuration.

Example Configuration
langchain4j.azure-open-ai.api-key=YOUR_KEY
langchain4j.azure-open-ai.endpoint=YOUR_ENDPOINT

You will additionally need to define a bean of type TokenCredentials.

One way to do this is to include the Azure SDK module.

The audio transcription model (dev.langchain4j.model.audio.AudioTranscriptionModel) is only created when it is configured, for example with langchain4j.azure-open-ai.audio-transcription-model.deployment-name.

17.5 Bedrock

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-bedrock")

Then add the necessary configuration.

Example Configuration
langchain4j.bedrock.region=us-east-1
langchain4j.bedrock.chat-model.model-id=anthropic.claude-3-haiku-20240307-v1:0

The model-id is the Amazon Bedrock model ID or inference profile ID to invoke. There is no default value, choose a model that is enabled for your account and region.

You will additionally need to define a bean of type AwsCredentialsProvider.

One way to do this is to include the AWS SDK module.

17.6 HuggingFace

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-hugging-face")

The models use the Micronaut HTTP client. Add a Micronaut HTTP client implementation to your application, for example the default Netty implementation:

runtimeOnly("io.micronaut:micronaut-http-client")

Then add the necessary configuration.

Example Configuration
langchain4j.hugging-face.access-token=YOUR_ACCESS_TOKEN

17.7 Jlama

Jlama runs the models in the JVM, without a model server. Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-jlama")

Then configure the model, which Jlama downloads from Hugging Face into its working directory on the first use:

Example Configuration
langchain4j.jlama.chat-model.model-name=tjake/Llama-3.2-1B-Instruct-JQ4
langchain4j.jlama.embedding-model.model-name=intfloat/e5-small-v2
Note
 Jlama uses the incubating Vector API of the JDK: run the application with --add-modules jdk.incubator.vector --enable-preview. Jlama is not supported in a GraalVM native image.

17.8 MistralAi

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-mistralai")

Then add the necessary configuration.

Example Configuration
langchain4j.mistral-ai.api-key=YOUR_KEY

17.9 Ollama

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-ollama")

Then add the necessary configuration.

Example Configuration
langchain4j.ollama.base-url=YOUR_URL

17.10 Oracle Cloud GenAI

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-oci-genai")

Setup a supported OCI authentication method.

Then add the necessary configuration to configure a chat model.

Example Configuration
langchain4j.oci-gen-ai.chat-model.model-name=orca-mini
langchain4j.oci-gen-ai.compartment-id=your-compartment

17.11 OpenAi

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-openai")

Provider modules use Micronaut’s HTTP client API and exclude LangChain4j’s JDK HTTP client. Add a concrete Micronaut HTTP client implementation to your application, for example the default Netty implementation:

runtimeOnly("io.micronaut:micronaut-http-client")

For tests that instantiate OpenAI models, add the same implementation to the test runtime classpath:

testRuntimeOnly("io.micronaut:micronaut-http-client")

Then add the necessary configuration.

Example Configuration
langchain4j.open-ai.api-key=YOUR_KEY

Audio

The audio transcription model (dev.langchain4j.model.audio.AudioTranscriptionModel) and the text-to-speech model (dev.langchain4j.model.audio.TextToSpeechModel) are only created when they are configured:

Example Configuration
langchain4j.open-ai.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.audio-transcription-model.model-name=gpt-4o-mini-transcribe
langchain4j.open-ai.text-to-speech-model.model-name=gpt-4o-mini-tts
langchain4j.open-ai.text-to-speech-model.voice=alloy

Responses API

The chat models of the Responses API are configured under langchain4j.open-ai-responses, and share the API key and other common settings of langchain4j.open-ai. Since the Chat Completions model is the default OpenAI chat model, configure the Responses models with a name, and select them with the name, for example with @AiService(named = "assistant"):

Example Configuration
langchain4j.open-ai.api-key=${OPENAI_API_KEY}
langchain4j.open-ai-responses.chat-models.assistant.model-name=gpt-5-mini
langchain4j.open-ai-responses.streaming-chat-models.assistant.model-name=gpt-5-mini
Note
 The Responses API client of LangChain4j appends /responses to the base URL. The default base URL, https://api.openai.com/v1, suits both APIs; set a custom base-url (of langchain4j.open-ai or of the Responses models) without a trailing slash.

17.12 Google AI Gemini

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-googleai-gemini")

Then add the necessary configuration.

Example Configuration
langchain4j.google-ai-gemini.api-key=YOUR_API_KEY

17.13 Google Gen AI

The Google Gen AI module uses the Google Gen AI SDK, which works with both the Gemini API and Vertex AI. It provides a chat model, a streaming chat model, an embedding model and an image model. Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-google-genai")

To use the Gemini API, configure an API key:

Example Configuration
langchain4j.google-gen-ai.api-key=${GEMINI_API_KEY}
langchain4j.google-gen-ai.chat-model.model-name=gemini-3.8-flash

To use Vertex AI, configure the project and the location. The requests are authenticated with the GoogleCredentials bean of the application if there is one, otherwise with the application default credentials:

Example Configuration
langchain4j.google-gen-ai.vertex-ai=true
langchain4j.google-gen-ai.project-id=my-project
langchain4j.google-gen-ai.location=us-central1

The models share a com.google.genai.Client bean configured under langchain4j.google-gen-ai, including the connection pool of its HTTP client:

The SDK uses OkHttp. To customize the HTTP client, for example to add interceptors, register a BeanCreatedEventListener for ClientOptions.Builder (or HttpOptions.Builder):

imports
package example.micronaut.googlegenai;

import com.google.genai.types.ClientOptions;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import jakarta.inject.Singleton;
import okhttp3.OkHttpClient;

import java.time.Duration;
@Singleton
public class HttpClientCustomizer implements BeanCreatedEventListener<ClientOptions.Builder> { // 

    @Override
    public ClientOptions.Builder onCreated(BeanCreatedEvent<ClientOptions.Builder> event) {
        OkHttpClient httpClient = new OkHttpClient.Builder()
            .callTimeout(Duration.ofMinutes(2))
            .build();
        return event.getBean().customHttpClient(httpClient); // 
    }
}

  1. The listener is notified when the client options of the Google Gen AI client are created.

  2. It sets the OkHttp client that the SDK uses.

Note
 The labels of the requests are not configurable, because they are only supported by Vertex AI. Set them with a BeanCreatedEventListener for the model builder, for example GoogleGenAiChatModel.Builder.

17.14 VertexAi

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-vertexai")

Then add the necessary configuration.

To provide explicit Google Cloud credentials, register a GoogleCredentials bean. When no such bean is present, the Vertex AI client uses Application Default Credentials.

Example Configuration
langchain4j.vertex-ai.endpoint=YOUR_ENDPOINT
langchain4j.vertex-ai.model-name=YOUR_MODEL
langchain4j.vertex-ai.project=YOUR_PROJECT
langchain4j.vertex-ai.location=YOUR_LOCATION
langchain4j.vertex-ai.publisher=YOUR_PUBLISHER

17.15 VertexAi Gemini

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-vertexai-gemini")

Then add the necessary configuration.

To provide explicit Google Cloud credentials, register a GoogleCredentials bean. When no such bean is present, the Vertex AI Gemini client uses Application Default Credentials.

Example Configuration
langchain4j.vertex-ai-gemini.model-name=YOUR_MODEL
langchain4j.vertex-ai-gemini.project=YOUR_PROJECT
langchain4j.vertex-ai-gemini.location=YOUR_LOCATION

18 Scoring Models

Scoring models (also called re-ranking models) score the relevance of text segments to a query. They are used to re-rank the content retrieved for retrieval augmented generation, see Configuring Retrieval.

Each module configures a dev.langchain4j.model.scoring.ScoringModel bean under langchain4j.<provider>.scoring-model (and named ones under langchain4j.<provider>.scoring-models.<name>), and an EmbeddingModel bean under langchain4j.<provider>.embedding-model. The models use the Micronaut HTTP client.

18.1 Cohere

Add the following dependency to use the Cohere models:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-cohere")
Example Configuration
langchain4j.cohere.api-key=${COHERE_API_KEY}
langchain4j.cohere.scoring-model.model-name=rerank-v3.5
langchain4j.cohere.embedding-model.model-name=embed-v4.0

18.2 Jina

Add the following dependency to use the Jina models:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-jina")
Example Configuration
langchain4j.jina.api-key=${JINA_API_KEY}
langchain4j.jina.scoring-model.model-name=jina-reranker-v2-base-multilingual
langchain4j.jina.embedding-model.model-name=jina-embeddings-v3

18.3 Voyage AI

Add the following dependency to use the Voyage AI models:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-voyageai")
Example Configuration
langchain4j.voyage-ai.api-key=${VOYAGE_API_KEY}
langchain4j.voyage-ai.scoring-model.model-name=rerank-2.5
langchain4j.voyage-ai.embedding-model.model-name=voyage-3.5

19 Embedding Stores

19.1 In-Memory

An in-memory embedding store is enabled by default, set the following property langchain4j.in-memory.embedding-store.enabled with value false to disable it.

The store is not attached to AI services automatically. See the Retrieval Augmented Generation section of the AI Service guide to declare a ContentRetriever bean.

19.2 Chroma

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-chroma")
Example Configuration
langchain4j.chroma.embedding-store.base-url=http://localhost:8000
langchain4j.chroma.embedding-store.collection-name=documents
langchain4j.chroma.embedding-store.api-version=V2

19.3 Elastic Search

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-elasticsearch")

Then configure the Elasticsearch client.

Example Configuration
elasticsearch.httpHosts=http://localhost:9200,http://127.0.0.2:9200
langchain4j.elasticsearch.embedding-stores.default.dimension=384

19.4 Milvus

Add the following dependency to use the Milvus embedding store, based on LangChain4j’s Milvus V2 integration:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-milvus")
Example Configuration
langchain4j.milvus-v2.embedding-store.uri=http://localhost:19530
langchain4j.milvus-v2.embedding-store.collection-name=documents
langchain4j.milvus-v2.embedding-store.dimension=384
langchain4j.milvus-v2.embedding-store.consistency-level=STRONG

The dimension must match the dimension of the embeddings of your embedding model. Milvus is eventually consistent by default: set consistency-level to STRONG to search the embeddings right after they are added.

19.5 MongoDB

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-mongodb-atlas")
Configuring a MongoDB server
mongodb.servers.default.uri: mongodb://username:password@localhost:27017/databaseName
Example Configuration
langchain4j.mongodb-atlas.embedding-stores.default.database-name=testdb
langchain4j.mongodb-atlas.embedding-stores.default.collection-name=testcol
langchain4j.mongodb-atlas.embedding-stores.default.index-name=testindex

19.6 Neo4j

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-neo4j")

Then configure the Neo4j driver.

Example Configuration
neo4j.uri=bolt://localhost
langchain4j.neo4j.embedding-stores.default.dimension=384

19.7 Oracle

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-oracle")

Then add one of the supported JDBC connection pools, for example Hikari:

runtimeOnly("io.micronaut.sql:micronaut-jdbc-hikari")
Example Configuration
datasources.default.dialect=oracle
langchain4j.oracle.embedding-stores.default.table=test
langchain4j.oracle.embedding-stores.default.table.create-option=create_if_not_exists

19.8 Open Search

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-opensearch")

Then configure the OpenSearch client.

Example Configuration
micronaut.opensearch.rest-client.http-hosts=http://localhost:9200,http://127.0.0.2:9200
langchain4j.opensearch.embedding-stores.default.dimension=384

19.9 PGVector

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-pgvector")

Then add one of the supported JDBC connection pools, for example Hikari:

runtimeOnly("io.micronaut.sql:micronaut-jdbc-hikari")
Example Configuration
datasources.default.dialect=postgres
langchain4j.pgvector.embedding-stores.default.table=mytable
langchain4j.pgvector.embedding-stores.default.dimension=384
test-resources.containers.postgres.image-name=pgvector/pgvector:pg16

19.10 Redis

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-redis")
Example Configuration
langchain4j.redis.embedding-store.host=localhost
langchain4j.redis.embedding-store.port=6379
langchain4j.redis.embedding-stores.default.dimension=384

19.11 Qdrant

Add the following dependency:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-qdrant")

To use Testcontainers & Test Resources add the following dependency:

testResourcesService("io.micronaut.langchain4j:micronaut-langchain4j-qdrant-testresource")
Note
 For Maven see Adding modules or libraries to the Test Resources service classpath.
Example Configuration
langchain4j.qdrant.embedding-store.host=localhost
langchain4j.qdrant.embedding-store.port=6334
langchain4j.qdrant.embedding-store.collection-name=mycollection

19.12 Weaviate

Add the following dependency to use the Weaviate embedding store:

implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-weaviate")
Example Configuration
langchain4j.weaviate.embedding-store.scheme=https
langchain4j.weaviate.embedding-store.host=my-cluster.weaviate.network
langchain4j.weaviate.embedding-store.api-key=${WEAVIATE_API_KEY}
langchain4j.weaviate.embedding-store.object-class=Document

20 Breaking Changes

This section documents breaking changes between Micronaut LangChain4j versions.

2.4.0

Updated default model names

The default model names of several providers referenced models that are retired or about to be retired. They now default to current models:

Provider Model type Previous default New default

Anthropic

Chat

claude-3-haiku-20240307

claude-haiku-4-5-20251001

Mistral AI

Chat

mistral-tiny

mistral-small-latest

OpenAI

Chat

gpt-3.5-turbo

gpt-4.1-mini

OpenAI

Embedding

text-embedding-ada-002

text-embedding-3-small

OpenAI

Image

dall-e-3

gpt-image-1

Vertex AI Gemini

Chat

gemini-1.5-flash-001

gemini-3.8-flash

Set the model-name property of the model to keep using a previous model. Note that embeddings produced by different embedding models are not compatible, so applications that stored embeddings created with the previous OpenAI default must either set model-name: text-embedding-ada-002 or re-embed their content.

Bedrock model property removed

The langchain4j.bedrock.model property was never applied to the Bedrock models. It has been removed. Configure the model with langchain4j.bedrock.chat-model.model-id (and the equivalent property of the streaming chat and embedding models) instead.

Chat memory of the AI services

The default chat memory provider of the AI services kept the conversations in memory, even when a persistent chat memory store (Redis, Neo4j, Cassandra, Oracle) was configured. It now keeps them in the persistent chat memory store if the application has exactly one. Set langchain4j.chat-memory.store: inMemory to keep them in memory.

Hugging Face uses the Micronaut HTTP client

The Hugging Face models use the Micronaut HTTP client instead of LangChain4j’s JDK HTTP client, like the OpenAI, Anthropic, Ollama, Mistral AI and Google AI Gemini models. Add a Micronaut HTTP client implementation, such as io.micronaut:micronaut-http-client, to the runtime classpath.

LangChain4j Community 1.22

The LangChain4j community modules (Redis, Neo4j, OCI GenAI) have been upgraded from 1.16.0-beta26 to 1.22.0-beta32, aligned with the LangChain4j 1.22.0 BOM. Refer to the LangChain4j Community release notes for changes.

Nullness annotations

Every package of Micronaut LangChain4j is now annotated with JSpecify @NullMarked: the types of the API are non-null, unless annotated with @Nullable. Kotlin reads these annotations, so Kotlin code sees non-null and nullable types where it previously saw platform types. For example, the getters of the optional properties of PgVectorEmbeddingStoreConfig now return nullable types.

21 Repository

You can find the source code of this project in this repository:

https://github.com/micronaut-projects/micronaut-langchain4j

22 Release History

For this project, you can find a list of releases (with release notes) here:

https://github.com/micronaut-projects/micronaut-langchain4j/releases