annotationProcessor("io.micronaut.langchain4j:micronaut-langchain4j-processor")
Table of Contents
Micronaut LangChain4j
Integration between Micronaut and Langchain4j
Version: 2.4.0
1 Introduction
This module provides integration between Micronaut and Langchain4j.
| This module is regarded as experimental and subject to change since the underlying technology (AI) is volatile and subject to change. |
Various modules are provided that allow automatically configuring common Langchain4j types like ChatModel, ImageModel etc. Refer to the sections below for the supported Langchain4j extensions.
2 Quick Start
Add the following annotation processor dependency:
<annotationProcessorPaths>
<path>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-processor</artifactId>
</path>
</annotationProcessorPaths>[tool.pyronaut.dependencies]
build = [
"io.micronaut.langchain4j:micronaut-langchain4j-processor",
]Then the core module:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-core")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-core</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-core",
]You are now ready to configure one of the Chat Language Models, for the quick start we will use Ollama:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-ollama")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-ollama</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-ollama",
]To test the integration add the test resources integration to your Maven build or Gradle build.
testResourcesService("io.micronaut.langchain4j:micronaut-langchain4j-ollama-testresource")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-ollama-testresource</artifactId>
<scope>testResourcesService</scope>
</dependency>[tool.pyronaut.dependencies]
test = [
"io.micronaut.langchain4j:micronaut-langchain4j-ollama-testresource",
]Add the necessary configuration to configure the model name you want to use:
Configuring the Model Name
langchain4j.ollama.model-name=orca-minilangchain4j.ollama.model-name: orca-mini"langchain4j.ollama.model-name" = "orca-mini"langchain4j.ollama.modelName = "orca-mini"{
"langchain4j.ollama.model-name" = "orca-mini"
}{
"langchain4j.ollama.model-name": "orca-mini"
}3 Response Streaming
It is possible to use response streaming.
First, you need to configure a streaming chat model with langchain4j.*.streaming-chat-model.*. For example, with OpenAI:
Example Configuration
langchain4j.open-ai.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.streaming-chat-model.model-name=gpt-4o
langchain4j.open-ai.streaming-chat-model.log-requests=true
langchain4j.open-ai.streaming-chat-model.log-responses=truelangchain4j:
open-ai:
api-key: ${OPENAI_API_KEY}
streaming-chat-model:
model-name: gpt-4o
log-requests: true
log-responses: true[langchain4j.open-ai]
api-key = "${OPENAI_API_KEY}"
[langchain4j.open-ai.streaming-chat-model]
model-name = "gpt-4o"
log-requests = true
log-responses = truelangchain4j {
openAi {
apiKey = "${OPENAI_API_KEY}"
streamingChatModel {
modelName = "gpt-4o"
logRequests = true
logResponses = true
}
}
}{
langchain4j {
open-ai {
api-key = "${OPENAI_API_KEY}"
streaming-chat-model {
model-name = "gpt-4o"
log-requests = true
log-responses = true
}
}
}
}{
"langchain4j": {
"open-ai": {
"api-key": "${OPENAI_API_KEY}",
"streaming-chat-model": {
"model-name": "gpt-4o",
"log-requests": true,
"log-responses": true
}
}
}
}Then, you will be able to inject a bean of type dev.langchain4j.model.chat.StreamingChatModel.
Asynchronous and Reactive AI Services
The methods of an AI Service can return asynchronous and reactive types:
package example.micronaut.aiservice.reactive;
import dev.langchain4j.service.SystemMessage;
import io.micronaut.langchain4j.annotation.AiService;
import org.reactivestreams.Publisher;
import java.util.concurrent.CompletableFuture;
@AiService
public interface AsyncFriend {
@SystemMessage("You are a good friend of mine. Answer using slang.")
CompletableFuture<String> chat(String userMessage); // (1)
@SystemMessage("You are a good friend of mine. Answer using slang.")
Publisher<String> stream(String userMessage); // (2)
}from abc import ABC, abstractmethod
from dev.langchain4j.service import SystemMessage
from java.util.concurrent import CompletableFuture
from micronaut.langchain4j.annotation import AiService
from org.reactivestreams import Publisher
@AiService
class AsyncFriend(ABC):
@SystemMessage("You are a good friend of mine. Answer using slang.")
@abstractmethod
def chat(self, user_message: str) -> CompletableFuture[str]: # (1)
...
@SystemMessage("You are a good friend of mine. Answer using slang.")
@abstractmethod
def stream(self, user_message: str) -> Publisher[str]: # (2)
...package example.micronaut.aiservice.reactive
import dev.langchain4j.service.SystemMessage
import io.micronaut.langchain4j.annotation.AiService
import org.reactivestreams.Publisher
import java.util.concurrent.CompletableFuture
@AiService
interface AsyncFriend {
@SystemMessage("You are a good friend of mine. Answer using slang.")
fun chat(userMessage: String): CompletableFuture<String> // (1)
@SystemMessage("You are a good friend of mine. Answer using slang.")
fun stream(userMessage: String): Publisher<String> // (2)
}package example.micronaut.aiservice.reactive
import dev.langchain4j.service.SystemMessage
import io.micronaut.langchain4j.annotation.AiService
import org.reactivestreams.Publisher
import java.util.concurrent.CompletableFuture
@AiService
interface AsyncFriend {
@SystemMessage("You are a good friend of mine. Answer using slang.")
CompletableFuture<String> chat(String userMessage) // (1)
@SystemMessage("You are a good friend of mine. Answer using slang.")
Publisher<String> stream(String userMessage) // (2)
}| 1 | A CompletableFuture (or CompletionStage) completes with the response of the chat model. |
| 2 | A Reactive Streams Publisher<String> emits the partial responses of the streaming chat model. |
The following return types are supported:
| Return type | Model | Notes |
|---|---|---|
|
Streaming chat model |
LangChain4j’s callback API |
|
Streaming chat model |
Works with every streaming chat model |
|
Chat model |
Requires a chat model with a non-blocking implementation ( |
|
Streaming chat model |
Requires a streaming chat model with a non-blocking implementation |
The AI service uses the chat model, the streaming chat model or both, depending on the return types of its methods.
The non-blocking implementations of LangChain4j 1.21 are those of the OpenAI and Anthropic models, which use the Micronaut HTTP client without blocking a thread. Asynchronous methods also run the tools without blocking when the @Tool methods return a CompletableFuture or a CompletionStage.
In Kotlin, add the dev.langchain4j:langchain4j-kotlin dependency to return a kotlinx.coroutines.flow.Flow<String> from an AI service method.
|
4 AI Service
You can also define new AI services:
Defining
@AiService interfacespackage example.micronaut.aiservice;
import dev.langchain4j.service.SystemMessage;
import io.micronaut.langchain4j.annotation.AiService;
@AiService // (1)
public interface Friend {
@SystemMessage("You are a good friend of mine. Answer using slang.") // (2)
String chat(String userMessage);
}Defining
@AiService interfacesfrom abc import ABC, abstractmethod
from dev.langchain4j.service import SystemMessage
from micronaut.langchain4j.annotation import AiService
@AiService # (1)
class Friend(ABC):
@SystemMessage("You are a good friend of mine. Answer using slang.") # (2)
@abstractmethod
def chat(self, user_message: str) -> str:
...Defining
@AiService interfacespackage example.micronaut.aiservice
import dev.langchain4j.service.SystemMessage
import io.micronaut.langchain4j.annotation.AiService
@AiService // (1)
interface Friend {
@SystemMessage("You are a good friend of mine. Answer using slang.") // (2)
fun chat(userMessage: String): String
}Defining
@AiService interfacespackage example.micronaut.aiservice
import dev.langchain4j.service.SystemMessage
import io.micronaut.langchain4j.annotation.AiService
@AiService // (1)
interface Friend {
@SystemMessage("You are a good friend of mine. Answer using slang.") // (2)
String chat(String userMessage)
}| 1 | Define an interface annotated with @AiService |
| 2 | Use Langchain4j annotations like @SystemMessage |
In Python an AI service is an abstract class (abc.ABC) whose methods are declared with … bodies; the decorators mirror the LangChain4j annotations.
|
LangChain4j builds the service implementation reflectively from the Java interface generated for the Python class and reads the @SystemMessage/@UserMessage annotations from it. @AiService implies the @AllowsReflection hint (io.micronaut.core.annotation), so the Python compiler copies the annotations onto the generated interface without any @AllowsReflection or micronaut.introspection.allow-reflection option.
|
You can now inject the @AiService definition into any Micronaut component including tests:
Calling
@AiService definitionspackage example.micronaut.aiservice;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import dev.langchain4j.model.chat.ChatModel;
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.testcontainers.junit.jupiter.Testcontainers;
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class AiServiceTest implements OllamaTestPropertyProvider {
@Test
void testAiService(Friend friend, ChatModel languageModel) {
String result = friend.chat("Hello");
assertNotNull(result);
assertNotNull(languageModel);
}
}Calling
@AiService definitionsfrom typing import Annotated
from dev.langchain4j.model.chat import ChatModel
from jakarta.inject import Inject
from micronaut.test.extensions.junit5.annotation import MicronautTest
from org.junit.jupiter.api import Test
from example.micronaut.aiservice.Friend import Friend
@MicronautTest(startApplication=False, environments=["ollama"])
class AiServiceTest:
friend: Annotated[Friend, Inject]
language_model: Annotated[ChatModel, Inject]
@Test
def test_ai_service(self):
result = self.friend.chat("Hello")
assert result is not None
assert self.language_model is not NoneCalling
@AiService definitionspackage example.micronaut.aiservice
import dev.langchain4j.model.chat.ChatModel
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import org.junit.jupiter.api.Assertions
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import org.testcontainers.junit.jupiter.Testcontainers
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
internal class AiServiceTest : OllamaTestPropertyProvider {
@Test
fun testAiService(friend: Friend, languageModel: ChatModel) {
val result: String = friend.chat("Hello")
Assertions.assertNotNull(result)
Assertions.assertNotNull(languageModel)
}
}Calling
@AiService definitionspackage example.micronaut.aiservice
import dev.langchain4j.model.chat.ChatModel
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import org.testcontainers.junit.jupiter.Testcontainers
import static org.junit.jupiter.api.Assertions.assertNotNull
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class AiServiceTest implements OllamaTestPropertyProvider {
@Test
void testAiService(Friend friend, ChatModel languageModel) {
String result = friend.chat("Hello")
assertNotNull(result)
assertNotNull(languageModel)
}
}Retrieval Augmented Generation
Retrieval is opt-in. An @AiService performs retrieval augmented generation (RAG) only when the application declares one of:
-
a
dev.langchain4j.rag.RetrievalAugmentorbean, or -
a
dev.langchain4j.rag.content.retriever.ContentRetrieverbean, which LangChain4j wraps in aDefaultRetrievalAugmentor.
A RetrievalAugmentor bean takes precedence over a ContentRetriever bean. For either type, a bean @Named after the service (for example @Named("expert") for @AiService("expert")) is preferred over the default bean. If several candidates exist and none is named after the service or marked @Primary, no retrieval is configured.
Defining an
@AiService that returns Result<String>package example.micronaut.aiservice.rag;
import dev.langchain4j.service.Result;
import io.micronaut.context.annotation.Requires;
import io.micronaut.langchain4j.annotation.AiService;
@Requires(property = "spec.name", value = "RagTest")
@AiService // (1)
public interface Expert {
Result<String> ask(String question); // (2)
}Defining an
@AiService that returns Result<String>from abc import ABC, abstractmethod
from dev.langchain4j.service import Result
from micronaut.context.annotation import Requires
from micronaut.langchain4j.annotation import AiService
@Requires(property="spec.name", value="RagTest")
@AiService # (1)
class Expert(ABC):
@abstractmethod
def ask(self, question: str) -> Result[str]: # (2)
...Defining an
@AiService that returns Result<String>package example.micronaut.aiservice.rag
import dev.langchain4j.service.Result
import io.micronaut.context.annotation.Requires
import io.micronaut.langchain4j.annotation.AiService
@Requires(property = "spec.name", value = "RagTest")
@AiService // (1)
interface Expert {
fun ask(question: String): Result<String> // (2)
}Defining an
@AiService that returns Result<String>package example.micronaut.aiservice.rag
import dev.langchain4j.service.Result
import io.micronaut.context.annotation.Requires
import io.micronaut.langchain4j.annotation.AiService
@Requires(property = "spec.name", value = "RagTest")
@AiService // (1)
interface Expert {
Result<String> ask(String question) // (2)
}| 1 | A plain @AiService; nothing on the interface refers to retrieval. |
| 2 | Returning dev.langchain4j.service.Result exposes the retrieved sources alongside the answer. |
Declaring a
ContentRetriever beanpackage example.micronaut.aiservice.rag;
import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.rag.content.retriever.ContentRetriever;
import dev.langchain4j.rag.content.retriever.EmbeddingStoreContentRetriever;
import dev.langchain4j.store.embedding.EmbeddingStore;
import io.micronaut.context.annotation.Factory;
import io.micronaut.context.annotation.Requires;
import jakarta.inject.Singleton;
@Requires(property = "spec.name", value = "RagTest")
@Factory
class RetrievalFactory {
@Singleton // (1)
ContentRetriever contentRetriever(EmbeddingStore<TextSegment> embeddingStore, EmbeddingModel embeddingModel) { // (2)
return EmbeddingStoreContentRetriever.builder()
.embeddingStore(embeddingStore)
.embeddingModel(embeddingModel)
.maxResults(3)
.build();
}
}Declaring a
ContentRetriever beanfrom dev.langchain4j.data.segment import TextSegment
from dev.langchain4j.model.embedding import EmbeddingModel
from dev.langchain4j.rag.content.retriever import ContentRetriever, EmbeddingStoreContentRetriever
from dev.langchain4j.store.embedding import EmbeddingStore
from jakarta.inject import Singleton
from micronaut.context.annotation import Factory, Requires
@Requires(property="spec.name", value="RagTest")
@Factory
class RetrievalFactory:
@Singleton # (1)
def contentRetriever(self, embedding_store: EmbeddingStore[TextSegment], embedding_model: EmbeddingModel) -> ContentRetriever: # (2)
return EmbeddingStoreContentRetriever.builder() \
.embeddingStore(embedding_store) \
.embeddingModel(embedding_model) \
.maxResults(3) \
.build()Declaring a
ContentRetriever beanpackage example.micronaut.aiservice.rag
import dev.langchain4j.data.segment.TextSegment
import dev.langchain4j.model.embedding.EmbeddingModel
import dev.langchain4j.rag.content.retriever.ContentRetriever
import dev.langchain4j.rag.content.retriever.EmbeddingStoreContentRetriever
import dev.langchain4j.store.embedding.EmbeddingStore
import io.micronaut.context.annotation.Factory
import io.micronaut.context.annotation.Requires
import jakarta.inject.Singleton
@Requires(property = "spec.name", value = "RagTest")
@Factory
internal class RetrievalFactory {
@Singleton // (1)
fun contentRetriever(embeddingStore: EmbeddingStore<TextSegment>, embeddingModel: EmbeddingModel): ContentRetriever = // (2)
EmbeddingStoreContentRetriever.builder()
.embeddingStore(embeddingStore)
.embeddingModel(embeddingModel)
.maxResults(3)
.build()
}Declaring a
ContentRetriever beanpackage example.micronaut.aiservice.rag
import dev.langchain4j.data.segment.TextSegment
import dev.langchain4j.model.embedding.EmbeddingModel
import dev.langchain4j.rag.content.retriever.ContentRetriever
import dev.langchain4j.rag.content.retriever.EmbeddingStoreContentRetriever
import dev.langchain4j.store.embedding.EmbeddingStore
import groovy.transform.CompileStatic
import io.micronaut.context.annotation.Factory
import io.micronaut.context.annotation.Requires
import jakarta.inject.Singleton
@Requires(property = "spec.name", value = "RagTest")
@Factory
@CompileStatic
class RetrievalFactory {
@Singleton // (1)
ContentRetriever contentRetriever(EmbeddingStore<TextSegment> embeddingStore, EmbeddingModel embeddingModel) { // (2)
EmbeddingStoreContentRetriever.builder()
.embeddingStore(embeddingStore)
.embeddingModel(embeddingModel)
.maxResults(3)
.build()
}
}| 1 | The bean is picked up by every @AiService that has no bean named after it. Add @Named to target one service. |
| 2 | Build the retriever from the configured EmbeddingStore and EmbeddingModel. |
Retrieved content reaches the model and the result
package example.micronaut.aiservice.rag;
import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.service.Result;
import dev.langchain4j.store.embedding.EmbeddingStore;
import io.micronaut.context.annotation.Property;
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.testcontainers.junit.jupiter.Testcontainers;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
@Property(name = "spec.name", value = "RagTest")
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class RagTest implements OllamaTestPropertyProvider {
@Test
void retrievedContentIsPassedToTheModel(Expert expert, EmbeddingStore<TextSegment> embeddingStore, EmbeddingModel embeddingModel) {
TextSegment fact = TextSegment.from("Micronaut LangChain4j integrates LangChain4j with the Micronaut framework.");
embeddingStore.add(embeddingModel.embed(fact).content(), fact); // (1)
Result<String> result = expert.ask("What does Micronaut LangChain4j do?"); // (2)
assertNotNull(result.content());
assertTrue(result.sources().stream().anyMatch(content -> content.textSegment().text().equals(fact.text()))); // (3)
}
}Retrieved content reaches the model and the result
from typing import Annotated
from dev.langchain4j.data.segment import TextSegment
from dev.langchain4j.model.embedding import EmbeddingModel
from dev.langchain4j.store.embedding import EmbeddingStore
from jakarta.inject import Inject
from micronaut.context.annotation import Property
from micronaut.test.extensions.junit5.annotation import MicronautTest
from org.junit.jupiter.api import Test
from example.micronaut.aiservice.rag.Expert import Expert
@Property(name="spec.name", value="RagTest")
@MicronautTest(startApplication=False, environments=["scripted-test"])
class RagTest:
expert: Annotated[Expert, Inject]
embedding_store: Annotated[EmbeddingStore[TextSegment], Inject]
embedding_model: Annotated[EmbeddingModel, Inject]
@Test
def test_retrieved_content_is_passed_to_the_model(self):
fact = TextSegment.from_("Micronaut LangChain4j integrates LangChain4j with the Micronaut framework.")
self.embedding_store.add(self.embedding_model.embed(fact).content(), fact) # (1)
result = self.expert.ask("What does Micronaut LangChain4j do?") # (2)
assert result.content() is not None
assert any(source.textSegment().text() == fact.text() for source in result.sources()) # (3)Retrieved content reaches the model and the result
package example.micronaut.aiservice.rag
import dev.langchain4j.data.segment.TextSegment
import dev.langchain4j.model.embedding.EmbeddingModel
import dev.langchain4j.store.embedding.EmbeddingStore
import io.micronaut.context.annotation.Property
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import org.junit.jupiter.api.Assertions.assertNotNull
import org.junit.jupiter.api.Assertions.assertTrue
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import org.testcontainers.junit.jupiter.Testcontainers
@Property(name = "spec.name", value = "RagTest")
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
internal class RagTest : OllamaTestPropertyProvider {
@Test
fun retrievedContentIsPassedToTheModel(expert: Expert, embeddingStore: EmbeddingStore<TextSegment>, embeddingModel: EmbeddingModel) {
val fact = TextSegment.from("Micronaut LangChain4j integrates LangChain4j with the Micronaut framework.")
embeddingStore.add(embeddingModel.embed(fact).content(), fact) // (1)
val result = expert.ask("What does Micronaut LangChain4j do?") // (2)
assertNotNull(result.content())
assertTrue(result.sources().any { it.textSegment().text() == fact.text() }) // (3)
}
}Retrieved content reaches the model and the result
package example.micronaut.aiservice.rag
import dev.langchain4j.data.segment.TextSegment
import dev.langchain4j.model.embedding.EmbeddingModel
import dev.langchain4j.service.Result
import dev.langchain4j.store.embedding.EmbeddingStore
import io.micronaut.context.annotation.Property
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import org.testcontainers.junit.jupiter.Testcontainers
import static org.junit.jupiter.api.Assertions.assertNotNull
import static org.junit.jupiter.api.Assertions.assertTrue
@Property(name = "spec.name", value = "RagTest")
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class RagTest implements OllamaTestPropertyProvider {
@Test
void retrievedContentIsPassedToTheModel(Expert expert, EmbeddingStore<TextSegment> embeddingStore, EmbeddingModel embeddingModel) {
TextSegment fact = TextSegment.from("Micronaut LangChain4j integrates LangChain4j with the Micronaut framework.")
embeddingStore.add(embeddingModel.embed(fact).content(), fact) // (1)
Result<String> result = expert.ask("What does Micronaut LangChain4j do?") // (2)
assertNotNull(result.content())
assertTrue(result.sources().any { it.textSegment().text() == fact.text() }) // (3)
}
}| 1 | Ingest a segment into the embedding store. |
| 2 | Call the service as usual. |
| 3 | The retrieved segment is available as a source of the result. |
To configure a single service, declare an AiServiceCustomizer bean for it and call contentRetriever(…) or retrievalAugmentor(…) on AiServiceCreationContext.aiServices(), see Customizing AI Services.
Before 2.3.0 an EmbeddingStoreContentRetriever was attached to every @AiService automatically whenever an EmbeddingModel bean and an EmbeddingStore bean existed. Because provider modules expose an embedding model and the in-memory embedding store is enabled by default, that was the case in most applications, causing an embedding round trip on every call and failures against models without embedding support. Declare a ContentRetriever bean as shown above to restore the previous behaviour.
|
Configuring Retrieval
Content retrievers and the ingestion of documents can also be configured, without code.
A content retriever configured under langchain4j.rag.content-retrievers.<name> is an EmbeddingStoreContentRetriever bean qualified with <name>, so the @AiService with that name uses it:
Example Configuration
langchain4j.rag.content-retrievers.expert.max-results=3
langchain4j.rag.content-retrievers.expert.min-score=0.6langchain4j:
rag:
content-retrievers:
expert:
max-results: 3
min-score: 0.6[langchain4j.rag.content-retrievers.expert]
max-results = 3
min-score = 0.6langchain4j {
rag {
contentRetrievers {
expert {
maxResults = 3
minScore = 0.6
}
}
}
}{
langchain4j {
rag {
content-retrievers {
expert {
max-results = 3
min-score = 0.6
}
}
}
}
}{
"langchain4j": {
"rag": {
"content-retrievers": {
"expert": {
"max-results": 3,
"min-score": 0.6
}
}
}
}
}| Property | Type | Description | Default value |
|---|---|---|---|
|
java.lang.String |
The name of the embedding store bean, or {@code null} for the default embedding store |
|
|
java.lang.String |
The name of the embedding model bean, or {@code null} for the default embedding model |
|
|
int |
The maximum number of segments retrieved |
|
|
java.lang.Double |
The minimum relevance score of the segments retrieved |
An ingestion configured under langchain4j.rag.ingestion.<name> loads the documents of its sources when the application starts, splits them into segments, embeds them and adds them to the embedding store. The sources are directories of the file system, or of the classpath with the classpath: prefix. The documents are parsed with the DocumentParser that LangChain4j finds on the classpath (for example dev.langchain4j:langchain4j-document-parser-apache-tika for PDF and Office documents), and as text otherwise.
Example Configuration
langchain4j.rag.ingestion.expert.sources[0]=classpath:docs
langchain4j.rag.ingestion.expert.sources[1]=/var/data/manuals
langchain4j.rag.ingestion.expert.glob=**.md
langchain4j.rag.ingestion.expert.max-segment-size=800
langchain4j.rag.ingestion.expert.max-overlap-size=80langchain4j:
rag:
ingestion:
expert:
sources:
- classpath:docs
- /var/data/manuals
glob: '**.md'
max-segment-size: 800
max-overlap-size: 80[langchain4j.rag.ingestion.expert]
sources = ["classpath:docs", "/var/data/manuals"]
glob = "**.md"
max-segment-size = 800
max-overlap-size = 80langchain4j {
rag {
ingestion {
expert {
sources = ["classpath:docs", "/var/data/manuals"]
glob = "**.md"
maxSegmentSize = 800
maxOverlapSize = 80
}
}
}
}{
langchain4j {
rag {
ingestion {
expert {
sources = ["classpath:docs", "/var/data/manuals"]
glob = "**.md"
max-segment-size = 800
max-overlap-size = 80
}
}
}
}
}{
"langchain4j": {
"rag": {
"ingestion": {
"expert": {
"sources": ["classpath:docs", "/var/data/manuals"],
"glob": "**.md",
"max-segment-size": 800,
"max-overlap-size": 80
}
}
}
}
}| Property | Type | Description | Default value |
|---|---|---|---|
|
java.lang.String |
The name of the embedding store bean, or {@code null} for the default embedding store |
|
|
java.lang.String |
The name of the embedding model bean, or {@code null} for the default embedding model |
|
|
java.util.List |
The locations of the documents |
|
|
java.lang.String |
The glob the documents match |
|
|
boolean |
Whether the documents of the sub-directories of the file system sources are loaded |
|
|
int |
The maximum size of a segment, in characters |
|
|
int |
The maximum overlap of two consecutive segments, in characters |
|
|
boolean |
Whether the documents of the sources are ingested when the application starts |
Both use the default EmbeddingStore and EmbeddingModel beans unless embedding-store and embedding-model name other beans. Each ingestion also creates an EmbeddingStoreIngestor bean qualified with its name, to ingest other documents programmatically. Set ingest-on-startup to false to only create the ingestor.
The in-memory embedding store keeps the segments in memory only, so the documents are ingested again on every start. With a persistent embedding store, disable ingest-on-startup once the documents have been ingested, or ingest them with the EmbeddingStoreIngestor bean when they change.
|
A retrieval augmentor configured under langchain4j.rag.retrieval-augmentors.<name> retrieves content from one or more content retrievers and re-ranks it with a scoring model, so that only the most relevant content reaches the model. It is a RetrievalAugmentor bean qualified with <name>, which takes precedence over a content retriever with the same name:
Example Configuration
langchain4j.cohere.api-key=${COHERE_API_KEY}
langchain4j.cohere.scoring-model.model-name=rerank-v3.5
langchain4j.rag.content-retrievers.expert.max-results=10
langchain4j.rag.retrieval-augmentors.expert.re-rank=true
langchain4j.rag.retrieval-augmentors.expert.min-score=0.7
langchain4j.rag.retrieval-augmentors.expert.max-results=3langchain4j:
cohere:
api-key: ${COHERE_API_KEY}
scoring-model:
model-name: rerank-v3.5
rag:
content-retrievers:
expert:
max-results: 10
retrieval-augmentors:
expert:
re-rank: true
min-score: 0.7
max-results: 3[langchain4j.cohere]
api-key = "${COHERE_API_KEY}"
[langchain4j.cohere.scoring-model]
model-name = "rerank-v3.5"
[langchain4j.rag.content-retrievers.expert]
max-results = 10
[langchain4j.rag.retrieval-augmentors.expert]
re-rank = true
min-score = 0.7
max-results = 3langchain4j {
cohere {
apiKey = "${COHERE_API_KEY}"
scoringModel {
modelName = "rerank-v3.5"
}
}
rag {
contentRetrievers {
expert {
maxResults = 10
}
}
retrievalAugmentors {
expert {
reRank = true
minScore = 0.7
maxResults = 3
}
}
}
}{
langchain4j {
cohere {
api-key = "${COHERE_API_KEY}"
scoring-model {
model-name = "rerank-v3.5"
}
}
rag {
content-retrievers {
expert {
max-results = 10
}
}
retrieval-augmentors {
expert {
re-rank = true
min-score = 0.7
max-results = 3
}
}
}
}
}{
"langchain4j": {
"cohere": {
"api-key": "${COHERE_API_KEY}",
"scoring-model": {
"model-name": "rerank-v3.5"
}
},
"rag": {
"content-retrievers": {
"expert": {
"max-results": 10
}
},
"retrieval-augmentors": {
"expert": {
"re-rank": true,
"min-score": 0.7,
"max-results": 3
}
}
}
}
}The content retriever retrieves 10 segments by embedding similarity, and the scoring model passes the 3 most relevant ones with a score of at least 0.7 to the model.
| Property | Type | Description | Default value |
|---|---|---|---|
|
java.util.List |
The names of the content retriever beans |
|
|
java.lang.String |
The name of the scoring model bean that re-ranks the content |
|
|
boolean |
Whether the content is re-ranked |
|
|
java.lang.Double |
The minimum score of the re-ranked content |
|
|
java.lang.Integer |
The maximum number of re-ranked contents passed to the model |
Multimodal Parameters
An AI service can send images, PDF files, audio and video to the models that support them, together with the user message. Annotate a String, URI or URL parameter, or a collection of them, with @ImageUrl, @PdfUrl, @AudioUrl or @VideoUrl: the argument is sent as the LangChain4j content of that type.
package example.micronaut.aiservice.multimodal;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import io.micronaut.langchain4j.annotation.AiService;
import io.micronaut.langchain4j.annotation.ImageUrl;
import io.micronaut.langchain4j.annotation.PdfUrl;
import java.net.URI;
@AiService
public interface PictureDescriber {
@SystemMessage("You describe pictures for visually impaired people")
String describe(@UserMessage String question,
@UserMessage @ImageUrl String picture); // (1)
@UserMessage("Summarize the {{topic}} of this document")
String summarize(@V("topic") String topic,
@UserMessage @PdfUrl URI document); // (2)
}from abc import ABC, abstractmethod
from typing import Annotated
from dev.langchain4j.service import SystemMessage, UserMessage, V
from java.net import URI
from micronaut.langchain4j.annotation import AiService, ImageUrl, PdfUrl
@AiService
class PictureDescriber(ABC):
@SystemMessage("You describe pictures for visually impaired people")
@abstractmethod
def describe(self, question: Annotated[str, UserMessage],
picture: Annotated[str, UserMessage, ImageUrl]) -> str: # (1)
...
@UserMessage("Summarize the {{topic}} of this document")
@abstractmethod
def summarize(self, topic: Annotated[str, V("topic")],
document: Annotated[URI, UserMessage, PdfUrl]) -> str: # (2)
...package example.micronaut.aiservice.multimodal
import dev.langchain4j.service.SystemMessage
import dev.langchain4j.service.UserMessage
import dev.langchain4j.service.V
import io.micronaut.langchain4j.annotation.AiService
import io.micronaut.langchain4j.annotation.ImageUrl
import io.micronaut.langchain4j.annotation.PdfUrl
import java.net.URI
@AiService
interface PictureDescriber {
@SystemMessage("You describe pictures for visually impaired people")
fun describe(@UserMessage question: String,
@UserMessage @ImageUrl picture: String): String // (1)
@UserMessage("Summarize the {{topic}} of this document")
fun summarize(@V("topic") topic: String,
@UserMessage @PdfUrl document: URI): String // (2)
}package example.micronaut.aiservice.multimodal
import dev.langchain4j.service.SystemMessage
import dev.langchain4j.service.UserMessage
import dev.langchain4j.service.V
import io.micronaut.langchain4j.annotation.AiService
import io.micronaut.langchain4j.annotation.ImageUrl
import io.micronaut.langchain4j.annotation.PdfUrl
@AiService
interface PictureDescriber {
@SystemMessage("You describe pictures for visually impaired people")
String describe(@UserMessage String question,
@UserMessage @ImageUrl String picture) // (1)
@UserMessage("Summarize the {{topic}} of this document")
String summarize(@V("topic") String topic,
@UserMessage @PdfUrl URI document) // (2)
}| 1 | The picture at the URL is sent after the text of the user message. Like any content parameter of a LangChain4j AI service, the parameter is also annotated with @UserMessage. |
| 2 | The content parameter can be combined with a user message template. |
Parameters of the LangChain4j content types (ImageContent, PdfFileContent, AudioContent, VideoContent, or a List of Content) annotated with @UserMessage are also supported, for example to send an image read from a file.
A test can check the contents that the model receives:
package example.micronaut.aiservice.multimodal;
import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.data.message.ImageContent;
import dev.langchain4j.data.message.PdfFileContent;
import dev.langchain4j.data.message.TextContent;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.response.ChatResponse;
import io.micronaut.context.annotation.Primary;
import io.micronaut.context.annotation.Property;
import io.micronaut.context.annotation.Requires;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Singleton;
import org.junit.jupiter.api.Test;
import java.net.URI;
import java.util.stream.Collectors;
import static org.junit.jupiter.api.Assertions.assertEquals;
@Property(name = "spec.name", value = "PictureDescriberTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false)
class PictureDescriberTest {
@Test
void sendsThePicture(PictureDescriber describer) {
assertEquals("text:What is it?|image:https://example.com/cat.png",
describer.describe("What is it?", "https://example.com/cat.png"));
}
@Test
void sendsTheDocument(PictureDescriber describer) {
assertEquals("text:Summarize the risks of this document|pdf:https://example.com/report.pdf",
describer.summarize("risks", URI.create("https://example.com/report.pdf")));
}
/**
* Answers with the contents of the user message.
*/
@Singleton
@Primary
@Requires(property = "spec.name", value = "PictureDescriberTest")
static class EchoChatModel implements ChatModel {
@Override
public ChatResponse doChat(ChatRequest request) {
UserMessage message = (UserMessage) request.messages().getLast();
String described = message.contents().stream().map(content -> switch (content) {
case TextContent text -> "text:" + text.text();
case ImageContent image -> "image:" + image.image().url();
case PdfFileContent pdf -> "pdf:" + pdf.pdfFile().url();
default -> content.type().name();
}).collect(Collectors.joining("|"));
return ChatResponse.builder().aiMessage(AiMessage.from(described)).build();
}
}
}from typing import Annotated
from dev.langchain4j.data.message import AiMessage, ImageContent, PdfFileContent, TextContent
from dev.langchain4j.model.chat import ChatModel
from dev.langchain4j.model.chat.request import ChatRequest
from dev.langchain4j.model.chat.response import ChatResponse
from jakarta.inject import Inject, Singleton
from java.net import URI
from micronaut.context.annotation import Primary, Property, Requires
from micronaut.test.extensions.junit5.annotation import MicronautTest
from org.junit.jupiter.api import Test
from example.micronaut.aiservice.multimodal.PictureDescriber import PictureDescriber
@Property(name="spec.name", value="PictureDescriberTest")
@Property(name="langchain4j.ollama.enabled", value="false")
@MicronautTest(startApplication=False)
class PictureDescriberTest:
describer: Annotated[PictureDescriber, Inject]
@Test
def sends_the_picture(self):
assert self.describer.describe("What is it?", "https://example.com/cat.png") == "text:What is it?|image:https://example.com/cat.png"
@Test
def sends_the_document(self):
assert self.describer.summarize("risks", URI.create("https://example.com/report.pdf")) == "text:Summarize the risks of this document|pdf:https://example.com/report.pdf"
# Answers with the contents of the user message.
@Singleton
@Primary
@Requires(property="spec.name", value="PictureDescriberTest")
class EchoChatModel(ChatModel):
def doChat(self, request: ChatRequest) -> ChatResponse:
messages = request.messages()
contents = messages.get(messages.size() - 1).contents()
described = []
for content in contents:
if isinstance(content, TextContent):
described.append("text:" + content.text())
elif isinstance(content, ImageContent):
described.append("image:" + str(content.image().url()))
elif isinstance(content, PdfFileContent):
described.append("pdf:" + str(content.pdfFile().url()))
return ChatResponse.builder().aiMessage(AiMessage.from_("|".join(described))).build()package example.micronaut.aiservice.multimodal
import dev.langchain4j.data.message.AiMessage
import dev.langchain4j.data.message.ImageContent
import dev.langchain4j.data.message.PdfFileContent
import dev.langchain4j.data.message.TextContent
import dev.langchain4j.data.message.UserMessage
import dev.langchain4j.model.chat.ChatModel
import dev.langchain4j.model.chat.request.ChatRequest
import dev.langchain4j.model.chat.response.ChatResponse
import io.micronaut.context.annotation.Primary
import io.micronaut.context.annotation.Property
import io.micronaut.context.annotation.Requires
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import jakarta.inject.Singleton
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Test
import java.net.URI
@Property(name = "spec.name", value = "PictureDescriberTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false)
internal class PictureDescriberTest {
@Test
fun sendsThePicture(describer: PictureDescriber) {
assertEquals("text:What is it?|image:https://example.com/cat.png",
describer.describe("What is it?", "https://example.com/cat.png"))
}
@Test
fun sendsTheDocument(describer: PictureDescriber) {
assertEquals("text:Summarize the risks of this document|pdf:https://example.com/report.pdf",
describer.summarize("risks", URI.create("https://example.com/report.pdf")))
}
/**
* Answers with the contents of the user message.
*/
@Singleton
@Primary
@Requires(property = "spec.name", value = "PictureDescriberTest")
class EchoChatModel : ChatModel {
override fun doChat(request: ChatRequest): ChatResponse {
val message = request.messages().last() as UserMessage
val described = message.contents().joinToString("|") {
when (it) {
is TextContent -> "text:${it.text()}"
is ImageContent -> "image:${it.image().url()}"
is PdfFileContent -> "pdf:${it.pdfFile().url()}"
else -> it.type().name
}
}
return ChatResponse.builder().aiMessage(AiMessage.from(described)).build()
}
}
}package example.micronaut.aiservice.multimodal
import dev.langchain4j.data.message.AiMessage
import dev.langchain4j.data.message.Content
import dev.langchain4j.data.message.ImageContent
import dev.langchain4j.data.message.PdfFileContent
import dev.langchain4j.data.message.TextContent
import dev.langchain4j.data.message.UserMessage
import dev.langchain4j.model.chat.ChatModel
import dev.langchain4j.model.chat.request.ChatRequest
import dev.langchain4j.model.chat.response.ChatResponse
import io.micronaut.context.annotation.Primary
import io.micronaut.context.annotation.Property
import io.micronaut.context.annotation.Requires
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import jakarta.inject.Singleton
import org.junit.jupiter.api.Test
import static org.junit.jupiter.api.Assertions.assertEquals
@Property(name = "spec.name", value = "PictureDescriberTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false)
class PictureDescriberTest {
@Test
void sendsThePicture(PictureDescriber describer) {
assertEquals("text:What is it?|image:https://example.com/cat.png",
describer.describe("What is it?", "https://example.com/cat.png"))
}
@Test
void sendsTheDocument(PictureDescriber describer) {
assertEquals("text:Summarize the risks of this document|pdf:https://example.com/report.pdf",
describer.summarize("risks", URI.create("https://example.com/report.pdf")))
}
/**
* Answers with the contents of the user message.
*/
@Singleton
@Primary
@Requires(property = "spec.name", value = "PictureDescriberTest")
static class EchoChatModel implements ChatModel {
@Override
ChatResponse doChat(ChatRequest request) {
UserMessage message = (UserMessage) request.messages().last()
String described = message.contents().collect { Content content ->
if (content instanceof TextContent) {
return "text:" + content.text()
}
if (content instanceof ImageContent) {
return "image:" + content.image().url()
}
if (content instanceof PdfFileContent) {
return "pdf:" + content.pdfFile().url()
}
content.type().name()
}.join("|")
ChatResponse.builder().aiMessage(AiMessage.from(described)).build()
}
}
}Customizing AI Services
An AiServiceCustomizer bean customizes the LangChain4j AiServices builder of an AI service before the service is built, for any setting that the annotations and the configuration do not cover:
package example.micronaut.aiservice.customizer;
import dev.langchain4j.service.SystemMessage;
import io.micronaut.langchain4j.annotation.AiService;
@AiService
public interface Concierge {
@SystemMessage("You are the concierge of a hotel.")
String ask(String question);
}from abc import ABC, abstractmethod
from dev.langchain4j.service import SystemMessage
from micronaut.langchain4j.annotation import AiService
@AiService
class Concierge(ABC):
@SystemMessage("You are the concierge of a hotel.")
@abstractmethod
def ask(self, question: str) -> str:
...package example.micronaut.aiservice.customizer
import dev.langchain4j.service.SystemMessage
import io.micronaut.langchain4j.annotation.AiService
@AiService
interface Concierge {
@SystemMessage("You are the concierge of a hotel.")
fun ask(question: String): String
}package example.micronaut.aiservice.customizer
import dev.langchain4j.service.SystemMessage
import io.micronaut.langchain4j.annotation.AiService
@AiService
interface Concierge {
@SystemMessage("You are the concierge of a hotel.")
String ask(String question)
}package example.micronaut.aiservice.customizer;
import io.micronaut.langchain4j.aiservices.AiServiceCreationContext;
import io.micronaut.langchain4j.aiservices.AiServiceCustomizer;
import jakarta.inject.Singleton;
import java.util.function.UnaryOperator;
@Singleton
public class ConciergeCustomizer implements AiServiceCustomizer<Concierge> { // (1)
@Override
public void customize(AiServiceCreationContext<Concierge> creationContext) {
UnaryOperator<String> transformer = systemMessage -> systemMessage + " Keep your answers short.";
creationContext.aiServices().systemMessageTransformer(transformer); // (2)
}
}from java.util.function import UnaryOperator
from jakarta.inject import Singleton
from micronaut.langchain4j.aiservices import AiServiceCreationContext, AiServiceCustomizer
from .Concierge import Concierge
class ShortAnswers(UnaryOperator):
def apply(self, system_message: str) -> str:
return system_message + " Keep your answers short."
@Singleton
class ConciergeCustomizer(AiServiceCustomizer[Concierge]): # (1)
def customize(self, creation_context: AiServiceCreationContext[Concierge]) -> None:
creation_context.aiServices().systemMessageTransformer(ShortAnswers()) # (2)package example.micronaut.aiservice.customizer
import io.micronaut.langchain4j.aiservices.AiServiceCreationContext
import io.micronaut.langchain4j.aiservices.AiServiceCustomizer
import jakarta.inject.Singleton
import java.util.function.UnaryOperator
@Singleton
class ConciergeCustomizer : AiServiceCustomizer<Concierge> { // (1)
override fun customize(creationContext: AiServiceCreationContext<Concierge>) {
val transformer = UnaryOperator<String> { systemMessage -> "$systemMessage Keep your answers short." }
creationContext.aiServices().systemMessageTransformer(transformer) // (2)
}
}package example.micronaut.aiservice.customizer
import io.micronaut.langchain4j.aiservices.AiServiceCreationContext
import io.micronaut.langchain4j.aiservices.AiServiceCustomizer
import jakarta.inject.Singleton
import java.util.function.UnaryOperator
@Singleton
class ConciergeCustomizer implements AiServiceCustomizer<Concierge> { // (1)
@Override
void customize(AiServiceCreationContext<Concierge> creationContext) {
UnaryOperator<String> transformer = { String systemMessage -> systemMessage + " Keep your answers short." } as UnaryOperator<String>
creationContext.aiServices().systemMessageTransformer(transformer) // (2)
}
}| 1 | The customizer is a bean whose type argument is the AI service it customizes. A customizer qualified with the name of the service (@Named), or selected with the customizer member of @AiService, takes precedence. |
| 2 | The AiServices builder of the service: here, a transformer that appends an instruction to the system message. |
The builder of the model beans (for example OpenAiChatModel.OpenAiChatModelBuilder) can be customized the same way with a BeanCreatedEventListener.
Moderation
The methods of an AI service annotated with LangChain4j’s @Moderate check the user message with the ModerationModel bean, in parallel with the call to the chat model, and throw a ModerationException when it is flagged:
package example.micronaut.aiservice.moderation;
import dev.langchain4j.service.Moderate;
import io.micronaut.langchain4j.annotation.AiService;
@AiService
public interface ModeratedAssistant {
@Moderate // (1)
String chat(String userMessage);
}from abc import ABC, abstractmethod
from dev.langchain4j.service import Moderate
from micronaut.langchain4j.annotation import AiService
@AiService
class ModeratedAssistant(ABC):
@Moderate # (1)
@abstractmethod
def chat(self, user_message: str) -> str:
...package example.micronaut.aiservice.moderation
import dev.langchain4j.service.Moderate
import io.micronaut.langchain4j.annotation.AiService
@AiService
interface ModeratedAssistant {
@Moderate // (1)
fun chat(userMessage: String): String
}package example.micronaut.aiservice.moderation
import dev.langchain4j.service.Moderate
import io.micronaut.langchain4j.annotation.AiService
@AiService
interface ModeratedAssistant {
@Moderate // (1)
String chat(String userMessage)
}| 1 | The user message is moderated. |
The moderation model of the AI service is the ModerationModel bean qualified with the name of the service, otherwise the default one, for example the OpenAI moderation model:
Example Configuration
langchain4j.open-ai.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.moderation-model.model-name=omni-moderation-latestlangchain4j:
open-ai:
api-key: ${OPENAI_API_KEY}
moderation-model:
model-name: omni-moderation-latest[langchain4j.open-ai]
api-key = "${OPENAI_API_KEY}"
[langchain4j.open-ai.moderation-model]
model-name = "omni-moderation-latest"langchain4j {
openAi {
apiKey = "${OPENAI_API_KEY}"
moderationModel {
modelName = "omni-moderation-latest"
}
}
}{
langchain4j {
open-ai {
api-key = "${OPENAI_API_KEY}"
moderation-model {
model-name = "omni-moderation-latest"
}
}
}
}{
"langchain4j": {
"open-ai": {
"api-key": "${OPENAI_API_KEY}",
"moderation-model": {
"model-name": "omni-moderation-latest"
}
}
}
}5 Agentic Service
Agentic Service
Micronaut lets you declare LangChain4j Agentic services and have concrete agents generated at runtime.
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-agentic")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-agentic</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-agentic",
]Annotate an interface with AgenticService and declare methods using LangChain4j Agentic annotations.
In Python an agentic service is an abstract class (abc.ABC) whose methods are declared with … bodies; the decorators mirror the LangChain4j annotations.
|
LangChain4j builds the agents reflectively from the Java interface generated for the Python class and reads the @Agent, @UserMessage, @SequenceAgent, … annotations from it. @AgenticService and the LangChain4j annotations imply the @AllowsReflection hint, so the Python compiler copies them onto the generated interfaces, including those of sub-agents and workflows declared without @AgenticService: no @AllowsReflection or micronaut.introspection.allow-reflection option is needed. The hint for the LangChain4j annotations is added by micronaut-langchain4j-processor, which must be on the compile classpath.
|
The integration interprets your annotations and uses Micronaut DI to wire the underlying LangChain4j builders. It supports:
-
Typed agents built via
AgenticServices.agentBuilder(Class)with@Agentmethods. -
Declarative workflow agents via
AgenticServices.createAgenticSystem(…). -
Micronaut model, memory, RAG, tool, and lifecycle integration for each generated agent builder.
Quick start
package example.micronaut.agentic;
import dev.langchain4j.agentic.Agent;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
/**
* Minimal typed Agentic service used by the test-suite to validate Micronaut integration.
*/
@AgenticService
public interface GreeterAgent {
@UserMessage("Say hello to {{name}}")
@Agent(description = "Greets a person by name")
String greet(@V("name") String name);
}from abc import ABC, abstractmethod
from typing import Annotated
from dev.langchain4j.agentic import Agent
from dev.langchain4j.service import UserMessage, V
from micronaut.langchain4j.agentic.annotation import AgenticService
# Minimal typed Agentic service used by the test-suite to validate Micronaut integration.
@AgenticService
class GreeterAgent(ABC):
@UserMessage("Say hello to {{name}}")
@Agent(description="Greets a person by name")
@abstractmethod
def greet(self, name: Annotated[str, V("name")]) -> str:
...package example.micronaut.agentic
import dev.langchain4j.agentic.Agent
import dev.langchain4j.service.UserMessage
import dev.langchain4j.service.V
import io.micronaut.langchain4j.agentic.annotation.AgenticService
/**
* Minimal typed Agentic service used by the test-suite to validate Micronaut integration.
*/
@AgenticService
interface GreeterAgent {
@UserMessage("Say hello to {{name}}")
@Agent(description = "Greets a person by name")
fun greet(@V("name") name: String): String
}package example.micronaut.agentic
import dev.langchain4j.agentic.Agent
import dev.langchain4j.service.UserMessage
import dev.langchain4j.service.V
import io.micronaut.langchain4j.agentic.annotation.AgenticService
/**
* Minimal typed Agentic service used by the test-suite to validate Micronaut integration.
*/
@AgenticService
interface GreeterAgent {
@UserMessage("Say hello to {{name}}")
@Agent(description = "Greets a person by name")
String greet(@V("name") String name)
}Usage in tests
package example.micronaut.agentic;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import static org.junit.jupiter.api.Assertions.assertNotNull;
@MicronautTest(startApplication = false, environments = "agentic-test")
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class AgenticServiceTest {
@Test
void testAgenticGreeter(GreeterAgent agent) {
String result = agent.greet("John");
assertNotNull(result);
}
}from typing import Annotated
from jakarta.inject import Inject
from micronaut.test.extensions.junit5.annotation import MicronautTest
from org.junit.jupiter.api import Test
from example.micronaut.agentic.GreeterAgent import GreeterAgent
@MicronautTest(startApplication=False, environments=["agentic-test"])
class AgenticServiceTest:
agent: Annotated[GreeterAgent, Inject]
@Test
def test_agentic_greeter(self):
result = self.agent.greet("John")
assert result is not Nonepackage example.micronaut.agentic
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import org.junit.jupiter.api.Assertions.assertNotNull
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
@MicronautTest(startApplication = false, environments = ["agentic-test"])
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
internal class AgenticServiceTest {
@Test
fun testAgenticGreeter(agent: GreeterAgent) {
val result = agent.greet("John")
assertNotNull(result)
}
}package example.micronaut.agentic
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import static org.junit.jupiter.api.Assertions.assertNotNull
@MicronautTest(startApplication = false, environments = "agentic-test")
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class AgenticServiceTest {
@Test
void testAgenticGreeter(GreeterAgent agent) {
String result = agent.greet("John")
assertNotNull(result)
}
}Declarative workflows
You can build workflows using LangChain4j’s declarative annotations. The integration routes construction through Micronaut DI so that supported workflow builders are Micronaut-managed beans (allowing listeners and customization).
Supported patterns:
The Micronaut-managed workflow builder lifecycle applies to the workflow types listed above. Supervisor and planner agents (SupervisorAgent, PlannerAgent) are supported too: see Supervisor agents.
Example: an evening planner (Sequence)
package example.micronaut.agentic;
import dev.langchain4j.agentic.declarative.SequenceAgent;
import dev.langchain4j.service.V;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
/**
* Declarative sequence workflow inspired by the "EveningPlannerAgent" example.
* This agent coordinates sub-agents and produces a final "plan" output.
*/
@AgenticService(outputKey = "plan")
public interface EveningPlannerAgent {
// Declarative sequence workflow definition (no method body needed)
@SequenceAgent(
subAgents = {
TravelRecommenderAgent.class,
RecipeAdvisorAgent.class,
PlanSynthesizerAgent.class
},
outputKey = "plan",
name = "planEvening"
)
String plan(@V("topic") String topic);
}from abc import ABC, abstractmethod
from typing import Annotated
from dev.langchain4j.agentic.declarative import SequenceAgent
from dev.langchain4j.service import V
from micronaut.langchain4j.agentic.annotation import AgenticService
from .PlanSynthesizerAgent import PlanSynthesizerAgent
from .RecipeAdvisorAgent import RecipeAdvisorAgent
from .TravelRecommenderAgent import TravelRecommenderAgent
# Declarative sequence workflow inspired by the "EveningPlannerAgent" example.
# This agent coordinates sub-agents and produces a final "plan" output.
@AgenticService(outputKey="plan")
class EveningPlannerAgent(ABC):
# Declarative sequence workflow definition (no method body needed)
@SequenceAgent(
subAgents=[
TravelRecommenderAgent,
RecipeAdvisorAgent,
PlanSynthesizerAgent
],
outputKey="plan",
name="planEvening"
)
@abstractmethod
def plan(self, topic: Annotated[str, V("topic")]) -> str:
...package example.micronaut.agentic
import dev.langchain4j.agentic.declarative.SequenceAgent
import dev.langchain4j.service.V
import io.micronaut.langchain4j.agentic.annotation.AgenticService
/**
* Declarative sequence workflow inspired by the "EveningPlannerAgent" example.
* This agent coordinates sub-agents and produces a final "plan" output.
*/
@AgenticService(outputKey = "plan")
interface EveningPlannerAgent {
// Declarative sequence workflow definition (no method body needed)
@SequenceAgent(
subAgents = [
TravelRecommenderAgent::class,
RecipeAdvisorAgent::class,
PlanSynthesizerAgent::class
],
outputKey = "plan",
name = "planEvening"
)
fun plan(@V("topic") topic: String): String
}package example.micronaut.agentic
import dev.langchain4j.agentic.declarative.SequenceAgent
import dev.langchain4j.service.V
import io.micronaut.langchain4j.agentic.annotation.AgenticService
/**
* Declarative sequence workflow inspired by the "EveningPlannerAgent" example.
* This agent coordinates sub-agents and produces a final "plan" output.
*/
@AgenticService(outputKey = "plan")
interface EveningPlannerAgent {
// Declarative sequence workflow definition (no method body needed)
@SequenceAgent(
subAgents = [
TravelRecommenderAgent,
RecipeAdvisorAgent,
PlanSynthesizerAgent
],
outputKey = "plan",
name = "planEvening"
)
String plan(@V("topic") String topic)
}Parallel workflow example
package example.micronaut.agentic;
import dev.langchain4j.agentic.Agent;
import dev.langchain4j.agentic.declarative.Output;
import dev.langchain4j.agentic.declarative.ParallelAgent;
import dev.langchain4j.agentic.declarative.ParallelExecutor;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import java.util.concurrent.Executor;
import java.util.concurrent.ForkJoinPool;
import static org.junit.jupiter.api.Assertions.assertFalse;
/**
* Validates declarative ParallelAgent workflow wiring through @AgenticService.
* Ensures Micronaut DI correctly builds the agentic system and executes the parallel plan.
*/
@MicronautTest(startApplication = false, environments = "agentic-test")
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class ParallelPlanningAgentTest {
@Test
void testDeclarativeParallel(EveningPlanner agent) {
String plan = agent.plan("jazz", "romantic");
System.out.println("parallel plan = " + plan);
assertFalse(plan.isEmpty());
}
@AgenticService
public interface MusicPlanner {
@UserMessage("""
Choose a band which plays music in the {{style}} style.
Answer with the name of the band only: no details, no explanation.
""")
@Agent(outputKey = "band")
String suggestBand(@V("style") String style);
}
@AgenticService
public interface DinnerPlanner {
@UserMessage("""
Choose a menu for dinner for the following mood: {{mood}}
Answer with the menu only: no details, no explanations.
""")
@Agent(outputKey = "menu")
String suggestMenu(@V("mood") String mood);
}
/**
* Demonstrates declarative ParallelAgent orchestration using Micronaut DI.
*/
@AgenticService(outputKey = "plan")
public interface EveningPlanner {
@ParallelAgent(
subAgents = {
MusicPlanner.class,
DinnerPlanner.class
},
outputKey = "plan",
name = "planEvening"
)
String plan(@V("style") String style, @V("mood") String mood);
// Use a shared executor for parallel execution
@ParallelExecutor
static Executor executor() {
return ForkJoinPool.commonPool();
}
// Aggregate the parallel outputs into a single "plan" string
@Output
static String aggregate(@V("band") String band, @V("menu") String menu) {
String c = band == null ? "" : band.trim();
String r = menu == null ? "" : menu.trim();
if (c.isEmpty() && r.isEmpty()) {
return "";
}
if (c.isEmpty()) {
return "Play: " + r;
}
if (r.isEmpty()) {
return "Menu: " + c;
}
return "Play: " + c + " | Menu: " + r;
}
}
}from abc import ABC, abstractmethod
from typing import Annotated
from dev.langchain4j.agentic import Agent
from dev.langchain4j.agentic.declarative import Output, ParallelAgent, ParallelExecutor
from dev.langchain4j.service import UserMessage, V
from jakarta.inject import Inject
from java.util.concurrent import Executor, ForkJoinPool
from micronaut.langchain4j.agentic.annotation import AgenticService
from micronaut.test.extensions.junit5.annotation import MicronautTest
from org.junit.jupiter.api import Test
@AgenticService
class MusicPlanner(ABC):
@UserMessage("""
Choose a band which plays music in the {{style}} style.
Answer with the name of the band only: no details, no explanation.
""")
@Agent(outputKey="band")
@abstractmethod
def suggest_band(self, style: Annotated[str, V("style")]) -> str:
...
@AgenticService
class DinnerPlanner(ABC):
@UserMessage("""
Choose a menu for dinner for the following mood: {{mood}}
Answer with the menu only: no details, no explanations.
""")
@Agent(outputKey="menu")
@abstractmethod
def suggest_menu(self, mood: Annotated[str, V("mood")]) -> str:
...
# Demonstrates declarative ParallelAgent orchestration using Micronaut DI.
@AgenticService(outputKey="plan")
class EveningPlanner(ABC):
@ParallelAgent(
subAgents=[
MusicPlanner,
DinnerPlanner
],
outputKey="plan",
name="planEvening"
)
@abstractmethod
def plan(self, style: Annotated[str, V("style")], mood: Annotated[str, V("mood")]) -> str:
...
# Use a shared executor for parallel execution
@ParallelExecutor
@staticmethod
def executor() -> Executor:
return ForkJoinPool.commonPool()
# Aggregate the parallel outputs into a single "plan" string
@Output
@staticmethod
def aggregate(band: Annotated[str, V("band")], menu: Annotated[str, V("menu")]) -> str:
c = "" if band is None else band.strip()
r = "" if menu is None else menu.strip()
if c == "" and r == "":
return ""
if c == "":
return f"Play: {r}"
if r == "":
return f"Menu: {c}"
return f"Play: {c} | Menu: {r}"
# Validates declarative ParallelAgent workflow wiring through @AgenticService.
# Ensures Micronaut DI correctly builds the agentic system and executes the parallel plan.
@MicronautTest(startApplication=False, environments=["agentic-test"])
class ParallelPlanningAgentTest:
agent: Annotated[EveningPlanner, Inject]
@Test
def test_declarative_parallel(self):
plan = self.agent.plan("jazz", "romantic")
print(f"parallel plan = {plan}")
assert plan != ""package example.micronaut.agentic
import dev.langchain4j.agentic.Agent
import dev.langchain4j.agentic.declarative.Output
import dev.langchain4j.agentic.declarative.ParallelAgent
import dev.langchain4j.agentic.declarative.ParallelExecutor
import dev.langchain4j.service.UserMessage
import dev.langchain4j.service.V
import io.micronaut.langchain4j.agentic.annotation.AgenticService
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import org.junit.jupiter.api.Assertions.assertFalse
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import java.util.concurrent.Executor
import java.util.concurrent.ForkJoinPool
/**
* Validates declarative ParallelAgent workflow wiring through @AgenticService.
* Ensures Micronaut DI correctly builds the agentic system and executes the parallel plan.
*/
@MicronautTest(startApplication = false, environments = ["agentic-test"])
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
internal class ParallelPlanningAgentTest {
@Test
fun testDeclarativeParallel(agent: EveningPlanner) {
val plan = agent.plan("jazz", "romantic")
println("parallel plan = $plan")
assertFalse(plan.isEmpty())
}
@AgenticService
interface MusicPlanner {
@UserMessage("""
Choose a band which plays music in the {{style}} style.
Answer with the name of the band only: no details, no explanation.
""")
@Agent(outputKey = "band")
fun suggestBand(@V("style") style: String): String
}
@AgenticService
interface DinnerPlanner {
@UserMessage("""
Choose a menu for dinner for the following mood: {{mood}}
Answer with the menu only: no details, no explanations.
""")
@Agent(outputKey = "menu")
fun suggestMenu(@V("mood") mood: String): String
}
/**
* Demonstrates declarative ParallelAgent orchestration using Micronaut DI.
*/
@AgenticService(outputKey = "plan")
interface EveningPlanner {
@ParallelAgent(
subAgents = [
MusicPlanner::class,
DinnerPlanner::class
],
outputKey = "plan",
name = "planEvening"
)
fun plan(@V("style") style: String, @V("mood") mood: String): String
companion object {
// Use a shared executor for parallel execution
@JvmStatic
@ParallelExecutor
fun executor(): Executor = ForkJoinPool.commonPool()
// Aggregate the parallel outputs into a single "plan" string
@JvmStatic
@Output
fun aggregate(@V("band") band: String?, @V("menu") menu: String?): String {
val c = band?.trim() ?: ""
val r = menu?.trim() ?: ""
if (c.isEmpty() && r.isEmpty()) {
return ""
}
if (c.isEmpty()) {
return "Play: $r"
}
if (r.isEmpty()) {
return "Menu: $c"
}
return "Play: $c | Menu: $r"
}
}
}
}package example.micronaut.agentic
import dev.langchain4j.agentic.Agent
import dev.langchain4j.agentic.declarative.Output
import dev.langchain4j.agentic.declarative.ParallelAgent
import dev.langchain4j.agentic.declarative.ParallelExecutor
import dev.langchain4j.service.UserMessage
import dev.langchain4j.service.V
import io.micronaut.langchain4j.agentic.annotation.AgenticService
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import java.util.concurrent.Executor
import java.util.concurrent.ForkJoinPool
import static org.junit.jupiter.api.Assertions.assertFalse
/**
* Validates declarative ParallelAgent workflow wiring through @AgenticService.
* Ensures Micronaut DI correctly builds the agentic system and executes the parallel plan.
*/
@MicronautTest(startApplication = false, environments = "agentic-test")
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class ParallelPlanningAgentTest {
@Test
void testDeclarativeParallel(EveningPlanner agent) {
String plan = agent.plan("jazz", "romantic")
println("parallel plan = " + plan)
assertFalse(plan.isEmpty())
}
@AgenticService
static interface MusicPlanner {
@UserMessage("""
Choose a band which plays music in the {{style}} style.
Answer with the name of the band only: no details, no explanation.
""")
@Agent(outputKey = "band")
String suggestBand(@V("style") String style)
}
@AgenticService
static interface DinnerPlanner {
@UserMessage("""
Choose a menu for dinner for the following mood: {{mood}}
Answer with the menu only: no details, no explanations.
""")
@Agent(outputKey = "menu")
String suggestMenu(@V("mood") String mood)
}
/**
* Demonstrates declarative ParallelAgent orchestration using Micronaut DI.
*/
@AgenticService(outputKey = "plan")
static interface EveningPlanner {
@ParallelAgent(
subAgents = [
MusicPlanner,
DinnerPlanner
],
outputKey = "plan",
name = "planEvening"
)
String plan(@V("style") String style, @V("mood") String mood)
// Use a shared executor for parallel execution
@ParallelExecutor
static Executor executor() {
ForkJoinPool.commonPool()
}
// Aggregate the parallel outputs into a single "plan" string
@Output
static String aggregate(@V("band") String band, @V("menu") String menu) {
String c = band == null ? "" : band.trim()
String r = menu == null ? "" : menu.trim()
if (c.isEmpty() && r.isEmpty()) {
return ""
}
if (c.isEmpty()) {
return "Play: " + r
}
if (r.isEmpty()) {
return "Menu: " + c
}
return "Play: " + c + " | Menu: " + r
}
}
}Loop example with customization via a listener
package example.micronaut.agentic;
import dev.langchain4j.agentic.Agent;
import dev.langchain4j.agentic.declarative.LoopAgent;
import dev.langchain4j.agentic.workflow.LoopAgentService;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import io.micronaut.context.annotation.Requires;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import io.micronaut.core.annotation.NonNull;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
import jakarta.inject.Singleton;
/**
* Demonstrates declarative LoopAgent orchestration using Micronaut DI.
*/
@AgenticService(outputKey = "translation")
public interface LoopingPlannerAgent {
@LoopAgent(
subAgents = {
TranslatorAgent.class
},
outputKey = "text",
maxIterations = 3
)
String translatesInLoop(@V("text") String text);
interface TranslatorAgent {
@UserMessage("""
You are a translator.
If the text is in English, translate to French.
If the text is in French, translate to German.
If the text is in German, translate to Spanish.
Translate this: "{{text}}". Answer with the translation only, no explanations, no details.
""")
@Agent(outputKey = "text")
String translate(@V("text") String text);
}
@Singleton
@Requires(property = "spec.name", value = "LoopingPlannerAgentTest")
class LoopBuilderListener implements BeanCreatedEventListener<LoopAgentService<?>> {
@Override
public LoopAgentService<?> onCreated(@NonNull BeanCreatedEvent<LoopAgentService<?>> event) {
LoopAgentService<?> builder = event.getBean();
builder.exitCondition((scope, idx) -> idx == 3);
return builder;
}
}
}from abc import ABC, abstractmethod
from typing import Annotated
from dev.langchain4j.agentic import Agent
from dev.langchain4j.agentic.declarative import LoopAgent
from dev.langchain4j.agentic.workflow import LoopAgentService
from dev.langchain4j.service import UserMessage, V
from jakarta.inject import Singleton
from micronaut.context.annotation import Requires
from micronaut.context.event import BeanCreatedEvent, BeanCreatedEventListener
from micronaut.langchain4j.agentic.annotation import AgenticService
class TranslatorAgent(ABC):
@UserMessage("""
You are a translator.
If the text is in English, translate to French.
If the text is in French, translate to German.
If the text is in German, translate to Spanish.
Translate this: "{{text}}". Answer with the translation only, no explanations, no details.
""")
@Agent(outputKey="text")
@abstractmethod
def translate(self, text: Annotated[str, V("text")]) -> str:
...
# Demonstrates declarative LoopAgent orchestration using Micronaut DI.
@AgenticService(outputKey="translation")
class LoopingPlannerAgent(ABC):
@LoopAgent(
subAgents=[
TranslatorAgent
],
outputKey="text",
maxIterations=3
)
@abstractmethod
def translates_in_loop(self, text: Annotated[str, V("text")]) -> str:
...
@Singleton
@Requires(property="spec.name", value="LoopingPlannerAgentTest")
class LoopBuilderListener(BeanCreatedEventListener[LoopAgentService]):
def onCreated(self, event: BeanCreatedEvent[LoopAgentService]) -> LoopAgentService:
builder = event.getBean()
builder.exitCondition(lambda scope, idx: idx == 3)
return builderpackage example.micronaut.agentic
import dev.langchain4j.agentic.Agent
import dev.langchain4j.agentic.declarative.LoopAgent
import dev.langchain4j.agentic.workflow.LoopAgentService
import dev.langchain4j.service.UserMessage
import dev.langchain4j.service.V
import io.micronaut.context.annotation.Requires
import io.micronaut.context.event.BeanCreatedEvent
import io.micronaut.context.event.BeanCreatedEventListener
import io.micronaut.langchain4j.agentic.annotation.AgenticService
import jakarta.inject.Singleton
/**
* Demonstrates declarative LoopAgent orchestration using Micronaut DI.
*/
@AgenticService(outputKey = "translation")
interface LoopingPlannerAgent {
@LoopAgent(
subAgents = [
TranslatorAgent::class
],
outputKey = "text",
maxIterations = 3
)
fun translatesInLoop(@V("text") text: String): String
interface TranslatorAgent {
@UserMessage("""
You are a translator.
If the text is in English, translate to French.
If the text is in French, translate to German.
If the text is in German, translate to Spanish.
Translate this: "{{text}}". Answer with the translation only, no explanations, no details.
""")
@Agent(outputKey = "text")
fun translate(@V("text") text: String): String
}
@Singleton
@Requires(property = "spec.name", value = "LoopingPlannerAgentTest")
class LoopBuilderListener : BeanCreatedEventListener<LoopAgentService<*>> {
override fun onCreated(event: BeanCreatedEvent<LoopAgentService<*>>): LoopAgentService<*> {
val builder = event.bean
builder.exitCondition { _, idx -> idx == 3 }
return builder
}
}
}package example.micronaut.agentic
import dev.langchain4j.agentic.Agent
import dev.langchain4j.agentic.declarative.LoopAgent
import dev.langchain4j.agentic.workflow.LoopAgentService
import dev.langchain4j.service.UserMessage
import dev.langchain4j.service.V
import io.micronaut.context.annotation.Requires
import io.micronaut.context.event.BeanCreatedEvent
import io.micronaut.context.event.BeanCreatedEventListener
import io.micronaut.langchain4j.agentic.annotation.AgenticService
import jakarta.inject.Singleton
/**
* Demonstrates declarative LoopAgent orchestration using Micronaut DI.
*/
@AgenticService(outputKey = "translation")
interface LoopingPlannerAgent {
@LoopAgent(
subAgents = [
TranslatorAgent
],
outputKey = "text",
maxIterations = 3
)
String translatesInLoop(@V("text") String text)
interface TranslatorAgent {
@UserMessage("""
You are a translator.
If the text is in English, translate to French.
If the text is in French, translate to German.
If the text is in German, translate to Spanish.
Translate this: "{{text}}". Answer with the translation only, no explanations, no details.
""")
@Agent(outputKey = "text")
String translate(@V("text") String text)
}
@Singleton
@Requires(property = "spec.name", value = "LoopingPlannerAgentTest")
class LoopBuilderListener implements BeanCreatedEventListener<LoopAgentService<?>> {
@Override
LoopAgentService<?> onCreated(BeanCreatedEvent<LoopAgentService<?>> event) {
LoopAgentService<?> builder = event.bean
builder.exitCondition { scope, idx -> idx == 3 }
builder
}
}
}Configuration
You can select which ChatModel bean to use per agent via configuration. The agent id is derived from the interface name by:
-
stripping a trailing "Agent" suffix
-
converting UpperCamel to lower-kebab (e.g. CreativeWriterAgent → creative-writer)
langchain4j.agentic.agents.<agent-id>.chat-modelExamples
langchain4j.agentic.agents.greeter.chat-model=friendly-chat-model
langchain4j.agentic.agents.creative-writer.chat-model=creative-chat-modellangchain4j:
agentic:
agents:
greeter:
chat-model: friendly-chat-model
creative-writer:
chat-model: creative-chat-model[langchain4j.agentic.agents.greeter]
chat-model = "friendly-chat-model"
[langchain4j.agentic.agents.creative-writer]
chat-model = "creative-chat-model"langchain4j {
agentic {
agents {
greeter {
chatModel = "friendly-chat-model"
}
creativeWriter {
chatModel = "creative-chat-model"
}
}
}
}{
langchain4j {
agentic {
agents {
greeter {
chat-model = "friendly-chat-model"
}
creative-writer {
chat-model = "creative-chat-model"
}
}
}
}
}{
"langchain4j": {
"agentic": {
"agents": {
"greeter": {
"chat-model": "friendly-chat-model"
},
"creative-writer": {
"chat-model": "creative-chat-model"
}
}
}
}
}If the property is not set and there is exactly one ChatModel bean in the context, it will be used automatically.
Memory
By default, agentic services reuse the core MessageWindowChatMemory built from the configured ChatMemoryStore. The core module publishes a MessageWindowChatMemory.Builder per available ChatMemoryStore (via @EachBean(ChatMemoryStore)), so agents automatically use the default builder when a single memory store is configured.
You can optionally override memory per agent:
langchain4j.agentic.agents.<agent-id>.memory.store
langchain4j.agentic.agents.<agent-id>.memory.max-messagesNotes:
- If memory.store is not set, the default MessageWindowChatMemory.Builder is used (as resolved by Micronaut DI).
- If multiple memory stores are available, set memory.store to select the store for an agent.
- If memory.max-messages is not set, the global core setting langchain4j.chat-memory-store.message-window.max-messages applies.
Examples
langchain4j.agentic.agents.greeter.memory.store=redis
langchain4j.agentic.agents.creative-writer.memory.max-messages=50langchain4j:
agentic:
agents:
greeter:
memory:
store: redis
creative-writer:
memory:
max-messages: 50[langchain4j.agentic.agents.greeter.memory]
store = "redis"
[langchain4j.agentic.agents.creative-writer.memory]
max-messages = 50langchain4j {
agentic {
agents {
greeter {
memory {
store = "redis"
}
}
creativeWriter {
memory {
maxMessages = 50
}
}
}
}
}{
langchain4j {
agentic {
agents {
greeter {
memory {
store = "redis"
}
}
creative-writer {
memory {
max-messages = 50
}
}
}
}
}
}{
"langchain4j": {
"agentic": {
"agents": {
"greeter": {
"memory": {
"store": "redis"
}
},
"creative-writer": {
"memory": {
"max-messages": 50
}
}
}
}
}
}Retrieval
Retrieval is opt-in. An agent performs retrieval augmented generation only when the application declares a dev.langchain4j.rag.RetrievalAugmentor bean or a dev.langchain4j.rag.content.retriever.ContentRetriever bean. A RetrievalAugmentor takes precedence over a ContentRetriever. For either type, a bean @Named after the agent id (for example @Named("greeter") for GreeterAgent) is preferred over the default bean. Embedding model and embedding store beans are never paired automatically.
Tools
You can request specific tool beans to be registered with the agent builder using the tools attribute.
Tool classes should be Micronaut beans containing methods annotated with dev.langchain4j.agent.tool.Tool.
Customization
You can customize builders using Micronaut lifecycle listeners. Because workflow builders are created as Micronaut beans during declarative system construction, your listeners can adjust names, exit conditions, parallelism, etc., before the system executes.
Typed agents can also be customized via BeanCreatedEventListener<AgentBuilder<?, ?>>; the LoopingPlannerAgent snippet above shows the same Micronaut lifecycle technique applied to LoopAgentService.
For example, this listener customizes every generated declarative agent builder before LangChain4j builds the agentic system:
package example.micronaut.agentic;
import dev.langchain4j.agentic.agent.AgentBuilder;
import io.micronaut.context.annotation.Requires;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import io.micronaut.core.annotation.NonNull;
import jakarta.inject.Singleton;
@Singleton
@Requires(env = "agentic-docs")
final class SupportAgentBuilderListener implements BeanCreatedEventListener<AgentBuilder<?, ?>> {
@Override
public AgentBuilder<?, ?> onCreated(@NonNull BeanCreatedEvent<AgentBuilder<?, ?>> event) {
AgentBuilder<?, ?> builder = event.getBean();
builder.name("customer-support-agent");
builder.outputKey("supportResponse");
return builder;
}
}from dev.langchain4j.agentic.agent import AgentBuilder
from jakarta.inject import Singleton
from micronaut.context.annotation import Requires
from micronaut.context.event import BeanCreatedEvent, BeanCreatedEventListener
@Singleton
@Requires(env="agentic-docs")
class SupportAgentBuilderListener(BeanCreatedEventListener[AgentBuilder]):
def onCreated(self, event: BeanCreatedEvent[AgentBuilder]) -> AgentBuilder:
builder = event.getBean()
builder.name("customer-support-agent")
builder.outputKey("supportResponse")
return builderpackage example.micronaut.agentic
import dev.langchain4j.agentic.agent.AgentBuilder
import io.micronaut.context.annotation.Requires
import io.micronaut.context.event.BeanCreatedEvent
import io.micronaut.context.event.BeanCreatedEventListener
import jakarta.inject.Singleton
@Singleton
@Requires(env = ["agentic-docs"])
internal class SupportAgentBuilderListener : BeanCreatedEventListener<AgentBuilder<*, *>> {
override fun onCreated(event: BeanCreatedEvent<AgentBuilder<*, *>>): AgentBuilder<*, *> {
val builder = event.bean
builder.name("customer-support-agent")
builder.outputKey("supportResponse")
return builder
}
}package example.micronaut.agentic
import dev.langchain4j.agentic.agent.AgentBuilder
import io.micronaut.context.annotation.Requires
import io.micronaut.context.event.BeanCreatedEvent
import io.micronaut.context.event.BeanCreatedEventListener
import jakarta.inject.Singleton
@Singleton
@Requires(env = "agentic-docs")
class SupportAgentBuilderListener implements BeanCreatedEventListener<AgentBuilder<?, ?>> {
@Override
AgentBuilder<?, ?> onCreated(BeanCreatedEvent<AgentBuilder<?, ?>> event) {
AgentBuilder<?, ?> builder = event.bean
builder.name("customer-support-agent")
builder.outputKey("supportResponse")
builder
}
}Workflow builders can be customized the same way. This example changes the loop exit condition for all LoopAgentService builders created by the agentic integration:
package example.micronaut.agentic;
import dev.langchain4j.agentic.workflow.LoopAgentService;
import io.micronaut.context.annotation.Requires;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import io.micronaut.core.annotation.NonNull;
import jakarta.inject.Singleton;
@Singleton
@Requires(env = "agentic-docs")
final class LoopAgentServiceListener implements BeanCreatedEventListener<LoopAgentService<?>> {
@Override
public LoopAgentService<?> onCreated(@NonNull BeanCreatedEvent<LoopAgentService<?>> event) {
LoopAgentService<?> builder = event.getBean();
builder.exitCondition((scope, iteration) -> iteration >= 3);
return builder;
}
}from dev.langchain4j.agentic.workflow import LoopAgentService
from jakarta.inject import Singleton
from micronaut.context.annotation import Requires
from micronaut.context.event import BeanCreatedEvent, BeanCreatedEventListener
@Singleton
@Requires(env="agentic-docs")
class LoopAgentServiceListener(BeanCreatedEventListener[LoopAgentService]):
def onCreated(self, event: BeanCreatedEvent[LoopAgentService]) -> LoopAgentService:
builder = event.getBean()
builder.exitCondition(lambda scope, iteration: iteration >= 3)
return builderpackage example.micronaut.agentic
import dev.langchain4j.agentic.workflow.LoopAgentService
import io.micronaut.context.annotation.Requires
import io.micronaut.context.event.BeanCreatedEvent
import io.micronaut.context.event.BeanCreatedEventListener
import jakarta.inject.Singleton
@Singleton
@Requires(env = ["agentic-docs"])
internal class LoopAgentServiceListener : BeanCreatedEventListener<LoopAgentService<*>> {
override fun onCreated(event: BeanCreatedEvent<LoopAgentService<*>>): LoopAgentService<*> {
val builder = event.bean
builder.exitCondition { _, iteration -> iteration >= 3 }
return builder
}
}package example.micronaut.agentic
import dev.langchain4j.agentic.workflow.LoopAgentService
import io.micronaut.context.annotation.Requires
import io.micronaut.context.event.BeanCreatedEvent
import io.micronaut.context.event.BeanCreatedEventListener
import jakarta.inject.Singleton
@Singleton
@Requires(env = "agentic-docs")
class LoopAgentServiceListener implements BeanCreatedEventListener<LoopAgentService<?>> {
@Override
LoopAgentService<?> onCreated(BeanCreatedEvent<LoopAgentService<?>> event) {
LoopAgentService<?> builder = event.bean
builder.exitCondition { scope, iteration -> iteration >= 3 }
builder
}
}The same approach applies to Micronaut-managed workflow services: SequentialAgentService, ParallelAgentService, ParallelMapperService, ConditionalAgentService, and LoopAgentService.
Supervisor agents
A supervisor agent lets the chat model decide which of its sub-agents to invoke, and with which arguments, until the request is addressed. Declare it with LangChain4j’s @SupervisorAgent on a method of an @AgenticService:
package example.micronaut.agentic;
import dev.langchain4j.agentic.declarative.SupervisorAgent;
import dev.langchain4j.agentic.supervisor.SupervisorResponseStrategy;
import dev.langchain4j.service.V;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
/**
* A supervisor agent: the chat model plans which sub-agent to invoke next until the request is addressed.
*/
@AgenticService
public interface TravelSupervisorAgent {
@SupervisorAgent(
subAgents = {
TravelRecommenderAgent.class,
RecipeAdvisorAgent.class
},
responseStrategy = SupervisorResponseStrategy.SUMMARY,
maxAgentsInvocations = 5
)
String organize(@V("request") String request);
}from abc import ABC, abstractmethod
from typing import Annotated
from dev.langchain4j.agentic.declarative import SupervisorAgent
from dev.langchain4j.agentic.supervisor import SupervisorResponseStrategy
from dev.langchain4j.service import V
from micronaut.langchain4j.agentic.annotation import AgenticService
from .RecipeAdvisorAgent import RecipeAdvisorAgent
from .TravelRecommenderAgent import TravelRecommenderAgent
# A supervisor agent: the chat model plans which sub-agent to invoke next until the request is addressed.
@AgenticService
class TravelSupervisorAgent(ABC):
@SupervisorAgent(
subAgents=[
TravelRecommenderAgent,
RecipeAdvisorAgent
],
responseStrategy=SupervisorResponseStrategy.SUMMARY,
maxAgentsInvocations=5
)
@abstractmethod
def organize(self, request: Annotated[str, V("request")]) -> str:
...package example.micronaut.agentic
import dev.langchain4j.agentic.declarative.SupervisorAgent
import dev.langchain4j.agentic.supervisor.SupervisorResponseStrategy
import dev.langchain4j.service.V
import io.micronaut.langchain4j.agentic.annotation.AgenticService
/**
* A supervisor agent: the chat model plans which sub-agent to invoke next until the request is addressed.
*/
@AgenticService
interface TravelSupervisorAgent {
@SupervisorAgent(
subAgents = [
TravelRecommenderAgent::class,
RecipeAdvisorAgent::class
],
responseStrategy = SupervisorResponseStrategy.SUMMARY,
maxAgentsInvocations = 5
)
fun organize(@V("request") request: String): String
}package example.micronaut.agentic
import dev.langchain4j.agentic.declarative.SupervisorAgent
import dev.langchain4j.agentic.supervisor.SupervisorResponseStrategy
import dev.langchain4j.service.V
import io.micronaut.langchain4j.agentic.annotation.AgenticService
/**
* A supervisor agent: the chat model plans which sub-agent to invoke next until the request is addressed.
*/
@AgenticService
interface TravelSupervisorAgent {
@SupervisorAgent(
subAgents = [
TravelRecommenderAgent,
RecipeAdvisorAgent
],
responseStrategy = SupervisorResponseStrategy.SUMMARY,
maxAgentsInvocations = 5
)
String organize(@V("request") String request)
}The supervisor plans with the chat model of the agentic service (see Configuration), and its sub-agents receive the Micronaut model, memory, retrieval and tool configuration of their agent id like any other agent. responseStrategy selects whether the supervisor answers with the last response of a sub-agent (LAST, the default), with the summary written by the planner (SUMMARY) or with the better of both (SCORED).
Tests can script the planner with a chat model bean that answers with the agent invocations:
package example.micronaut.agentic;
import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.data.message.SystemMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.response.ChatResponse;
import io.micronaut.context.annotation.Property;
import io.micronaut.context.annotation.Primary;
import io.micronaut.context.annotation.Requires;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Singleton;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false, environments = "supervisor-test")
class TravelSupervisorAgentTest {
@Test
void supervisorInvokesTheSubAgents(TravelSupervisorAgent supervisor) {
assertEquals("Visit Italy and cook a pizza", supervisor.organize("I love jazz, plan my holidays"));
}
/**
* Plays the planner of the supervisor: recommends a country, then a recipe, then answers.
*/
@Singleton
@Primary
@Requires(env = "supervisor-test")
static class ScriptedPlannerChatModel implements ChatModel {
@Override
public ChatResponse doChat(ChatRequest request) {
String user = ((UserMessage) request.messages().getLast()).singleText();
boolean planner = request.messages().stream()
.anyMatch(message -> message instanceof SystemMessage system && system.text().contains("planner"));
if (!planner) {
return answer(user.contains("travel expert") ? "Italy" : "Italy - Pizza");
}
if (user.contains("'Italy - Pizza'")) {
return answer("{\"agentName\": \"done\", \"arguments\": {\"response\": \"Visit Italy and cook a pizza\"}}");
}
if (user.contains("'Italy'")) {
return answer("{\"agentName\": \"suggestRecipe\", \"arguments\": {\"country\": \"Italy\"}}");
}
return answer("{\"agentName\": \"recommendCountry\", \"arguments\": {\"topic\": \"jazz\"}}");
}
private static ChatResponse answer(String text) {
return ChatResponse.builder().aiMessage(AiMessage.from(text)).build();
}
}
}from typing import Annotated
from dev.langchain4j.data.message import AiMessage, SystemMessage
from dev.langchain4j.model.chat import ChatModel
from dev.langchain4j.model.chat.request import ChatRequest
from dev.langchain4j.model.chat.response import ChatResponse
from jakarta.inject import Inject, Singleton
from micronaut.context.annotation import Primary, Property, Requires
from micronaut.test.extensions.junit5.annotation import MicronautTest
from org.junit.jupiter.api import Test
from example.micronaut.agentic.TravelSupervisorAgent import TravelSupervisorAgent
@Property(name="langchain4j.ollama.enabled", value="false")
@MicronautTest(startApplication=False, environments=["supervisor-test"])
class TravelSupervisorAgentTest:
supervisor: Annotated[TravelSupervisorAgent, Inject]
@Test
def supervisor_invokes_the_sub_agents(self):
assert self.supervisor.organize("I love jazz, plan my holidays") == "Visit Italy and cook a pizza"
def _answer(text: str) -> ChatResponse:
return ChatResponse.builder().aiMessage(AiMessage.from_(text)).build()
# Plays the planner of the supervisor: recommends a country, then a recipe, then answers.
@Singleton
@Primary
@Requires(env="supervisor-test")
class ScriptedPlannerChatModel(ChatModel):
def doChat(self, request: ChatRequest) -> ChatResponse:
messages = request.messages()
user = messages.get(messages.size() - 1).singleText()
planner = any(isinstance(message, SystemMessage) and "planner" in message.text() for message in messages)
if not planner:
return _answer("Italy" if "travel expert" in user else "Italy - Pizza")
if "'Italy - Pizza'" in user:
return _answer('{"agentName": "done", "arguments": {"response": "Visit Italy and cook a pizza"}}')
if "'Italy'" in user:
return _answer('{"agentName": "suggest_recipe", "arguments": {"country": "Italy"}}')
return _answer('{"agentName": "recommend_country", "arguments": {"topic": "jazz"}}')package example.micronaut.agentic
import dev.langchain4j.data.message.AiMessage
import dev.langchain4j.data.message.SystemMessage
import dev.langchain4j.data.message.UserMessage
import dev.langchain4j.model.chat.ChatModel
import dev.langchain4j.model.chat.request.ChatRequest
import dev.langchain4j.model.chat.response.ChatResponse
import io.micronaut.context.annotation.Primary
import io.micronaut.context.annotation.Property
import io.micronaut.context.annotation.Requires
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import jakarta.inject.Singleton
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Test
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false, environments = ["supervisor-test"])
internal class TravelSupervisorAgentTest {
@Test
fun supervisorInvokesTheSubAgents(supervisor: TravelSupervisorAgent) {
assertEquals("Visit Italy and cook a pizza", supervisor.organize("I love jazz, plan my holidays"))
}
/**
* Plays the planner of the supervisor: recommends a country, then a recipe, then answers.
*/
@Singleton
@Primary
@Requires(env = ["supervisor-test"])
class ScriptedPlannerChatModel : ChatModel {
override fun doChat(request: ChatRequest): ChatResponse {
val user = (request.messages().last() as UserMessage).singleText()
val planner = request.messages().any { it is SystemMessage && it.text().contains("planner") }
if (!planner) {
return answer(if (user.contains("travel expert")) "Italy" else "Italy - Pizza")
}
return when {
user.contains("'Italy - Pizza'") -> answer("""{"agentName": "done", "arguments": {"response": "Visit Italy and cook a pizza"}}""")
user.contains("'Italy'") -> answer("""{"agentName": "suggestRecipe", "arguments": {"country": "Italy"}}""")
else -> answer("""{"agentName": "recommendCountry", "arguments": {"topic": "jazz"}}""")
}
}
private fun answer(text: String): ChatResponse = ChatResponse.builder().aiMessage(AiMessage.from(text)).build()
}
}package example.micronaut.agentic
import dev.langchain4j.data.message.AiMessage
import dev.langchain4j.data.message.SystemMessage
import dev.langchain4j.data.message.UserMessage
import dev.langchain4j.model.chat.ChatModel
import dev.langchain4j.model.chat.request.ChatRequest
import dev.langchain4j.model.chat.response.ChatResponse
import io.micronaut.context.annotation.Primary
import io.micronaut.context.annotation.Property
import io.micronaut.context.annotation.Requires
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import jakarta.inject.Singleton
import org.junit.jupiter.api.Test
import static org.junit.jupiter.api.Assertions.assertEquals
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false, environments = "supervisor-test")
class TravelSupervisorAgentTest {
@Test
void supervisorInvokesTheSubAgents(TravelSupervisorAgent supervisor) {
assertEquals("Visit Italy and cook a pizza", supervisor.organize("I love jazz, plan my holidays"))
}
/**
* Plays the planner of the supervisor: recommends a country, then a recipe, then answers.
*/
@Singleton
@Primary
@Requires(env = "supervisor-test")
static class ScriptedPlannerChatModel implements ChatModel {
@Override
ChatResponse doChat(ChatRequest request) {
String user = ((UserMessage) request.messages().last()).singleText()
boolean planner = request.messages().any { it instanceof SystemMessage && it.text().contains("planner") }
if (!planner) {
return answer(user.contains("travel expert") ? "Italy" : "Italy - Pizza")
}
if (user.contains("'Italy - Pizza'")) {
return answer('{"agentName": "done", "arguments": {"response": "Visit Italy and cook a pizza"}}')
}
if (user.contains("'Italy'")) {
return answer('{"agentName": "suggestRecipe", "arguments": {"country": "Italy"}}')
}
answer('{"agentName": "recommendCountry", "arguments": {"topic": "jazz"}}')
}
private static ChatResponse answer(String text) {
ChatResponse.builder().aiMessage(AiMessage.from(text)).build()
}
}
}Supplier methods
LangChain4j agents can declare static supplier methods, such as @ChatModelSupplier, @ToolsSupplier, @ToolProviderSupplier, @ContentRetrieverSupplier, @ChatMemoryProviderSupplier, @ParallelExecutor or @McpClientSupplier. With Micronaut, the parameters of these methods are beans: annotate a parameter with @Named to select a bean by name.
LangChain4j calls a @ChatModelSupplier or @StreamingChatModelSupplier method that has parameters for each invocation of the agent within a workflow: a standalone typed agent ignores it.
|
MCP client agents
An MCP client agent calls a tool of an MCP server as a step of a workflow. Add the LangChain4j agentic MCP dependency, together with an MCP client, for example the one of Micronaut MCP:
implementation("dev.langchain4j:langchain4j-agentic-mcp")<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-agentic-mcp</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"dev.langchain4j:langchain4j-agentic-mcp",
]
implementation("io.micronaut.mcp:micronaut-mcp-client-langchain4j")<dependency>
<groupId>io.micronaut.mcp</groupId>
<artifactId>micronaut-mcp-client-langchain4j</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.mcp:micronaut-mcp-client-langchain4j",
]The @McpClientSupplier method receives the McpClient bean of the server:
package example.micronaut.agentic;
import dev.langchain4j.agentic.declarative.McpClientAgent;
import dev.langchain4j.agentic.declarative.McpClientSupplier;
import dev.langchain4j.agentic.declarative.SequenceAgent;
import dev.langchain4j.mcp.client.McpClient;
import dev.langchain4j.service.V;
import io.micronaut.context.annotation.Requires;
import io.micronaut.langchain4j.agentic.annotation.AgenticService;
import jakarta.inject.Named;
/**
* A workflow whose sub-agent calls a tool of an MCP server through a Micronaut MCP client bean.
*/
@Requires(beans = McpClient.class)
@AgenticService
public interface WorldClockAgent {
@SequenceAgent(outputKey = "time", subAgents = CurrentTimeAgent.class)
String time(@V("zone") String zone);
interface CurrentTimeAgent {
@McpClientAgent(toolName = "current-time", outputKey = "time") // (1)
String currentTime(@V("zone") String zone);
@McpClientSupplier
static McpClient mcpClient(@Named("time-server") McpClient mcpClient) { // (2)
return mcpClient;
}
}
}from abc import ABC, abstractmethod
from typing import Annotated
from dev.langchain4j.agentic.declarative import McpClientAgent, McpClientSupplier, SequenceAgent
from dev.langchain4j.mcp.client import McpClient
from dev.langchain4j.service import V
from jakarta.inject import Named
from micronaut.context.annotation import Requires
from micronaut.langchain4j.agentic.annotation import AgenticService
# In Python a static function compiles to a static interface method of an introduction type, so the sub-agent
# is an agentic service too
@Requires(beans=[McpClient])
@AgenticService
class CurrentTimeAgent(ABC):
@McpClientAgent(toolName="current-time", outputKey="time") # (1)
@abstractmethod
def current_time(self, zone: Annotated[str, V("zone")]) -> str:
...
@staticmethod
@McpClientSupplier
def mcp_client(mcp_client: Annotated[McpClient, Named("time-server")]) -> McpClient: # (2)
return mcp_client
# A workflow whose sub-agent calls a tool of an MCP server through a Micronaut MCP client bean.
@Requires(beans=[McpClient])
@AgenticService
class WorldClockAgent(ABC):
@SequenceAgent(outputKey="time", subAgents=[CurrentTimeAgent])
@abstractmethod
def time(self, zone: Annotated[str, V("zone")]) -> str:
...package example.micronaut.agentic
import dev.langchain4j.agentic.declarative.McpClientAgent
import dev.langchain4j.agentic.declarative.McpClientSupplier
import dev.langchain4j.agentic.declarative.SequenceAgent
import dev.langchain4j.mcp.client.McpClient
import dev.langchain4j.service.V
import io.micronaut.context.annotation.Requires
import io.micronaut.langchain4j.agentic.annotation.AgenticService
import jakarta.inject.Named
/**
* A workflow whose sub-agent calls a tool of an MCP server through a Micronaut MCP client bean.
*/
@Requires(beans = [McpClient::class])
@AgenticService
interface WorldClockAgent {
@SequenceAgent(outputKey = "time", subAgents = [CurrentTimeAgent::class])
fun time(@V("zone") zone: String): String
interface CurrentTimeAgent {
@McpClientAgent(toolName = "current-time", outputKey = "time") // (1)
fun currentTime(@V("zone") zone: String): String
companion object {
@McpClientSupplier
@JvmStatic
fun mcpClient(@Named("time-server") mcpClient: McpClient): McpClient = mcpClient // (2)
}
}
}package example.micronaut.agentic
import dev.langchain4j.agentic.declarative.McpClientAgent
import dev.langchain4j.agentic.declarative.McpClientSupplier
import dev.langchain4j.agentic.declarative.SequenceAgent
import dev.langchain4j.mcp.client.McpClient
import dev.langchain4j.service.V
import io.micronaut.context.annotation.Requires
import io.micronaut.langchain4j.agentic.annotation.AgenticService
import jakarta.inject.Named
/**
* A workflow whose sub-agent calls a tool of an MCP server through a Micronaut MCP client bean.
*/
@Requires(beans = McpClient)
@AgenticService
interface WorldClockAgent {
@SequenceAgent(outputKey = "time", subAgents = CurrentTimeAgent)
String time(@V("zone") String zone)
interface CurrentTimeAgent {
@McpClientAgent(toolName = "current-time", outputKey = "time") // (1)
String currentTime(@V("zone") String zone)
@McpClientSupplier
static McpClient mcpClient(@Named("time-server") McpClient mcpClient) { // (2)
mcpClient
}
}
}| 1 | The tool of the MCP server to call. The arguments of the tool are the parameters of the method, read from the agentic scope. |
| 2 | The Micronaut MCP client bean named time-server, configured under micronaut.mcp.client.http.time-server (see Tools for an MCP client configuration). |
In Python, a class with a static function compiles to a Java interface only when it is an introduction type, so the MCP client agent above is an @AgenticService too.
|
A2A client agents
LangChain4j’s @A2AClientAgent invokes a remote agent with the Agent2Agent (A2A) protocol. It requires the dev.langchain4j:langchain4j-agentic-a2a dependency, and the URL of the remote agent server is set with the a2aServerUrl attribute or returned by a static @A2AServerUrlSupplier method without parameters. A static @A2AClientCustomizer method can customize the A2A client.
6 Structured Outputs
When an @AiService or @AgenticService method returns a POJO (or a List or Set of POJOs), LangChain4j asks the model for a JSON answer: it derives a JSON schema from the return type reflectively and sends it as the response format, or appends format instructions to the user message when the model does not support JSON schemas.
With Micronaut JSON Schema, the schema is generated at compile time instead, and the generated schema is the one the model receives. Compared with the schema LangChain4j derives:
-
the descriptions come from the Javadoc (or KDoc, or Python docstrings) of the type and its properties, and the validation constraints (
format,pattern,minimum, …) are passed on to the model in the descriptions; -
no reflection is needed to build the schema, which suits GraalVM native images;
-
the schema only contains the properties of the type, whereas the reflective schema of a class generated by a compiler (for example, the Java class of a Python class) can contain synthetic or internal fields.
Add the JSON Schema annotation processor and the runtime support:
annotationProcessor("io.micronaut.jsonschema:micronaut-json-schema-processor")<annotationProcessorPaths>
<path>
<groupId>io.micronaut.jsonschema</groupId>
<artifactId>micronaut-json-schema-processor</artifactId>
</path>
</annotationProcessorPaths>[tool.pyronaut.dependencies]
build = [
"io.micronaut.jsonschema:micronaut-json-schema-processor",
]
implementation("io.micronaut.jsonschema:micronaut-json-schema-utils")<dependency>
<groupId>io.micronaut.jsonschema</groupId>
<artifactId>micronaut-json-schema-utils</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.jsonschema:micronaut-json-schema-utils",
]Then annotate the types returned by your services with io.micronaut.jsonschema.JsonSchema:
A structured output type
package example.structured;
import io.micronaut.jsonschema.JsonSchema;
/**
* A conference talk recommended to an attendee.
*
* @param title The title of the talk
* @param reason Why the talk matches the interests of the attendee
* @param score How well the talk matches the interests, from 1 to 5
*/
@JsonSchema // (1)
public record TalkRecommendation(String title, String reason, int score) {
}A structured output type
from dataclasses import dataclass
from micronaut.jsonschema import JsonSchema
from micronaut.core.annotation import Introspected
@JsonSchema # (1)
@Introspected
@dataclass
class TalkRecommendation:
"""A conference talk recommended to an attendee."""
title: str
"""The title of the talk"""
reason: str
"""Why the talk matches the interests of the attendee"""
score: int
"""How well the talk matches the interests, from 1 to 5"""A structured output type
package example.structured
import io.micronaut.jsonschema.JsonSchema
/**
* A conference talk recommended to an attendee.
*/
@JsonSchema // (1)
data class TalkRecommendation(
/** The title of the talk */
var title: String = "",
/** Why the talk matches the interests of the attendee */
var reason: String = "",
/** How well the talk matches the interests, from 1 to 5 */
var score: Int = 0,
)A structured output type
package example.structured
import groovy.transform.EqualsAndHashCode
import io.micronaut.jsonschema.JsonSchema
@JsonSchema(description = "A conference talk recommended to an attendee.") // (1)
@EqualsAndHashCode
class TalkRecommendation {
String title
String reason
int score
}| 1 | Micronaut JSON Schema generates the schema of the type, with its documentation as descriptions. |
The Groovy compiler does not keep the Groovydoc of the classes, so set the description with the description member of @JsonSchema.
|
The class is declared @Introspected so that LangChain4j, which parses the answer of the model with Jackson, can construct it. In Python, do not name the @JsonSchema types in the micronaut.introspection.allow-reflection compiler option (the schema would be generated twice, failing the compilation): the integration does not need the annotation at runtime.
|
Micronaut JSON Schema describes the properties of a type: the components of a record, the accessors of a bean, or the fields of a class annotated with @Introspected(accessKind = Introspected.AccessKind.FIELD). LangChain4j also reads and writes plain fields, which are not properties for Micronaut. For a class whose state is only held in such fields, the generated schema has no properties: a warning is logged, and the schema LangChain4j derives is sent instead.
|
No other change is needed: the AI services and agents returning these types send the generated schemas.
An
@AiService returning structured outputspackage example.micronaut.aiservice.structured;
import dev.langchain4j.service.SystemMessage;
import example.structured.TalkRecommendation;
import io.micronaut.langchain4j.annotation.AiService;
import java.util.List;
@AiService
public interface TalkAdvisor {
@SystemMessage("You recommend conference talks to attendees.")
TalkRecommendation recommend(String interests); // (1)
@SystemMessage("You recommend conference talks to attendees.")
List<TalkRecommendation> shortlist(String interests); // (2)
}An
@AiService returning structured outputsfrom abc import ABC, abstractmethod
from dev.langchain4j.service import SystemMessage
from micronaut.langchain4j.annotation import AiService
from example.structured.TalkRecommendation import TalkRecommendation
@AiService
class TalkAdvisor(ABC):
@SystemMessage("You recommend conference talks to attendees.")
@abstractmethod
def recommend(self, interests: str) -> TalkRecommendation: # (1)
...
@SystemMessage("You recommend conference talks to attendees.")
@abstractmethod
def shortlist(self, interests: str) -> list[TalkRecommendation]: # (2)
...An
@AiService returning structured outputspackage example.micronaut.aiservice.structured
import example.structured.TalkRecommendation
import dev.langchain4j.service.SystemMessage
import io.micronaut.langchain4j.annotation.AiService
@AiService
interface TalkAdvisor {
@SystemMessage("You recommend conference talks to attendees.")
fun recommend(interests: String): TalkRecommendation // (1)
@SystemMessage("You recommend conference talks to attendees.")
fun shortlist(interests: String): List<TalkRecommendation> // (2)
}An
@AiService returning structured outputspackage example.micronaut.aiservice.structured
import example.structured.TalkRecommendation
import dev.langchain4j.service.SystemMessage
import io.micronaut.langchain4j.annotation.AiService
@AiService
interface TalkAdvisor {
@SystemMessage("You recommend conference talks to attendees.")
TalkRecommendation recommend(String interests) // (1)
@SystemMessage("You recommend conference talks to attendees.")
List<TalkRecommendation> shortlist(String interests) // (2)
}| 1 | The schema generated for TalkRecommendation is sent as the response format. |
| 2 | For a list, LangChain4j asks for an object whose values property holds the elements: the generated schema describes the elements. |
Checking the schema received by the model
package example.micronaut.aiservice.structured;
import dev.langchain4j.model.chat.request.json.JsonArraySchema;
import dev.langchain4j.model.chat.request.json.JsonObjectSchema;
import dev.langchain4j.model.chat.request.json.JsonSchema;
import example.structured.TalkRecommendation;
import io.micronaut.langchain4j.jsonschema.StructuredOutputSchemaProvider;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Inject;
import org.junit.jupiter.api.Test;
import java.util.List;
import java.util.Set;
import static org.junit.jupiter.api.Assertions.assertEquals;
@MicronautTest(startApplication = false, environments = "structured-output")
class StructuredOutputTest {
@Inject
TalkAdvisor talkAdvisor;
@Inject
StructuredOutputChatModel chatModel;
@Inject
StructuredOutputSchemaProvider schemaProvider;
@Test
void sendsTheGeneratedSchema() {
TalkRecommendation recommendation = talkAdvisor.recommend("LLMs and Java"); // (1)
assertEquals(new TalkRecommendation("Structured outputs", "You like LLMs", 5), recommendation);
JsonSchema schema = chatModel.lastRequest().responseFormat().jsonSchema(); // (2)
assertEquals("TalkRecommendation", schema.name());
assertEquals(schemaProvider.findSchema(TalkRecommendation.class).orElseThrow(), schema.rootElement()); // (3)
JsonObjectSchema root = (JsonObjectSchema) schema.rootElement();
assertEquals("A conference talk recommended to an attendee.", root.description());
assertEquals(Set.of("title", "reason", "score"), root.properties().keySet());
assertEquals("Why the talk matches the interests of the attendee", root.properties().get("reason").description());
}
@Test
void sendsTheGeneratedSchemaOfListElements() {
List<TalkRecommendation> shortlist = talkAdvisor.shortlist("LLMs and Java");
assertEquals(1, shortlist.size());
JsonSchema schema = chatModel.lastRequest().responseFormat().jsonSchema();
assertEquals("List_of_TalkRecommendation", schema.name());
JsonArraySchema values = (JsonArraySchema) ((JsonObjectSchema) schema.rootElement()).properties().get("values");
assertEquals(schemaProvider.findSchema(TalkRecommendation.class).orElseThrow(), values.items());
}
}Checking the schema received by the model
from typing import Annotated
from micronaut.langchain4j.jsonschema import StructuredOutputSchemaProvider
from jakarta.inject import Inject
from micronaut.test.extensions.junit5.annotation import MicronautTest
from org.junit.jupiter.api import Test
from example.micronaut.aiservice.structured.StructuredOutputChatModel import StructuredOutputChatModel
from example.micronaut.aiservice.structured.TalkAdvisor import TalkAdvisor
from example.structured.TalkRecommendation import TalkRecommendation
@MicronautTest(startApplication=False, environments=["structured-output"])
class StructuredOutputTest:
talk_advisor: Annotated[TalkAdvisor, Inject]
chat_model: Annotated[StructuredOutputChatModel, Inject]
schema_provider: Annotated[StructuredOutputSchemaProvider, Inject]
@Test
def test_sends_the_generated_schema(self):
recommendation = self.talk_advisor.recommend("LLMs and Java") # (1)
assert recommendation.title == "Structured outputs"
assert recommendation.score == 5
schema = self.chat_model.last_request().responseFormat().jsonSchema() # (2)
assert schema.name() == "TalkRecommendation"
assert schema.rootElement().equals(self.schema_provider.findSchema(TalkRecommendation).orElseThrow()) # (3)
root = schema.rootElement()
assert root.description() == "A conference talk recommended to an attendee."
assert set(str(name) for name in root.properties().keySet()) == {"title", "reason", "score"}
@Test
def test_sends_the_generated_schema_of_list_elements(self):
shortlist = self.talk_advisor.shortlist("LLMs and Java")
assert len(shortlist) == 1
schema = self.chat_model.last_request().responseFormat().jsonSchema()
assert schema.name() == "List_of_TalkRecommendation"
values = schema.rootElement().properties().get("values")
assert values.items().equals(self.schema_provider.findSchema(TalkRecommendation).orElseThrow())Checking the schema received by the model
package example.micronaut.aiservice.structured
import example.structured.TalkRecommendation
import dev.langchain4j.model.chat.request.json.JsonArraySchema
import dev.langchain4j.model.chat.request.json.JsonObjectSchema
import io.micronaut.langchain4j.jsonschema.StructuredOutputSchemaProvider
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import jakarta.inject.Inject
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Test
@MicronautTest(startApplication = false, environments = ["structured-output"])
class StructuredOutputTest {
@Inject
lateinit var talkAdvisor: TalkAdvisor
@Inject
lateinit var chatModel: StructuredOutputChatModel
@Inject
lateinit var schemaProvider: StructuredOutputSchemaProvider
@Test
fun sendsTheGeneratedSchema() {
val recommendation = talkAdvisor.recommend("LLMs and Java") // (1)
assertEquals(TalkRecommendation("Structured outputs", "You like LLMs", 5), recommendation)
val schema = chatModel.lastRequest!!.responseFormat().jsonSchema() // (2)
assertEquals("TalkRecommendation", schema.name())
assertEquals(schemaProvider.findSchema(TalkRecommendation::class.java).orElseThrow(), schema.rootElement()) // (3)
val root = schema.rootElement() as JsonObjectSchema
assertEquals("A conference talk recommended to an attendee.", root.description())
assertEquals(setOf("title", "reason", "score"), root.properties().keys)
assertEquals("Why the talk matches the interests of the attendee", root.properties()["reason"]!!.description())
}
@Test
fun sendsTheGeneratedSchemaOfListElements() {
val shortlist = talkAdvisor.shortlist("LLMs and Java")
assertEquals(1, shortlist.size)
val schema = chatModel.lastRequest!!.responseFormat().jsonSchema()
assertEquals("List_of_TalkRecommendation", schema.name())
val values = (schema.rootElement() as JsonObjectSchema).properties()["values"] as JsonArraySchema
assertEquals(schemaProvider.findSchema(TalkRecommendation::class.java).orElseThrow(), values.items())
}
}Checking the schema received by the model
package example.micronaut.aiservice.structured
import example.structured.TalkRecommendation
import dev.langchain4j.model.chat.request.json.JsonArraySchema
import dev.langchain4j.model.chat.request.json.JsonObjectSchema
import dev.langchain4j.model.chat.request.json.JsonSchema
import io.micronaut.langchain4j.jsonschema.StructuredOutputSchemaProvider
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import jakarta.inject.Inject
import org.junit.jupiter.api.Test
import static org.junit.jupiter.api.Assertions.assertEquals
@MicronautTest(startApplication = false, environments = "structured-output")
class StructuredOutputTest {
@Inject
TalkAdvisor talkAdvisor
@Inject
StructuredOutputChatModel chatModel
@Inject
StructuredOutputSchemaProvider schemaProvider
@Test
void sendsTheGeneratedSchema() {
TalkRecommendation recommendation = talkAdvisor.recommend("LLMs and Java") // (1)
assertEquals(new TalkRecommendation(title: "Structured outputs", reason: "You like LLMs", score: 5), recommendation)
JsonSchema schema = chatModel.lastRequest.responseFormat().jsonSchema() // (2)
assertEquals("TalkRecommendation", schema.name())
assertEquals(schemaProvider.findSchema(TalkRecommendation).orElseThrow(), schema.rootElement()) // (3)
JsonObjectSchema root = (JsonObjectSchema) schema.rootElement()
assertEquals("A conference talk recommended to an attendee.", root.description())
assertEquals(["title", "reason", "score"] as Set, root.properties().keySet())
}
@Test
void sendsTheGeneratedSchemaOfListElements() {
List<TalkRecommendation> shortlist = talkAdvisor.shortlist("LLMs and Java")
assertEquals(1, shortlist.size())
JsonSchema schema = chatModel.lastRequest.responseFormat().jsonSchema()
assertEquals("List_of_TalkRecommendation", schema.name())
JsonArraySchema values = (JsonArraySchema) ((JsonObjectSchema) schema.rootElement()).properties().get("values")
assertEquals(schemaProvider.findSchema(TalkRecommendation).orElseThrow(), values.items())
}
}| 1 | Call the service as usual. |
| 2 | The test chat model records the request it receives. |
| 3 | The response format carries the schema generated at compile time. |
The schema of a type returned directly, in a List or a Set, or wrapped in Result, CompletableFuture or CompletionStage is replaced. References to the schemas of other @JsonSchema types are resolved and inlined, except recursive ones, which are sent as definitions ($defs). LangChain4j still parses the answer and decides whether the response format is sent; streaming methods are unaffected.
To supply the schemas from another source, register a bean of type StructuredOutputSchemaProvider.
Models without JSON schema support
LangChain4j only sends the schema to the chat models that declare the RESPONSE_FORMAT_JSON_SCHEMA capability. For the other models, it appends format instructions derived from the type to the user message, and the generated schema is not used.
Some models accept a JSON schema on each request but do not always declare the capability. Google AI Gemini declares it only when a JSON response format is configured on the model, yet it honors the response format of each request: with the micronaut-langchain4j-googleai-gemini module, the services returning a type with a generated schema send it to Gemini without further configuration. Providers declare such models with a bean of type JsonSchemaResponseFormatSupport.
For other models, set langchain4j.structured-output.force-json-schema-response-format to true to send the schema anyway, or to false to never send it to the models that do not declare the capability:
langchain4j.structured-output.force-json-schema-response-format=truelangchain4j.structured-output.force-json-schema-response-format: true"langchain4j.structured-output.force-json-schema-response-format" = truelangchain4j.structuredOutput.forceJsonSchemaResponseFormat = true{
"langchain4j.structured-output.force-json-schema-response-format" = true
}{
"langchain4j.structured-output.force-json-schema-response-format": true
}Only use it with models that support a JSON schema response format: a model that does not may ignore it or reject the request.
To use the schemas LangChain4j derives instead of the generated ones, set langchain4j.structured-output.enabled to false.
7 Testing
Micronaut LangChain4j includes a small evaluation API for asserting AI responses in tests.
The EvaluationRequest record captures the original user text, optional grounding context, and generated response.
An Evaluator consumes that request and returns an EvaluationResult.
Built-in evaluators include:
-
RelevancyEvaluatorfor checking whether the response answers the user request. -
FactCheckingEvaluatorfor checking whether the response is grounded in the supplied context.
When an AI service returns dev.langchain4j.service.Result<T>, you can reuse retrieved sources as evaluation context:
Defining an
@AiService that returns Result<String>package example.micronaut.aiservice.evaluation;
import dev.langchain4j.service.Result;
import dev.langchain4j.service.SystemMessage;
import io.micronaut.context.annotation.Requires;
import io.micronaut.langchain4j.annotation.AiService;
@Requires(property = "spec.name", value = "AiServiceEvaluationExample")
@AiService
public interface EvaluatingFriend {
@SystemMessage("You are a good friend of mine. Answer using slang.")
Result<String> chat(String userMessage);
}Defining an
@AiService that returns Result<String>from abc import ABC, abstractmethod
from dev.langchain4j.service import Result, SystemMessage
from micronaut.context.annotation import Requires
from micronaut.langchain4j.annotation import AiService
@Requires(property="spec.name", value="AiServiceEvaluationExample")
@AiService
class EvaluatingFriend(ABC):
@SystemMessage("You are a good friend of mine. Answer using slang.")
@abstractmethod
def chat(self, user_message: str) -> Result[str]:
...Defining an
@AiService that returns Result<String>package example.micronaut.aiservice.evaluation
import dev.langchain4j.service.Result
import dev.langchain4j.service.SystemMessage
import io.micronaut.context.annotation.Requires
import io.micronaut.langchain4j.annotation.AiService
@Requires(property = "spec.name", value = "AiServiceEvaluationExample")
@AiService
interface EvaluatingFriend {
@SystemMessage("You are a good friend of mine. Answer using slang.")
fun chat(userMessage: String): Result<String>
}Defining an
@AiService that returns Result<String>package example.micronaut.aiservice.evaluation
import dev.langchain4j.service.Result
import dev.langchain4j.service.SystemMessage
import io.micronaut.context.annotation.Requires
import io.micronaut.langchain4j.annotation.AiService
@Requires(property = "spec.name", value = "AiServiceEvaluationExample")
@AiService
interface EvaluatingFriend {
@SystemMessage("You are a good friend of mine. Answer using slang.")
Result<String> chat(String userMessage)
}Evaluating an AI service response
package example.micronaut.aiservice.evaluation;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.service.Result;
import io.micronaut.context.annotation.Property;
import io.micronaut.langchain4j.evaluation.EvaluationRequest;
import io.micronaut.langchain4j.evaluation.EvaluationResult;
import io.micronaut.langchain4j.evaluation.RelevancyEvaluator;
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.testcontainers.junit.jupiter.Testcontainers;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
@Property(name = "spec.name", value = "AiServiceEvaluationExample")
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class AiServiceEvaluationExample implements OllamaTestPropertyProvider {
@Test
void evaluatesAiServiceResponse(EvaluatingFriend friend, ChatModel chatModel) {
String userText = "Reply with exactly: Micronaut is a JVM framework.";
Result<String> response = friend.chat(userText);
RelevancyEvaluator evaluator = new RelevancyEvaluator(chatModel);
EvaluationResult evaluation = evaluator.evaluate(EvaluationRequest.from(userText, response));
assertNotNull(evaluation);
assertFalse(evaluation.feedback().isBlank());
}
}Evaluating an AI service response
from typing import Annotated
from dev.langchain4j.model.chat import ChatModel
from jakarta.inject import Inject
from micronaut.context.annotation import Property
from micronaut.langchain4j.evaluation import EvaluationRequest, RelevancyEvaluator
from micronaut.test.extensions.junit5.annotation import MicronautTest
from org.junit.jupiter.api import Test
from example.micronaut.aiservice.evaluation.EvaluatingFriend import EvaluatingFriend
@Property(name="spec.name", value="AiServiceEvaluationExample")
@MicronautTest(startApplication=False, environments=["ollama"])
class AiServiceEvaluationExample:
friend: Annotated[EvaluatingFriend, Inject]
chat_model: Annotated[ChatModel, Inject]
@Test
def evaluates_ai_service_response(self):
user_text = "Reply with exactly: Micronaut is a JVM framework."
response = self.friend.chat(user_text)
evaluator = RelevancyEvaluator(self.chat_model)
evaluation = evaluator.evaluate(EvaluationRequest.from_(user_text, response))
assert evaluation is not None
assert evaluation.feedback().strip() != ""Evaluating an AI service response
package example.micronaut.aiservice.evaluation
import dev.langchain4j.model.chat.ChatModel
import io.micronaut.context.annotation.Property
import io.micronaut.langchain4j.evaluation.EvaluationRequest
import io.micronaut.langchain4j.evaluation.RelevancyEvaluator
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import org.junit.jupiter.api.Assertions.assertFalse
import org.junit.jupiter.api.Assertions.assertNotNull
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import org.testcontainers.junit.jupiter.Testcontainers
@Property(name = "spec.name", value = "AiServiceEvaluationExample")
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
internal class AiServiceEvaluationExample : OllamaTestPropertyProvider {
@Test
fun evaluatesAiServiceResponse(friend: EvaluatingFriend, chatModel: ChatModel) {
val userText = "Reply with exactly: Micronaut is a JVM framework."
val response = friend.chat(userText)
val evaluator = RelevancyEvaluator(chatModel)
val evaluation = evaluator.evaluate(EvaluationRequest.from(userText, response))
assertNotNull(evaluation)
assertFalse(evaluation.feedback().isBlank())
}
}Evaluating an AI service response
package example.micronaut.aiservice.evaluation
import dev.langchain4j.model.chat.ChatModel
import dev.langchain4j.service.Result
import io.micronaut.context.annotation.Property
import io.micronaut.langchain4j.evaluation.EvaluationRequest
import io.micronaut.langchain4j.evaluation.EvaluationResult
import io.micronaut.langchain4j.evaluation.RelevancyEvaluator
import io.micronaut.langchain4j.testutils.OllamaTestPropertyProvider
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import org.testcontainers.junit.jupiter.Testcontainers
import static org.junit.jupiter.api.Assertions.assertFalse
import static org.junit.jupiter.api.Assertions.assertNotNull
@Property(name = "spec.name", value = "AiServiceEvaluationExample")
@Testcontainers(disabledWithoutDocker = true)
@MicronautTest(startApplication = false)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class AiServiceEvaluationExample implements OllamaTestPropertyProvider {
@Test
void evaluatesAiServiceResponse(EvaluatingFriend friend, ChatModel chatModel) {
String userText = "Reply with exactly: Micronaut is a JVM framework."
Result<String> response = friend.chat(userText)
RelevancyEvaluator evaluator = new RelevancyEvaluator(chatModel)
EvaluationResult evaluation = evaluator.evaluate(EvaluationRequest.from(userText, response))
assertNotNull(evaluation)
assertFalse(evaluation.feedback().isBlank())
}
}FactCheckingEvaluator requires non-empty context, which makes it a good fit for RAG-style responses backed by retrieved sources.
Evaluation samples
The Micronaut LangChain4j test module runs evaluations from sample files and provides a scripted chat model for tests. Add it to the test classpath:
testImplementation("io.micronaut.langchain4j:micronaut-langchain4j-test")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-test</artifactId>
<scope>test</scope>
</dependency>[tool.pyronaut.dependencies]
test = [
"io.micronaut.langchain4j:micronaut-langchain4j-test",
]A sample file lists the inputs to send to the AI service, with the optional context that the answers must be grounded in, for example src/test/resources/samples/support.yml:
samples:
- name: framework
input: What is Micronaut?
context: Micronaut is a modern JVM framework for building microservices and serverless applications.
- name: native
input: Can a Micronaut application be compiled to a native executable?
context: Micronaut applications can be compiled to native executables with GraalVM.A JUnit parameterized test then evaluates the answer to each sample:
package example.micronaut.testing;
import dev.langchain4j.model.chat.ChatModel;
import io.micronaut.context.annotation.Bean;
import io.micronaut.context.annotation.Factory;
import io.micronaut.context.annotation.Primary;
import io.micronaut.context.annotation.Property;
import io.micronaut.context.annotation.Requires;
import io.micronaut.langchain4j.evaluation.RelevancyEvaluator;
import io.micronaut.langchain4j.test.EvaluationSample;
import io.micronaut.langchain4j.test.EvaluationSamplesSource;
import io.micronaut.langchain4j.test.ScriptedChatModel;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Inject;
import jakarta.inject.Named;
import org.junit.jupiter.params.ParameterizedTest;
import static io.micronaut.langchain4j.test.EvaluationAssertions.assertPasses;
@Property(name = "spec.name", value = "SupportAssistantEvaluationTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@Property(name = "langchain4j.evaluation.chat-model", value = "judge") // (1)
@MicronautTest(startApplication = false)
class SupportAssistantEvaluationTest {
@Inject
SupportAssistant assistant;
@Inject
RelevancyEvaluator relevancy; // (2)
@ParameterizedTest
@EvaluationSamplesSource("samples/support.yml") // (3)
void answersEachSample(EvaluationSample sample) {
String answer = assistant.answer(sample.input());
assertPasses(sample, relevancy.evaluate(sample.request(answer))); // (4)
}
@Factory
@Requires(property = "spec.name", value = "SupportAssistantEvaluationTest")
static class Models {
@Bean
@Primary
ChatModel assistantModel() { // (5)
return ScriptedChatModel.builder()
.whenUserMessageContains("native").respond("Yes, with GraalVM.")
.otherwise("Micronaut is a JVM framework for microservices.")
.build();
}
@Bean
@Named("judge")
ChatModel judge() {
return ScriptedChatModel.respondingWith("PASS\nThe answer is relevant.");
}
}
}from typing import Annotated
from dev.langchain4j.model.chat import ChatModel
from jakarta.inject import Inject, Named
from micronaut.context.annotation import Bean, Factory, Primary, Property, Requires
from micronaut.langchain4j.evaluation import RelevancyEvaluator
from micronaut.langchain4j.test import EvaluationAssertions, EvaluationSamples, ScriptedChatModel
from micronaut.test.extensions.junit5.annotation import MicronautTest
from org.junit.jupiter.api import Test
from example.micronaut.testing.SupportAssistant import SupportAssistant
@Property(name="spec.name", value="SupportAssistantEvaluationTest")
@Property(name="langchain4j.ollama.enabled", value="false")
@Property(name="langchain4j.evaluation.chat-model", value="judge") # (1)
@MicronautTest(startApplication=False)
class SupportAssistantEvaluationTest:
assistant: Annotated[SupportAssistant, Inject]
relevancy: Annotated[RelevancyEvaluator, Inject] # (2)
@Test
def answers_each_sample(self):
for sample in EvaluationSamples.load("samples/support.yml"): # (3)
answer = self.assistant.answer(sample.input())
EvaluationAssertions.assertPasses(sample, self.relevancy.evaluate(sample.request(answer))) # (4)
@Requires(property="spec.name", value="SupportAssistantEvaluationTest")
@Factory
class Models:
@Bean
@Primary
def assistant_model(self) -> ChatModel: # (5)
return ScriptedChatModel.builder() \
.whenUserMessageContains("native").respond("Yes, with GraalVM.") \
.otherwise("Micronaut is a JVM framework for microservices.") \
.build()
@Bean
@Named("judge")
def judge(self) -> ChatModel:
return ScriptedChatModel.respondingWith("PASS\nThe answer is relevant.")package example.micronaut.testing
import dev.langchain4j.model.chat.ChatModel
import io.micronaut.context.annotation.Bean
import io.micronaut.context.annotation.Factory
import io.micronaut.context.annotation.Primary
import io.micronaut.context.annotation.Property
import io.micronaut.context.annotation.Requires
import io.micronaut.langchain4j.evaluation.RelevancyEvaluator
import io.micronaut.langchain4j.test.EvaluationAssertions.assertPasses
import io.micronaut.langchain4j.test.EvaluationSample
import io.micronaut.langchain4j.test.EvaluationSamplesSource
import io.micronaut.langchain4j.test.ScriptedChatModel
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import jakarta.inject.Inject
import jakarta.inject.Named
import org.junit.jupiter.params.ParameterizedTest
@Property(name = "spec.name", value = "SupportAssistantEvaluationTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@Property(name = "langchain4j.evaluation.chat-model", value = "judge") // (1)
@MicronautTest(startApplication = false)
internal class SupportAssistantEvaluationTest {
@Inject
lateinit var assistant: SupportAssistant
@Inject
lateinit var relevancy: RelevancyEvaluator // (2)
@ParameterizedTest
@EvaluationSamplesSource("samples/support.yml") // (3)
fun answersEachSample(sample: EvaluationSample) {
val answer = assistant.answer(sample.input())
assertPasses(sample, relevancy.evaluate(sample.request(answer))) // (4)
}
@Factory
@Requires(property = "spec.name", value = "SupportAssistantEvaluationTest")
class Models {
@Bean
@Primary
fun assistantModel(): ChatModel = // (5)
ScriptedChatModel.builder()
.whenUserMessageContains("native").respond("Yes, with GraalVM.")
.otherwise("Micronaut is a JVM framework for microservices.")
.build()
@Bean
@Named("judge")
fun judge(): ChatModel = ScriptedChatModel.respondingWith("PASS\nThe answer is relevant.")
}
}package example.micronaut.testing
import dev.langchain4j.model.chat.ChatModel
import io.micronaut.context.annotation.Bean
import io.micronaut.context.annotation.Factory
import io.micronaut.context.annotation.Primary
import io.micronaut.context.annotation.Property
import io.micronaut.context.annotation.Requires
import io.micronaut.langchain4j.evaluation.RelevancyEvaluator
import io.micronaut.langchain4j.test.EvaluationSample
import io.micronaut.langchain4j.test.EvaluationSamplesSource
import io.micronaut.langchain4j.test.ScriptedChatModel
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import jakarta.inject.Inject
import jakarta.inject.Named
import org.junit.jupiter.params.ParameterizedTest
import static io.micronaut.langchain4j.test.EvaluationAssertions.assertPasses
@Property(name = "spec.name", value = "SupportAssistantEvaluationTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@Property(name = "langchain4j.evaluation.chat-model", value = "judge") // (1)
@MicronautTest(startApplication = false)
class SupportAssistantEvaluationTest {
@Inject
SupportAssistant assistant
@Inject
RelevancyEvaluator relevancy // (2)
@ParameterizedTest
@EvaluationSamplesSource("samples/support.yml") // (3)
void answersEachSample(EvaluationSample sample) {
String answer = assistant.answer(sample.input())
assertPasses(sample, relevancy.evaluate(sample.request(answer))) // (4)
}
@Factory
@Requires(property = "spec.name", value = "SupportAssistantEvaluationTest")
static class Models {
@Bean
@Primary
ChatModel assistantModel() { // (5)
ScriptedChatModel.builder()
.whenUserMessageContains("native").respond("Yes, with GraalVM.")
.otherwise("Micronaut is a JVM framework for microservices.")
.build()
}
@Bean
@Named("judge")
ChatModel judge() {
ScriptedChatModel.respondingWith("PASS\nThe answer is relevant.")
}
}
}| 1 | The module exposes RelevancyEvaluator and FactCheckingEvaluator beans, judged by the ChatModel bean named by langchain4j.evaluation.chat-model, otherwise by the default ChatModel. In a real evaluation the judge is a capable model. |
| 2 | The evaluator is injected like any bean. |
| 3 | @EvaluationSamplesSource runs the test once per sample of the file, which EvaluationSamples.load(…) also reads. |
| 4 | EvaluationAssertions.assertPasses fails the test with the feedback of the evaluator. |
| 5 | ScriptedChatModel answers with the first rule that matches the request and records the requests, so a test does not need a real model. |
| In Python, the test loops over the samples, since JUnit parameterized tests are not available. |
ScriptedChatModel can also request tool calls and stream:
-
whenUserMessageContains(text),whenToolResult()andwhen(predicate)select the requests, andrespond(…)orcallTool(name, argumentsJson)answer them, in order.otherwise(…)answers the other requests. -
ScriptedChatModel.respondingWith(responses…)answers the requests with the responses, in order. -
streaming()returns aStreamingChatModelwith the same rules, which streams the text of the responses word by word. -
requests()returns the requests received, for example to check the messages sent to the model.
8 Tools
Tools allow AI models to request specific actions that extends beyond their built-in capabilities.
package example.micronaut.aiservice.tools;
import dev.langchain4j.agent.tool.Tool;
import jakarta.inject.Singleton;
import java.time.LocalDate;
@Singleton // (1)
public class LegalDocumentTools {
@Tool("Returns the last time the PRIVACY document was updated") // (2)
public LocalDate lastUpdatePrivacy() {
return LocalDate.of(2013, 3, 9); // (3)
}
}from dev.langchain4j.agent.tool import Tool
from jakarta.inject import Singleton
from java.time import LocalDate
@Singleton # (1)
class LegalDocumentTools:
@Tool("Returns the last time the PRIVACY document was updated") # (2)
def lastUpdatePrivacy(self) -> LocalDate:
return LocalDate.of(2013, 3, 9) # (3)package example.micronaut.aiservice.tools
import dev.langchain4j.agent.tool.Tool
import jakarta.inject.Singleton
import java.time.LocalDate
@Singleton // (1)
class LegalDocumentTools {
@Tool("Returns the last time the PRIVACY document was updated") // (2)
fun lastUpdatePrivacy(): LocalDate = LocalDate.of(2013, 3, 9) // (3)
}package example.micronaut.aiservice.tools
import dev.langchain4j.agent.tool.Tool
import groovy.transform.CompileStatic
import jakarta.inject.Singleton
import java.time.LocalDate
@Singleton // (1)
@CompileStatic
class LegalDocumentTools {
@Tool("Returns the last time the PRIVACY document was updated") // (2)
LocalDate lastUpdatePrivacy() {
return LocalDate.of(2013, 3, 9) // (3)
}
}| 1 | You should annotate the classes with @Tool methods with @Singleton. |
| 2 | The @Tool value specifies the description of the tool. The annotated method returns the date when a company PRIVACY document was last updated.
A model would have no way to know the last date when the PRIVACY document was updated. Hence, it is a good candidate for a tool. |
| 3 | The dates are harcoded for the purpose of this example, but they could have been retrieved from a database or external API. |
You can supply the tools to use to an @AiService:
package example.micronaut.aiservice.tools;
import io.micronaut.langchain4j.annotation.AiService;
@AiService(tools = LegalDocumentTools.class)
public interface CompanyBot {
String ask(String question);
}from abc import ABC, abstractmethod
from micronaut.langchain4j.annotation import AiService
from .LegalDocumentTools import LegalDocumentTools
@AiService(tools=[LegalDocumentTools])
class CompanyBot(ABC):
@abstractmethod
def ask(self, question: str) -> str:
...package example.micronaut.aiservice.tools
import io.micronaut.langchain4j.annotation.AiService
@AiService(tools = [LegalDocumentTools::class])
interface CompanyBot {
fun ask(question: String): String
}package example.micronaut.aiservice.tools
import io.micronaut.langchain4j.annotation.AiService
@AiService(tools = LegalDocumentTools.class)
interface CompanyBot {
String ask(String question)
}
The tool is registered under the method name, which the model sees, so the Python example keeps the Java name lastUpdatePrivacy.
|
The tools are not discovered by LangChain4j scanning the tool class: Micronaut builds the ToolSpecification of each @Tool method from its compile-time metadata (the method and parameter names, @Tool, @P, @ToolMemoryId) and invokes the method through its Micronaut ExecutableMethod. Tool classes therefore need no reflection, neither in a GraalVM native image nor for classes written in Python. The tools of a service are also available programmatically, as LangChain4j AiServiceTool instances, from tools.ToolRegistry#getAiServiceTools(java.util.Set).
LangChain4j’s @CompensateFor compensating actions are not supported, since LangChain4j only registers them for the tools it discovers reflectively. A warning is logged when a tool bean used by a service declares one, because the action never runs, even with compensateOnToolErrors.
|
Tool Providers
Tools can also be provided dynamically by a LangChain4j ToolProvider bean, for example the tool provider of the Micronaut MCP LangChain4j client, which provides the tools of the configured MCP servers.
package example.micronaut.aiservice.tools;
import dev.langchain4j.agent.tool.ToolSpecification;
import dev.langchain4j.service.tool.ToolProvider;
import dev.langchain4j.service.tool.ToolProviderRequest;
import dev.langchain4j.service.tool.ToolProviderResult;
import jakarta.inject.Named;
import jakarta.inject.Singleton;
@Singleton
@Named("weather") // (1)
public class WeatherToolProvider implements ToolProvider {
private final ToolSpecification forecast = ToolSpecification.builder()
.name("forecast")
.description("Returns the weather forecast")
.build();
@Override
public ToolProviderResult provideTools(ToolProviderRequest request) { // (2)
return ToolProviderResult.builder()
.add(forecast, (toolRequest, memoryId) -> "sunny")
.build();
}
}from dev.langchain4j.agent.tool import ToolSpecification
from dev.langchain4j.service.tool import ToolExecutor, ToolProvider, ToolProviderRequest, ToolProviderResult
from jakarta.inject import Named, Singleton
class ForecastExecutor(ToolExecutor):
def execute(self, tool_request, memory_id) -> str:
return "sunny"
@Singleton
@Named("weather") # (1)
class WeatherToolProvider(ToolProvider):
def provideTools(self, request: ToolProviderRequest) -> ToolProviderResult: # (2)
forecast = ToolSpecification.builder() \
.name("forecast") \
.description("Returns the weather forecast") \
.build()
return ToolProviderResult.builder() \
.add(forecast, ForecastExecutor()) \
.build()package example.micronaut.aiservice.tools
import dev.langchain4j.agent.tool.ToolSpecification
import dev.langchain4j.service.tool.ToolExecutor
import dev.langchain4j.service.tool.ToolProvider
import dev.langchain4j.service.tool.ToolProviderRequest
import dev.langchain4j.service.tool.ToolProviderResult
import jakarta.inject.Named
import jakarta.inject.Singleton
@Singleton
@Named("weather") // (1)
class WeatherToolProvider : ToolProvider {
private val forecast = ToolSpecification.builder()
.name("forecast")
.description("Returns the weather forecast")
.build()
override fun provideTools(request: ToolProviderRequest): ToolProviderResult = // (2)
ToolProviderResult.builder()
.add(forecast, ToolExecutor { _, _ -> "sunny" })
.build()
}package example.micronaut.aiservice.tools
import dev.langchain4j.agent.tool.ToolSpecification
import dev.langchain4j.service.tool.ToolExecutor
import dev.langchain4j.service.tool.ToolProvider
import dev.langchain4j.service.tool.ToolProviderRequest
import dev.langchain4j.service.tool.ToolProviderResult
import jakarta.inject.Named
import jakarta.inject.Singleton
@Singleton
@Named("weather") // (1)
class WeatherToolProvider implements ToolProvider {
private final ToolSpecification forecast = ToolSpecification.builder()
.name("forecast")
.description("Returns the weather forecast")
.build()
@Override
ToolProviderResult provideTools(ToolProviderRequest request) { // (2)
ToolProviderResult.builder()
.add(forecast, { toolRequest, memoryId -> "sunny" } as ToolExecutor)
.build()
}
}| 1 | The tool provider is a bean qualified with a name. |
| 2 | It returns the specifications of the tools and the executors that invoke them. |
Name the tool providers of an AI service with the toolProviders member of @AiService:
package example.micronaut.aiservice.tools;
import io.micronaut.langchain4j.annotation.AiService;
@AiService(toolProviders = "weather") // (1)
public interface WeatherAssistant {
String ask(String question);
}from abc import ABC, abstractmethod
from micronaut.langchain4j.annotation import AiService
@AiService(toolProviders=["weather"]) # (1)
class WeatherAssistant(ABC):
@abstractmethod
def ask(self, question: str) -> str:
...package example.micronaut.aiservice.tools
import io.micronaut.langchain4j.annotation.AiService
@AiService(toolProviders = ["weather"]) // (1)
interface WeatherAssistant {
fun ask(question: String): String
}package example.micronaut.aiservice.tools
import io.micronaut.langchain4j.annotation.AiService
@AiService(toolProviders = "weather") // (1)
interface WeatherAssistant {
String ask(String question)
}| 1 | The AI service uses the tools of the ToolProvider bean named weather. |
An AI service uses the following tool providers:
-
The
ToolProviderbeans named by thetoolProvidersmember of@AiService. Set the member to an empty array to use no tool provider. -
Otherwise, the
ToolProviderbean qualified with the name of the service, given by thenamedmember of@AiService. -
Otherwise, the default
ToolProviderbean: the onlyToolProviderbean without a qualifier (or annotated with@Primary). Tool providers qualified with a name are only used by the services that select them, and when there are several unqualified tool providers none of them is used implicitly, so that tools are never exposed to a service by accident.
The following LangChain4j beans are also applied to AI services, preferring a bean qualified with the name of the service over the default bean:
-
ToolExecutionErrorHandler- converts an exception thrown by a tool into the result sent to the model. -
ToolArgumentsErrorHandler- handles arguments the model sent that cannot be parsed. -
ToolSearchStrategy- selects the tools sent to the model when there are many tools.
Tool Calling Configuration
The tool calling loop of the AI services can be configured under langchain4j.ai-services. The default entry applies to every AI service, an entry with the name of a service replaces it for that service:
Example Configuration
langchain4j.ai-services.default.max-tool-calling-round-trips=10
langchain4j.ai-services.weather.execute-tools-concurrently=true
langchain4j.ai-services.weather.tool-executor=virtuallangchain4j:
ai-services:
default:
max-tool-calling-round-trips: 10
weather:
execute-tools-concurrently: true
tool-executor: virtual[langchain4j.ai-services.default]
max-tool-calling-round-trips = 10
[langchain4j.ai-services.weather]
execute-tools-concurrently = true
tool-executor = "virtual"langchain4j {
aiServices {
'default' {
maxToolCallingRoundTrips = 10
}
weather {
executeToolsConcurrently = true
toolExecutor = "virtual"
}
}
}{
langchain4j {
ai-services {
default {
max-tool-calling-round-trips = 10
}
weather {
execute-tools-concurrently = true
tool-executor = "virtual"
}
}
}
}{
"langchain4j": {
"ai-services": {
"default": {
"max-tool-calling-round-trips": 10
},
"weather": {
"execute-tools-concurrently": true,
"tool-executor": "virtual"
}
}
}
}| Property | Type | Description | Default value |
|---|---|---|---|
|
java.lang.Integer |
Sets the maximum number of tool calling round trips with the model per AI service call. Defaults to LangChain4j’s limit. |
|
|
boolean |
Sets whether the tools requested in a single model response are executed concurrently. Defaults to {@code false}. |
|
|
java.lang.String |
Sets the name of the Micronaut executor (for example {@code blocking} or {@code virtual}) that executes the tools when {@link #isExecuteToolsConcurrently()} is enabled. Defaults to LangChain4j’s executor. |
MCP Tools
The tools of Model Context Protocol (MCP) servers can be provided to AI services with the Micronaut MCP LangChain4j client:
implementation("io.micronaut.mcp:micronaut-mcp-client-langchain4j")<dependency>
<groupId>io.micronaut.mcp</groupId>
<artifactId>micronaut-mcp-client-langchain4j</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.mcp:micronaut-mcp-client-langchain4j",
]The client creates a LangChain4j McpClient bean for each configured MCP server, named after its configuration, and a ToolProvider bean that provides the tools of all of them.
Example Configuration
micronaut.mcp.client.http.github.url=https://example.com/mcpmicronaut:
mcp:
client:
http:
github:
url: https://example.com/mcp[micronaut.mcp.client.http.github]
url = "https://example.com/mcp"micronaut {
mcp {
client {
http {
github {
url = "https://example.com/mcp"
}
}
}
}
}{
micronaut {
mcp {
client {
http {
github {
url = "https://example.com/mcp"
}
}
}
}
}
}{
"micronaut": {
"mcp": {
"client": {
"http": {
"github": {
"url": "https://example.com/mcp"
}
}
}
}
}
}This ToolProvider is unqualified, so when it is the only unqualified one it is the default tool provider of the AI services that do not select tool providers (see the previous section): they receive the tools of every configured MCP server. To restrict a service to some MCP servers, name their clients with the mcpClients member of @AiService:
package example.micronaut.aiservice.mcp;
import io.micronaut.langchain4j.annotation.AiService;
@AiService(mcpClients = "github") // (1)
public interface GitHubAssistant {
String ask(String question);
}from abc import ABC, abstractmethod
from micronaut.langchain4j.annotation import AiService
@AiService(mcpClients=["github"]) # (1)
class GitHubAssistant(ABC):
@abstractmethod
def ask(self, question: str) -> str:
...package example.micronaut.aiservice.mcp
import io.micronaut.langchain4j.annotation.AiService
@AiService(mcpClients = ["github"]) // (1)
interface GitHubAssistant {
fun ask(question: String): String
}package example.micronaut.aiservice.mcp
import io.micronaut.langchain4j.annotation.AiService
@AiService(mcpClients = "github") // (1)
interface GitHubAssistant {
String ask(String question)
}| 1 | The AI service only receives the tools of the MCP server configured under micronaut.mcp.client.http.github. |
When mcpClients is set, no ToolProvider bean is applied implicitly. Combine it with toolProviders to add other tool providers. The mcpClients member requires dev.langchain4j:langchain4j-mcp on the classpath, which the Micronaut MCP LangChain4j client provides.
To expose the tools of a Micronaut application as an MCP server, see the Micronaut MCP server documentation.
9 Guardrails
Guardrails validate the input of an AI service before it is sent to the model, and the output of the model before it is returned. They are declared with LangChain4j’s @InputGuardrails and @OutputGuardrails annotations, on an AI service method or on the AI service to apply to all its methods.
The guardrail classes are resolved as Micronaut beans, so they can inject other beans:
package example.micronaut.aiservice.guardrails;
import dev.langchain4j.guardrail.InputGuardrail;
import dev.langchain4j.guardrail.InputGuardrailRequest;
import dev.langchain4j.guardrail.InputGuardrailResult;
import jakarta.inject.Singleton;
@Singleton // (1)
public class PromptInjectionGuard implements InputGuardrail {
@Override
public InputGuardrailResult validate(InputGuardrailRequest request) { // (2)
if (request.userMessage().singleText().toLowerCase().contains("ignore previous instructions")) {
return fatal("Possible prompt injection"); // (3)
}
return success();
}
}from dev.langchain4j.guardrail import InputGuardrail, InputGuardrailRequest, InputGuardrailResult
from jakarta.inject import Singleton
@Singleton # (1)
class PromptInjectionGuard(InputGuardrail):
def validate(self, request: InputGuardrailRequest) -> InputGuardrailResult: # (2)
if "ignore previous instructions" in request.userMessage().singleText().lower():
return self.fatal("Possible prompt injection") # (3)
return self.success()package example.micronaut.aiservice.guardrails
import dev.langchain4j.guardrail.InputGuardrail
import dev.langchain4j.guardrail.InputGuardrailRequest
import dev.langchain4j.guardrail.InputGuardrailResult
import jakarta.inject.Singleton
@Singleton // (1)
class PromptInjectionGuard : InputGuardrail {
override fun validate(request: InputGuardrailRequest): InputGuardrailResult { // (2)
if (request.userMessage().singleText().lowercase().contains("ignore previous instructions")) {
return fatal("Possible prompt injection") // (3)
}
return success()
}
}package example.micronaut.aiservice.guardrails
import dev.langchain4j.guardrail.InputGuardrail
import dev.langchain4j.guardrail.InputGuardrailRequest
import dev.langchain4j.guardrail.InputGuardrailResult
import jakarta.inject.Singleton
@Singleton // (1)
class PromptInjectionGuard implements InputGuardrail {
@Override
InputGuardrailResult validate(InputGuardrailRequest request) { // (2)
if (request.userMessage().singleText().toLowerCase().contains("ignore previous instructions")) {
return fatal("Possible prompt injection") // (3)
}
return success()
}
}| 1 | The guardrail is a bean. |
| 2 | An input guardrail validates the user message, and can read the rest of the request (the chat memory, the augmentation result…). |
| 3 | A fatal result stops the invocation: the AI service throws an InputGuardrailException. |
package example.micronaut.aiservice.guardrails;
import dev.langchain4j.service.guardrail.InputGuardrails;
import io.micronaut.langchain4j.annotation.AiService;
@AiService
public interface GuardedAssistant {
@InputGuardrails(PromptInjectionGuard.class) // (1)
String chat(String userMessage);
}from abc import ABC, abstractmethod
from dev.langchain4j.service.guardrail import InputGuardrails
from micronaut.langchain4j.annotation import AiService
from .PromptInjectionGuard import PromptInjectionGuard
@AiService
class GuardedAssistant(ABC):
@InputGuardrails([PromptInjectionGuard]) # (1)
@abstractmethod
def chat(self, user_message: str) -> str:
...package example.micronaut.aiservice.guardrails
import dev.langchain4j.service.guardrail.InputGuardrails
import io.micronaut.langchain4j.annotation.AiService
@AiService
interface GuardedAssistant {
@InputGuardrails(PromptInjectionGuard::class) // (1)
fun chat(userMessage: String): String
}package example.micronaut.aiservice.guardrails
import dev.langchain4j.service.guardrail.InputGuardrails
import io.micronaut.langchain4j.annotation.AiService
@AiService
interface GuardedAssistant {
@InputGuardrails(PromptInjectionGuard) // (1)
String chat(String userMessage)
}| 1 | The guardrail validates the input of this method. |
Output guardrails are declared the same way with @OutputGuardrails. They can ask the model to retry or reprompt it, up to the maxRetries of the annotation.
A guardrail class that is not a bean is instantiated with its public no-argument constructor, which requires reflection in a GraalVM native image. The guardrails provided by LangChain4j’s dev.langchain4j:langchain4j-guardrails module (for example the prompt injection and moderation guardrails) can be used this way, or registered as beans with a factory to configure them.
The guardrails also produce events, see AI Service Events.
10 Reflection and GraalVM Native Image
LangChain4j implements AI services and agents with java.lang.reflect.Proxy and reads their annotations (@SystemMessage, @UserMessage, @V, @Agent, @SequenceAgent, @Output, …), parameters and return types reflectively. Micronaut LangChain4j takes care of what this requires, so applications declare no reflection configuration:
-
Tools are not discovered reflectively. The
ToolSpecificationof each@Toolmethod is built from the compile-time metadata of the method, and the method is invoked through its MicronautExecutableMethod, so tool classes need no reflection at all. -
GraalVM Native Image:
micronaut-langchain4j-processorwrites the native image metadata (META-INF/native-image/io.micronaut.langchain4j/<type>/reachability-metadata.json) of every@AiService, every@AgenticServiceand every type declaring@Agent, workflow (@SequenceAgent,@ParallelAgent,@LoopAgent, …) or other declarative agent (@HumanInTheLoop,@A2AClientAgent,@McpClientAgent,@RegistryAgent) methods: the reflective access to their methods, and the proxy classes LangChain4j creates for them. -
Python:
@AiService,@AgenticServiceand the LangChain4j annotations imply the@AllowsReflectionhint (io.micronaut.core.annotation), so the Python compiler copies the annotations onto the Java types it generates. Neither@AllowsReflectionnor themicronaut.introspection.allow-reflectioncompiler option is needed. The hint for the LangChain4j annotations is added bymicronaut-langchain4j-processor, which must be on the compile classpath of the Python sources. -
Structured outputs and tool types: LangChain4j derives the JSON schema of the structured outputs of AI services and agents (the records or POJOs their methods return), and of the parameters and results of tools, from their fields, and maps them with Jackson, reflectively. The processor also writes the native image metadata of these types and of the types of their fields, so they need neither
@ReflectiveAccessnor hand-written metadata. Withmicronaut-langchain4j-serdethey are mapped with Micronaut Serialization instead of Jackson, see Micronaut Serialization.
The processor must be on the annotation processor path of the classes declaring the services and agents (annotationProcessor("io.micronaut.langchain4j:micronaut-langchain4j-processor") in Gradle) for the native image metadata to be generated.
|
Native Image Support by Module
Several LangChain4j providers map their REST APIs with Jackson, reflectively, and do not ship native image metadata. The Micronaut modules of these providers (Ollama, Mistral AI, Google AI Gemini, Hugging Face, Cohere, Jina and Voyage AI) ship the reachability metadata of their request and response types, so applications need no reflection configuration for them either. The test-suite-graal project verifies the following in a native image:
| Module | Status |
|---|---|
|
Tested: chat, streaming chat, embeddings |
|
Tested: chat |
|
Tested: chat, streaming chat, embeddings |
|
Tested: chat, streaming chat, embeddings, tool calling through an AI service |
|
Tested: chat, streaming chat, embeddings |
|
Tested: model creation. The AWS SDK provides its own native image metadata. |
|
Tested: scoring, embeddings |
|
Tested: scoring |
|
Tested: scoring, embeddings |
|
Reachability metadata provided, not tested |
|
Not tested. Known issue: the Azure SDK ( |
|
Not tested: they rely on the native image support of the Google Cloud and OCI SDKs |
|
Not supported: Jlama runs the model in the JVM with the Vector API |
In-memory embedding store (core) |
Tested |
Other embedding stores |
Not tested: they rely on the native image support of their client libraries |
11 Micronaut Serialization
LangChain4j maps the structured outputs of AI services and agents (the records or POJOs their methods return), the arguments and results of tools and the arguments of agents to and from JSON with Jackson, reflectively. With the following dependency, it maps them with Micronaut Serialization instead:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-serde")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-serde</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-serde",
]Micronaut Serialization maps a type with the introspection generated at compile time for it, without reflection, when the type is annotated with @Serdeable or imported with @SerdeImport. With micronaut-serde-processor and micronaut-langchain4j-processor on the annotation processor path, the public records returned by AI services and agents, and the public records used as parameters or results of tools, are imported automatically, together with the records they are made of.
annotationProcessor("io.micronaut.serde:micronaut-serde-processor")<annotationProcessorPaths>
<path>
<groupId>io.micronaut.serde</groupId>
<artifactId>micronaut-serde-processor</artifactId>
</path>
</annotationProcessorPaths>[tool.pyronaut.dependencies]
build = [
"io.micronaut.serde:micronaut-serde-processor",
]The other types are still mapped by Micronaut Serialization: with their introspection when they are @Introspected, and otherwise with an introspection that micronaut-reflection builds reflectively from their fields, of any visibility, and their accessors, as LangChain4j’s Jackson mapping does. Enable the DEBUG level of the io.micronaut.langchain4j.serde logger to see the types mapped reflectively. Jackson annotations (@JsonProperty, @JsonCreator, …) are only honoured on the types with a compile-time introspection: Micronaut Serialization maps them onto the introspection when it is generated.
The mapping follows the defaults of LangChain4j’s codec rather than the configuration of the application, whose JSON mapping is left unchanged: unknown properties are rejected (LangChain4j relies on it to find the JSON in the answers of the models), enums are read case insensitively, null values are written, and LocalDate, LocalTime and LocalDateTime are also read in the object shape of the JSON schema LangChain4j derives for them ({"year": 2001, "month": 2, "day": 3}).
| Micronaut Serialization replaces Jackson for these mappings only. Jackson remains a dependency of LangChain4j, which also uses it for the requests and responses of the model providers, the persistence of chat memories and agentic scopes, and to read the Jackson annotations of the types it derives JSON schemas from. |
12 Observability
Every chat model, streaming chat model and embedding model whose LangChain4j builder accepts listeners receives the ChatModelListener (or embedding model listener) beans of the application. Define a listener as a bean to observe the requests, responses and errors of all models:
package example.micronaut.observability;
import dev.langchain4j.model.chat.listener.ChatModelListener;
import dev.langchain4j.model.chat.listener.ChatModelResponseContext;
import dev.langchain4j.model.output.TokenUsage;
import jakarta.inject.Singleton;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Singleton // (1)
public class TokenUsageListener implements ChatModelListener {
private static final Logger LOG = LoggerFactory.getLogger(TokenUsageListener.class);
@Override
public void onResponse(ChatModelResponseContext responseContext) { // (2)
TokenUsage tokenUsage = responseContext.chatResponse().tokenUsage();
LOG.info("Tokens used: {}", tokenUsage);
}
}from dev.langchain4j.model.chat.listener import ChatModelListener, ChatModelResponseContext
from jakarta.inject import Singleton
from org.slf4j import LoggerFactory
LOG = LoggerFactory.getLogger("example.micronaut.observability.TokenUsageListener")
@Singleton # (1)
class TokenUsageListener(ChatModelListener):
def onResponse(self, response_context: ChatModelResponseContext) -> None: # (2)
token_usage = response_context.chatResponse().tokenUsage()
LOG.info("Tokens used: {}", token_usage)package example.micronaut.observability
import dev.langchain4j.model.chat.listener.ChatModelListener
import dev.langchain4j.model.chat.listener.ChatModelResponseContext
import jakarta.inject.Singleton
import org.slf4j.LoggerFactory
@Singleton // (1)
class TokenUsageListener : ChatModelListener {
override fun onResponse(responseContext: ChatModelResponseContext) { // (2)
val tokenUsage = responseContext.chatResponse().tokenUsage()
LOG.info("Tokens used: {}", tokenUsage)
}
companion object {
private val LOG = LoggerFactory.getLogger(TokenUsageListener::class.java)
}
}package example.micronaut.observability
import dev.langchain4j.model.chat.listener.ChatModelListener
import dev.langchain4j.model.chat.listener.ChatModelResponseContext
import dev.langchain4j.model.output.TokenUsage
import groovy.util.logging.Slf4j
import jakarta.inject.Singleton
@Slf4j
@Singleton // (1)
class TokenUsageListener implements ChatModelListener {
@Override
void onResponse(ChatModelResponseContext responseContext) { // (2)
TokenUsage tokenUsage = responseContext.chatResponse().tokenUsage()
log.info("Tokens used: {}", tokenUsage)
}
}| 1 | The listener is a bean, so it is injected into every model. |
| 2 | It is notified of every chat model response, here to log the token usage. |
Micrometer
To record the OpenTelemetry GenAI metrics of the chat models with Micronaut Micrometer, add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-micrometer")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-micrometer</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-micrometer",
]When there is a MeterRegistry bean, LangChain4j’s MicrometerMetricsChatModelListener is registered. It records:
-
gen_ai.client.token.usage- the input and output tokens, tagged with the provider, the operation, the request and response model names and the token type. -
gen_ai.client.operation.duration- the duration of the chat model requests.
When there is also an ObservationRegistry bean (for example with micronaut-micrometer-observation) and dev.langchain4j:langchain4j-observation is on the classpath, LangChain4j’s ObservationChatModelListener is registered instead. It records each chat model request as a Micrometer observation, which produces the same metrics and, with a tracing bridge, spans.
Set langchain4j.micrometer.enabled to false to disable these listeners.
AI Service Events
The AI services publish LangChain4j’s AI service events as Micronaut application events. Observe them with @EventListener methods or ApplicationEventListener beans:
package example.micronaut.aiservice.events;
import dev.langchain4j.observability.api.event.AiServiceCompletedEvent;
import dev.langchain4j.observability.api.event.ToolExecutedEvent;
import io.micronaut.runtime.event.annotation.EventListener;
import jakarta.inject.Singleton;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Singleton
public class AiServiceAuditor {
private static final Logger LOG = LoggerFactory.getLogger(AiServiceAuditor.class);
@EventListener // (1)
void onCompleted(AiServiceCompletedEvent event) {
LOG.info("AI service {} completed invocation {}",
event.invocationContext().interfaceName(),
event.invocationContext().invocationId()); // (2)
}
@EventListener
void onToolExecuted(ToolExecutedEvent event) { // (3)
LOG.info("Tool {} returned {}", event.request().name(), event.resultText());
}
}from dev.langchain4j.observability.api.event import AiServiceCompletedEvent, ToolExecutedEvent
from jakarta.inject import Singleton
from micronaut.runtime.event.annotation import EventListener
from org.slf4j import LoggerFactory
LOG = LoggerFactory.getLogger("example.micronaut.aiservice.events.AiServiceAuditor")
@Singleton
class AiServiceAuditor:
@EventListener # (1)
def onCompleted(self, event: AiServiceCompletedEvent) -> None:
context = event.invocationContext()
LOG.info("AI service {} completed invocation {}",
context.interfaceName(), context.invocationId()) # (2)
@EventListener
def onToolExecuted(self, event: ToolExecutedEvent) -> None: # (3)
LOG.info("Tool {} returned {}", event.request().name(), event.resultText())package example.micronaut.aiservice.events
import dev.langchain4j.observability.api.event.AiServiceCompletedEvent
import dev.langchain4j.observability.api.event.ToolExecutedEvent
import io.micronaut.runtime.event.annotation.EventListener
import jakarta.inject.Singleton
import org.slf4j.LoggerFactory
@Singleton
open class AiServiceAuditor {
@EventListener // (1)
open fun onCompleted(event: AiServiceCompletedEvent) {
LOG.info("AI service {} completed invocation {}",
event.invocationContext().interfaceName(),
event.invocationContext().invocationId()) // (2)
}
@EventListener
open fun onToolExecuted(event: ToolExecutedEvent) { // (3)
LOG.info("Tool {} returned {}", event.request().name(), event.resultText())
}
companion object {
private val LOG = LoggerFactory.getLogger(AiServiceAuditor::class.java)
}
}package example.micronaut.aiservice.events
import dev.langchain4j.observability.api.event.AiServiceCompletedEvent
import dev.langchain4j.observability.api.event.ToolExecutedEvent
import groovy.util.logging.Slf4j
import io.micronaut.runtime.event.annotation.EventListener
import jakarta.inject.Singleton
@Slf4j
@Singleton
class AiServiceAuditor {
@EventListener // (1)
void onCompleted(AiServiceCompletedEvent event) {
log.info("AI service {} completed invocation {}",
event.invocationContext().interfaceName(),
event.invocationContext().invocationId()) // (2)
}
@EventListener
void onToolExecuted(ToolExecutedEvent event) { // (3)
log.info("Tool {} returned {}", event.request().name(), event.resultText())
}
}| 1 | An event listener method receives the events of every AI service. |
| 2 | The invocation context identifies the AI service, the method and the invocation, which correlates the events of one invocation. |
| 3 | Each tool execution produces an event with the request and the result of the tool. |
The following events are published: AiServiceStartedEvent, AiServiceRequestIssuedEvent, AiServiceResponseReceivedEvent, AiServiceCompletedEvent, AiServiceErrorEvent, InputGuardrailExecutedEvent, OutputGuardrailExecutedEvent, ToolExecutedEvent and ToolCompensatedEvent (all in the dev.langchain4j.observability.api.event package). A listener of a super type, such as AiServiceEvent, receives all of them. An event type is only published when there is a listener for it.
Beans implementing LangChain4j’s AiServiceListener (for example AiServiceCompletedListener) are also registered with every AI service.
13 HTTP Client
The OpenAI, Anthropic, Ollama, Mistral AI, Google AI Gemini, Hugging Face, Cohere, Jina and Voyage AI models send their requests with the Micronaut HTTP client instead of the HTTP client of LangChain4j. These modules exclude LangChain4j’s JDK HTTP client, so add a Micronaut HTTP client implementation to your application:
runtimeOnly("io.micronaut:micronaut-http-client")<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-http-client</artifactId>
<scope>runtime</scope>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut:micronaut-http-client",
]Because the requests go through the Micronaut HTTP client:
-
The
micronaut.http.client.*configuration applies to them, for example the proxy, the SSL configuration, the connection pool and the event loop. -
The
timeoutof a model sets the read timeout of its requests. -
HTTP client filters apply to them, for example to add headers or to record requests:
package example.micronaut.http;
import io.micronaut.http.MutableHttpRequest;
import io.micronaut.http.annotation.ClientFilter;
import io.micronaut.http.annotation.RequestFilter;
@ClientFilter("/v1/**") // (1)
public class RequestSourceFilter {
@RequestFilter
public void addSource(MutableHttpRequest<?> request) {
request.header("X-Request-Source", "my-service"); // (2)
}
}from micronaut.http import MutableHttpRequest
from micronaut.http.annotation import ClientFilter, RequestFilter
@ClientFilter("/v1/**") # (1)
class RequestSourceFilter:
@RequestFilter
def addSource(self, request: MutableHttpRequest) -> None:
request.header("X-Request-Source", "my-service") # (2)package example.micronaut.http
import io.micronaut.http.MutableHttpRequest
import io.micronaut.http.annotation.ClientFilter
import io.micronaut.http.annotation.RequestFilter
@ClientFilter("/v1/**") // (1)
class RequestSourceFilter {
@RequestFilter
fun addSource(request: MutableHttpRequest<*>) {
request.header("X-Request-Source", "my-service") // (2)
}
}package example.micronaut.http
import io.micronaut.http.MutableHttpRequest
import io.micronaut.http.annotation.ClientFilter
import io.micronaut.http.annotation.RequestFilter
@ClientFilter("/v1/**") // (1)
class RequestSourceFilter {
@RequestFilter
void addSource(MutableHttpRequest<?> request) {
request.header("X-Request-Source", "my-service") // (2)
}
}| 1 | The filter matches the request paths of the provider, here the /v1/ paths of the OpenAI API. |
| 2 | A header is added to every request.
|
Model Credentials
The credentials of a model are usually its configured API key. To send other credentials with each request, for example a short-lived token or the token of the current user, declare a ModelAuthProvider bean. The providers are asked in order for each request of the models that use the Micronaut HTTP client: the first value that is not null replaces the Authorization header.
package example.micronaut.http;
import io.micronaut.context.annotation.Requires;
import io.micronaut.context.annotation.Value;
import io.micronaut.langchain4j.http.client.ModelAuthProvider;
import io.micronaut.langchain4j.http.client.ModelAuthRequest;
import jakarta.inject.Singleton;
import org.jspecify.annotations.Nullable;
@Singleton
@Requires(property = "ai-gateway.host") // (1)
public class GatewayAuthProvider implements ModelAuthProvider {
private final String gatewayHost;
public GatewayAuthProvider(@Value("${ai-gateway.host}") String gatewayHost) {
this.gatewayHost = gatewayHost;
}
@Override
public @Nullable String authorization(ModelAuthRequest request) {
if (!gatewayHost.equals(request.uri().getHost())) {
return null; // (2)
}
return "Bearer " + currentToken(); // (3)
}
private String currentToken() {
// for example a short-lived token of a secret manager, refreshed when it expires
return System.getenv().getOrDefault("AI_GATEWAY_TOKEN", "");
}
}import os
from typing import Annotated
from micronaut.langchain4j.http.client import ModelAuthProvider, ModelAuthRequest
from jakarta.inject import Singleton
from micronaut.context.annotation import Requires, Value
@Singleton
@Requires(property="ai-gateway.host") # (1)
class GatewayAuthProvider(ModelAuthProvider):
def __init__(self, gateway_host: Annotated[str, Value("${ai-gateway.host}")]):
self.gateway_host = gateway_host
def authorization(self, request: ModelAuthRequest) -> str | None:
if self.gateway_host != request.uri().getHost():
return None # (2)
return "Bearer " + self._current_token() # (3)
def _current_token(self) -> str:
# for example a short-lived token of a secret manager, refreshed when it expires
return os.environ.get("AI_GATEWAY_TOKEN", "")package example.micronaut.http
import io.micronaut.context.annotation.Requires
import io.micronaut.context.annotation.Value
import io.micronaut.langchain4j.http.client.ModelAuthProvider
import io.micronaut.langchain4j.http.client.ModelAuthRequest
import jakarta.inject.Singleton
@Singleton
@Requires(property = "ai-gateway.host") // (1)
class GatewayAuthProvider(
@Value("\${ai-gateway.host}") private val gatewayHost: String
) : ModelAuthProvider {
override fun authorization(request: ModelAuthRequest): String? {
if (gatewayHost != request.uri().host) {
return null // (2)
}
return "Bearer ${currentToken()}" // (3)
}
private fun currentToken(): String =
// for example a short-lived token of a secret manager, refreshed when it expires
System.getenv().getOrDefault("AI_GATEWAY_TOKEN", "")
}package example.micronaut.http
import io.micronaut.context.annotation.Requires
import io.micronaut.context.annotation.Value
import io.micronaut.langchain4j.http.client.ModelAuthProvider
import io.micronaut.langchain4j.http.client.ModelAuthRequest
import jakarta.inject.Singleton
@Singleton
@Requires(property = "ai-gateway.host") // (1)
class GatewayAuthProvider implements ModelAuthProvider {
private final String gatewayHost
GatewayAuthProvider(@Value('${ai-gateway.host}') String gatewayHost) {
this.gatewayHost = gatewayHost
}
@Override
String authorization(ModelAuthRequest request) {
if (gatewayHost != request.uri().host) {
return null // (2)
}
"Bearer ${currentToken()}" // (3)
}
private static String currentToken() {
// for example a short-lived token of a secret manager, refreshed when it expires
System.getenv().getOrDefault("AI_GATEWAY_TOKEN", "")
}
}| 1 | The provider is only enabled when the gateway is configured. |
| 2 | Returning null keeps the credentials of the model configuration, here for the models that are not served by the gateway. |
| 3 | The value of the Authorization header. |
The provider is called on the thread that sends the request, also for the streaming and asynchronous requests, so it can read the context of the current HTTP request.
Token Propagation
When the application is secured with Micronaut Security, the bearer token of the user that calls the application can be sent to the models in place of their API key, for example to an OpenAI-compatible gateway that authorizes each user:
langchain4j.token-propagation.enabled=true
langchain4j.token-propagation.uri-regex=https://ai-gateway[.]example[.]com/.*langchain4j:
token-propagation:
enabled: true
uri-regex: https://ai-gateway[.]example[.]com/.*[langchain4j.token-propagation]
enabled = true
uri-regex = "https://ai-gateway[.]example[.]com/.*"langchain4j {
tokenPropagation {
enabled = true
uriRegex = "https://ai-gateway[.]example[.]com/.*"
}
}{
langchain4j {
token-propagation {
enabled = true
uri-regex = "https://ai-gateway[.]example[.]com/.*"
}
}
}{
"langchain4j": {
"token-propagation": {
"enabled": true,
"uri-regex": "https://ai-gateway[.]example[.]com/.*"
}
}
}uri-regex limits the propagation to the models whose URLs match. The token is the one that Micronaut Security authenticated for the current request.
The token propagation of Micronaut Security itself (micronaut.security.token.propagation) does not apply to the models: it never replaces the Authorization header that a request already has, and the models send their API key.
|
Streaming and asynchronous requests
The streaming responses (server-sent events) and the non-blocking requests of the asynchronous AI services are also handled by the Micronaut HTTP client.
The other providers (Azure OpenAI, Bedrock, Vertex AI, OCI GenAI and Google Gen AI) use the HTTP client of their cloud SDK.
14 Image Models
Image models generate images from a text prompt. The OpenAI, Azure OpenAI, Vertex AI and Google Gen AI modules configure a dev.langchain4j.model.image.ImageModel bean, under langchain4j.<provider>.image-model (and named ones under langchain4j.<provider>.image-models.<name>):
Example Configuration
langchain4j.open-ai.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.image-model.model-name=gpt-image-1
langchain4j.open-ai.image-model.size=1024x1024langchain4j:
open-ai:
api-key: ${OPENAI_API_KEY}
image-model:
model-name: gpt-image-1
size: 1024x1024[langchain4j.open-ai]
api-key = "${OPENAI_API_KEY}"
[langchain4j.open-ai.image-model]
model-name = "gpt-image-1"
size = "1024x1024"langchain4j {
openAi {
apiKey = "${OPENAI_API_KEY}"
imageModel {
modelName = "gpt-image-1"
size = "1024x1024"
}
}
}{
langchain4j {
open-ai {
api-key = "${OPENAI_API_KEY}"
image-model {
model-name = "gpt-image-1"
size = "1024x1024"
}
}
}
}{
"langchain4j": {
"open-ai": {
"api-key": "${OPENAI_API_KEY}",
"image-model": {
"model-name": "gpt-image-1",
"size": "1024x1024"
}
}
}
}Inject the image model where images are generated:
package example.micronaut.image;
import dev.langchain4j.data.image.Image;
import dev.langchain4j.model.image.ImageModel;
import jakarta.inject.Singleton;
@Singleton
public class ImageGenerator {
private final ImageModel imageModel;
public ImageGenerator(ImageModel imageModel) { // (1)
this.imageModel = imageModel;
}
public Image generate(String prompt) {
return imageModel.generate(prompt).content(); // (2)
}
}from dev.langchain4j.data.image import Image
from dev.langchain4j.model.image import ImageModel
from jakarta.inject import Singleton
@Singleton
class ImageGenerator:
def __init__(self, image_model: ImageModel): # (1)
self.image_model = image_model
def generate(self, prompt: str) -> Image:
return self.image_model.generate(prompt).content() # (2)package example.micronaut.image
import dev.langchain4j.data.image.Image
import dev.langchain4j.model.image.ImageModel
import jakarta.inject.Singleton
@Singleton
class ImageGenerator(private val imageModel: ImageModel) { // (1)
fun generate(prompt: String): Image =
imageModel.generate(prompt).content() // (2)
}package example.micronaut.image
import dev.langchain4j.data.image.Image
import dev.langchain4j.model.image.ImageModel
import jakarta.inject.Singleton
@Singleton
class ImageGenerator {
private final ImageModel imageModel
ImageGenerator(ImageModel imageModel) { // (1)
this.imageModel = imageModel
}
Image generate(String prompt) {
imageModel.generate(prompt).content() // (2)
}
}| 1 | The ImageModel bean is injected. |
| 2 | The model generates an image. Depending on the model, the Image holds a URL (url()) or base64 data (base64Data() with its mimeType()): gpt-image-1, for example, only returns base64 data. |
15 Test Resources
Micronaut Test Resources starts the services the tests need in containers. Micronaut LangChain4j provides test resources for Ollama and Qdrant, and the embedding and chat memory stores backed by a database (PGVector, Oracle, Redis, Neo4j, Cassandra, MongoDB, JDBC) can use the database test resources of Micronaut.
Add the test resources of Micronaut LangChain4j to the test resources service classpath of your Gradle build or Maven build.
Ollama
testResourcesService("io.micronaut.langchain4j:micronaut-langchain4j-ollama-testresource")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-ollama-testresource</artifactId>
<scope>testResourcesService</scope>
</dependency>[tool.pyronaut.dependencies]
test = [
"io.micronaut.langchain4j:micronaut-langchain4j-ollama-testresource",
]When langchain4j.ollama.base-url is not configured, an Ollama container with the configured model is started, and the base URL is resolved from it. The image is langchain4j/ollama-<model-name>, which bundles the model:
Example Configuration
langchain4j.ollama.model-name=orca-minilangchain4j.ollama.model-name: orca-mini"langchain4j.ollama.model-name" = "orca-mini"langchain4j.ollama.modelName = "orca-mini"{
"langchain4j.ollama.model-name" = "orca-mini"
}{
"langchain4j.ollama.model-name": "orca-mini"
}Qdrant
testResourcesService("io.micronaut.langchain4j:micronaut-langchain4j-qdrant-testresource")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-qdrant-testresource</artifactId>
<scope>testResourcesService</scope>
</dependency>[tool.pyronaut.dependencies]
test = [
"io.micronaut.langchain4j:micronaut-langchain4j-qdrant-testresource",
]When the host and port of the Qdrant embedding store are not configured, a Qdrant container is started with a collection of 384 dimensions (cosine distance) named after collection-name, and langchain4j.qdrant.embedding-store.host and port are resolved from it:
Example Configuration
langchain4j.qdrant.embedding-store.collection-name=testlangchain4j.qdrant.embedding-store.collection-name: test"langchain4j.qdrant.embedding-store.collection-name" = "test"langchain4j.qdrant.embeddingStore.collectionName = "test"{
"langchain4j.qdrant.embedding-store.collection-name" = "test"
}{
"langchain4j.qdrant.embedding-store.collection-name": "test"
}16 Experimental Features
The features of this section integrate experimental LangChain4j APIs: they are annotated with @Experimental and can change in a minor release, following LangChain4j.
Decision Services
A decision service answers questions with a LangChain4j DecisionModel, a model that returns the probability of each possible answer instead of generated text: yes/no questions, a choice between options, or a position on a scale. Annotate an interface with @DecisionService and its methods with LangChain4j’s @Decide:
package example.micronaut.decision;
import dev.langchain4j.service.V;
import dev.langchain4j.service.decision.Choice;
import dev.langchain4j.service.decision.Decide;
import io.micronaut.langchain4j.annotation.DecisionService;
@DecisionService // (1)
public interface TicketTriage {
@Decide("Is this message spam?")
boolean isSpam(@V("message") String message); // (2)
@Decide("Which team should handle this ticket?")
Team route(@V("ticket") String ticket); // (3)
@Decide("Which team should handle this ticket?")
Choice<Team> routeWithProbabilities(@V("ticket") String ticket); // (4)
}from abc import ABC, abstractmethod
from typing import Annotated
from dev.langchain4j.service import V
from dev.langchain4j.service.decision import Choice, Decide
from micronaut.langchain4j.annotation import DecisionService
from .Team import Team
@DecisionService # (1)
class TicketTriage(ABC):
@Decide("Is this message spam?")
@abstractmethod
def is_spam(self, message: Annotated[str, V("message")]) -> bool: # (2)
...
@Decide("Which team should handle this ticket?")
@abstractmethod
def route(self, ticket: Annotated[str, V("ticket")]) -> Team: # (3)
...
@Decide("Which team should handle this ticket?")
@abstractmethod
def route_with_probabilities(self, ticket: Annotated[str, V("ticket")]) -> Choice[Team]: # (4)
...package example.micronaut.decision
import dev.langchain4j.service.V
import dev.langchain4j.service.decision.Choice
import dev.langchain4j.service.decision.Decide
import io.micronaut.langchain4j.annotation.DecisionService
@DecisionService // (1)
interface TicketTriage {
@Decide("Is this message spam?")
fun isSpam(@V("message") message: String): Boolean // (2)
@Decide("Which team should handle this ticket?")
fun route(@V("ticket") ticket: String): Team // (3)
@Decide("Which team should handle this ticket?")
fun routeWithProbabilities(@V("ticket") ticket: String): Choice<Team> // (4)
}package example.micronaut.decision
import dev.langchain4j.service.V
import dev.langchain4j.service.decision.Choice
import dev.langchain4j.service.decision.Decide
import io.micronaut.langchain4j.annotation.DecisionService
@DecisionService // (1)
interface TicketTriage {
@Decide("Is this message spam?")
boolean isSpam(@V("message") String message) // (2)
@Decide("Which team should handle this ticket?")
Team route(@V("ticket") String ticket) // (3)
@Decide("Which team should handle this ticket?")
Choice<Team> routeWithProbabilities(@V("ticket") String ticket) // (4)
}| 1 | The interface is implemented with LangChain4j’s DecisionServices. It uses the DecisionModel bean named after the service (@DecisionService("support")), otherwise the default DecisionModel bean, and the ThresholdProvider bean selected the same way. |
| 2 | A boolean method asks a yes/no question: the answer is true when the probability of "yes" reaches the threshold. |
| 3 | An enum method chooses one of the constants. |
| 4 | Choice, Scale, YesNoAnswer and DecisionResult also return the probabilities. A record whose fields are annotated with @Decide asks several questions in a single call. |
The parameters are sent to the model as the input, by name, so name them with @V unless the code is compiled with -parameters.
The constants of the enum can be described with LangChain4j’s @Description:
package example.micronaut.decision;
import dev.langchain4j.model.output.structured.Description;
public enum Team {
@Description("Payments, invoices and refunds") // (1)
BILLING,
@Description("Problems using the product, crashes and errors")
SUPPORT
}from enum import Enum
class Team(Enum):
BILLING = "BILLING" # (1)
SUPPORT = "SUPPORT"package example.micronaut.decision
import dev.langchain4j.model.output.structured.Description
enum class Team {
@Description("Payments, invoices and refunds") // (1)
BILLING,
@Description("Problems using the product, crashes and errors")
SUPPORT
}package example.micronaut.decision
import dev.langchain4j.model.output.structured.Description
enum Team {
@Description("Payments, invoices and refunds") // (1)
BILLING,
@Description("Problems using the product, crashes and errors")
SUPPORT
}| 1 | The description helps the model choose. |
A Python Enum compiles to a Java enum whose constants have no @Description: give them self-explanatory names.
|
A test can provide its own DecisionModel bean:
package example.micronaut.decision;
import dev.langchain4j.model.decision.DecisionModel;
import dev.langchain4j.model.decision.request.ChoiceQuestion;
import dev.langchain4j.model.decision.request.DecisionRequest;
import dev.langchain4j.model.decision.response.ChoiceAnswer;
import dev.langchain4j.model.decision.response.DecisionResponse;
import dev.langchain4j.model.decision.response.YesNoAnswer;
import io.micronaut.context.annotation.Property;
import io.micronaut.context.annotation.Requires;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Singleton;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
@Property(name = "spec.name", value = "TicketTriageTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false)
class TicketTriageTest {
@Test
void triagesTickets(TicketTriage triage) {
assertFalse(triage.isSpam("I was charged twice"));
assertEquals(Team.BILLING, triage.route("I was charged twice"));
assertEquals(0.9, triage.routeWithProbabilities("I was charged twice").probabilityOf(Team.BILLING));
}
/**
* Answers no to the yes/no questions and chooses the first option of the other questions.
*/
@Singleton
@Requires(property = "spec.name", value = "TicketTriageTest")
static class FirstOptionDecisionModel implements DecisionModel {
@Override
public DecisionResponse doDecide(DecisionRequest request) {
DecisionResponse.Builder response = DecisionResponse.builder();
request.questions().forEach((name, question) -> {
if (question instanceof ChoiceQuestion choice) {
String first = choice.options().keySet().iterator().next();
ChoiceAnswer.Builder answer = ChoiceAnswer.builder().value(first).options(choice.options().keySet());
choice.options().keySet().forEach(option -> answer.probability(option, option.equals(first) ? 0.9 : 0.1));
response.answer(name, answer.build());
} else {
response.answer(name, YesNoAnswer.of(0.1));
}
});
return response.build();
}
}
}from typing import Annotated
from dev.langchain4j.model.decision import DecisionModel
from dev.langchain4j.model.decision.request import ChoiceQuestion, DecisionRequest
from dev.langchain4j.model.decision.response import ChoiceAnswer, DecisionResponse, YesNoAnswer
from jakarta.inject import Inject, Singleton
from micronaut.context.annotation import Property, Requires
from micronaut.test.extensions.junit5.annotation import MicronautTest
from org.junit.jupiter.api import Test
from example.micronaut.decision.Team import Team
from example.micronaut.decision.TicketTriage import TicketTriage
@Property(name="spec.name", value="TicketTriageTest")
@Property(name="langchain4j.ollama.enabled", value="false")
@MicronautTest(startApplication=False)
class TicketTriageTest:
triage: Annotated[TicketTriage, Inject]
@Test
def triages_tickets(self):
assert not self.triage.is_spam("I was charged twice")
assert self.triage.route("I was charged twice") == Team.BILLING
assert self.triage.route_with_probabilities("I was charged twice").probabilityOf(Team.BILLING) == 0.9
# Answers no to the yes/no questions and chooses the first option of the other questions.
@Singleton
@Requires(property="spec.name", value="TicketTriageTest")
class FirstOptionDecisionModel(DecisionModel):
def doDecide(self, request: DecisionRequest) -> DecisionResponse:
response = DecisionResponse.builder()
for name in request.questions().keySet():
question = request.questions().get(name)
if isinstance(question, ChoiceQuestion):
options = list(question.options().keySet())
answer = ChoiceAnswer.builder().value(options[0]).options(question.options().keySet())
for option in options:
answer.probability(option, 0.9 if option == options[0] else 0.1)
response.answer(name, answer.build())
else:
response.answer(name, YesNoAnswer.of(0.1))
return response.build()package example.micronaut.decision
import dev.langchain4j.model.decision.DecisionModel
import dev.langchain4j.model.decision.request.ChoiceQuestion
import dev.langchain4j.model.decision.request.DecisionRequest
import dev.langchain4j.model.decision.response.ChoiceAnswer
import dev.langchain4j.model.decision.response.DecisionResponse
import dev.langchain4j.model.decision.response.YesNoAnswer
import io.micronaut.context.annotation.Property
import io.micronaut.context.annotation.Requires
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import jakarta.inject.Singleton
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Assertions.assertFalse
import org.junit.jupiter.api.Test
@Property(name = "spec.name", value = "TicketTriageTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false)
internal class TicketTriageTest {
@Test
fun triagesTickets(triage: TicketTriage) {
assertFalse(triage.isSpam("I was charged twice"))
assertEquals(Team.BILLING, triage.route("I was charged twice"))
assertEquals(0.9, triage.routeWithProbabilities("I was charged twice").probabilityOf(Team.BILLING))
}
/**
* Answers no to the yes/no questions and chooses the first option of the other questions.
*/
@Singleton
@Requires(property = "spec.name", value = "TicketTriageTest")
class FirstOptionDecisionModel : DecisionModel {
override fun doDecide(request: DecisionRequest): DecisionResponse {
val response = DecisionResponse.builder()
request.questions().forEach { (name, question) ->
if (question is ChoiceQuestion) {
val first = question.options().keys.first()
val answer = ChoiceAnswer.builder().value(first).options(question.options().keys)
question.options().keys.forEach { answer.probability(it, if (it == first) 0.9 else 0.1) }
response.answer(name, answer.build())
} else {
response.answer(name, YesNoAnswer.of(0.1))
}
}
return response.build()
}
}
}package example.micronaut.decision
import dev.langchain4j.model.decision.DecisionModel
import dev.langchain4j.model.decision.request.ChoiceQuestion
import dev.langchain4j.model.decision.request.DecisionRequest
import dev.langchain4j.model.decision.response.ChoiceAnswer
import dev.langchain4j.model.decision.response.DecisionResponse
import dev.langchain4j.model.decision.response.YesNoAnswer
import io.micronaut.context.annotation.Property
import io.micronaut.context.annotation.Requires
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
import jakarta.inject.Singleton
import org.junit.jupiter.api.Test
import static org.junit.jupiter.api.Assertions.assertEquals
import static org.junit.jupiter.api.Assertions.assertFalse
@Property(name = "spec.name", value = "TicketTriageTest")
@Property(name = "langchain4j.ollama.enabled", value = "false")
@MicronautTest(startApplication = false)
class TicketTriageTest {
@Test
void triagesTickets(TicketTriage triage) {
assertFalse(triage.isSpam("I was charged twice"))
assertEquals(Team.BILLING, triage.route("I was charged twice"))
assertEquals(0.9d, triage.routeWithProbabilities("I was charged twice").probabilityOf(Team.BILLING))
}
/**
* Answers no to the yes/no questions and chooses the first option of the other questions.
*/
@Singleton
@Requires(property = "spec.name", value = "TicketTriageTest")
static class FirstOptionDecisionModel implements DecisionModel {
@Override
DecisionResponse doDecide(DecisionRequest request) {
DecisionResponse.Builder response = DecisionResponse.builder()
request.questions().each { name, question ->
if (question instanceof ChoiceQuestion) {
String first = question.options().keySet().first()
ChoiceAnswer.Builder answer = ChoiceAnswer.builder().value(first).options(question.options().keySet())
question.options().keySet().each { answer.probability(it, it == first ? 0.9d : 0.1d) }
response.answer(name, answer.build())
} else {
response.answer(name, YesNoAnswer.of(0.1d))
}
}
response.build()
}
}
}Routing Chat Models
A routing chat model is a ChatModel that sends each request to one of several chat models, for example a small model for simple requests and a large one for the others. Configure it under langchain4j.routing-chat-models.<name>: the model is a ChatModel bean of that name, which an @AiService with the same name uses.
langchain4j.routing-chat-models.assistant.routes.fast=small
langchain4j.routing-chat-models.assistant.routes.expert=large
langchain4j.routing-chat-models.assistant.descriptions.fast=Greetings, small talk and simple questions
langchain4j.routing-chat-models.assistant.descriptions.expert=Mathematics, reasoning and code
langchain4j.routing-chat-models.assistant.default-route=fastlangchain4j:
routing-chat-models:
assistant:
routes:
fast: small
expert: large
descriptions:
fast: Greetings, small talk and simple questions
expert: Mathematics, reasoning and code
default-route: fast[langchain4j.routing-chat-models.assistant]
default-route = "fast"
[langchain4j.routing-chat-models.assistant.routes]
fast = "small"
expert = "large"
[langchain4j.routing-chat-models.assistant.descriptions]
fast = "Greetings, small talk and simple questions"
expert = "Mathematics, reasoning and code"langchain4j {
routingChatModels {
assistant {
routes {
fast = "small"
expert = "large"
}
descriptions {
fast = "Greetings, small talk and simple questions"
expert = "Mathematics, reasoning and code"
}
defaultRoute = "fast"
}
}
}{
langchain4j {
routing-chat-models {
assistant {
routes {
fast = "small"
expert = "large"
}
descriptions {
fast = "Greetings, small talk and simple questions"
expert = "Mathematics, reasoning and code"
}
default-route = "fast"
}
}
}
}{
"langchain4j": {
"routing-chat-models": {
"assistant": {
"routes": {
"fast": "small",
"expert": "large"
},
"descriptions": {
"fast": "Greetings, small talk and simple questions",
"expert": "Mathematics, reasoning and code"
},
"default-route": "fast"
}
}
}
}routes maps each route to the name of a ChatModel bean, and descriptions describes the requests of each route. By default, the route is chosen by asking a DecisionModel which description matches the request: the DecisionModel bean named by decision-model, otherwise the default one. question, min-probability and fallback-strategy (DEFAULT_ROUTE or FAIL) tune that choice. Set router to the name of a ChatModelRouter bean to choose the route in another way.
Agent Skills
Agent Skills are folders of instructions and resources that a model loads when a request needs them, instead of keeping them all in its context. Add the LangChain4j Skills dependency:
implementation("dev.langchain4j:langchain4j-skills")<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-skills</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"dev.langchain4j:langchain4j-skills",
]Each skill is a directory with a SKILL.md file, whose front matter names and describes it, for example src/main/resources/skills/packing/SKILL.md:
---
name: packing
description: Prepares a packing list for a trip
---
List the clothes for the weather of the destination, then the travel documents.Configure a set of skills, loaded from a classpath directory and/or a file system directory, under langchain4j.skills.<name>:
langchain4j.skills.travel.classpath=skillslangchain4j:
skills:
travel:
classpath: skills[langchain4j.skills.travel]
classpath = "skills"langchain4j {
skills {
travel {
classpath = "skills"
}
}
}{
langchain4j {
skills {
travel {
classpath = "skills"
}
}
}
}{
"langchain4j": {
"skills": {
"travel": {
"classpath": "skills"
}
}
}
}Then select it with the skills member of @AiService:
package example.micronaut.aiservice.skills;
import dev.langchain4j.service.SystemMessage;
import io.micronaut.langchain4j.annotation.AiService;
@AiService(skills = "travel") // (1)
public interface TravelAgent {
@SystemMessage("You are a travel agent") // (2)
String chat(String message);
}from abc import ABC, abstractmethod
from dev.langchain4j.service import SystemMessage
from micronaut.langchain4j.annotation import AiService
@AiService(skills="travel") # (1)
class TravelAgent(ABC):
@SystemMessage("You are a travel agent") # (2)
@abstractmethod
def chat(self, message: str) -> str:
...package example.micronaut.aiservice.skills
import dev.langchain4j.service.SystemMessage
import io.micronaut.langchain4j.annotation.AiService
@AiService(skills = "travel") // (1)
interface TravelAgent {
@SystemMessage("You are a travel agent") // (2)
fun chat(message: String): String
}package example.micronaut.aiservice.skills
import dev.langchain4j.service.SystemMessage
import io.micronaut.langchain4j.annotation.AiService
@AiService(skills = "travel") // (1)
interface TravelAgent {
@SystemMessage("You are a travel agent") // (2)
String chat(String message)
}| 1 | The service receives the activate_skill tool, which returns the instructions of a skill to the model. |
| 2 | The available skills are listed after the system message. |
17 Chat Language Models
The following modules provide integration with Langchain4j Language Models.
Each module configures one or more ChatModel beans, making them available for dependency injection based on configuration.
17.1 ChatModel Example
This example, asks a chat model to generate the list of the top 3 albums of a Jazz musician.
package example.micronaut;
import dev.langchain4j.data.message.ChatMessage;
import dev.langchain4j.data.message.SystemMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.response.ChatResponse;
import jakarta.inject.Singleton;
import java.util.List;
@Singleton
public class MusicianAssistant {
private static final SystemMessage SYSTEM_MSG = SystemMessage.from("""
You are an expert in Jazz music.
Reply with only the names of the artists, albums, etc.
Be very concise.
If a list is given, separate the items with commas.""");
private final ChatModel model;
public MusicianAssistant(ChatModel model) { // (1)
this.model = model;
}
public Musician generateTopThreeAlbums(String name) {
List<ChatMessage> messages = generateTopThreeAlbumsMessages(name);
ChatResponse albums = model.chat(messages);
String topThreeAlbums = albums.aiMessage().text();
return new Musician(name, topThreeAlbums);
}
private static List<ChatMessage> generateTopThreeAlbumsMessages(String name) {
return List.of(SYSTEM_MSG, UserMessage.from(
String.format("Only list the top 3 albums of %s", name)
));
}
}from dev.langchain4j.data.message import SystemMessage, UserMessage
from dev.langchain4j.model.chat import ChatModel
from jakarta.inject import Singleton
from java.util import List
from .Musician import Musician
SYSTEM_MSG = SystemMessage.from_("""\
You are an expert in Jazz music.
Reply with only the names of the artists, albums, etc.
Be very concise.
If a list is given, separate the items with commas.""")
@Singleton
class MusicianAssistant:
def __init__(self, model: ChatModel): # (1)
self.model = model
def generate_top_three_albums(self, name: str) -> Musician:
messages = generate_top_three_albums_messages(name)
albums = self.model.chat(messages)
top_three_albums = albums.aiMessage().text()
return Musician(name, top_three_albums)
def generate_top_three_albums_messages(name: str):
return List.of(SYSTEM_MSG, UserMessage.from_(
f"Only list the top 3 albums of {name}"
))package example.micronaut
import dev.langchain4j.data.message.ChatMessage
import dev.langchain4j.data.message.SystemMessage
import dev.langchain4j.data.message.UserMessage
import dev.langchain4j.model.chat.ChatModel
import jakarta.inject.Singleton
@Singleton
class MusicianAssistant(private val model: ChatModel) { // (1)
fun generateTopThreeAlbums(name: String): Musician {
val messages = generateTopThreeAlbumsMessages(name)
val albums = model.chat(messages)
val topThreeAlbums = albums.aiMessage().text()
return Musician(name, topThreeAlbums)
}
private fun generateTopThreeAlbumsMessages(name: String): List<ChatMessage> {
return listOf(
SYSTEM_MSG, UserMessage.from(
String.format("Only list the top 3 albums of %s", name)
)
)
}
companion object {
private val SYSTEM_MSG: SystemMessage = SystemMessage.from(
"""
You are an expert in Jazz music.
Reply with only the names of the artists, albums, etc.
Be very concise.
If a list is given, separate the items with commas.
""".trimIndent()
)
}
}package example.micronaut
import dev.langchain4j.data.message.ChatMessage
import dev.langchain4j.data.message.SystemMessage
import dev.langchain4j.data.message.UserMessage
import dev.langchain4j.model.chat.ChatModel
import dev.langchain4j.model.chat.response.ChatResponse
import jakarta.inject.Singleton
import groovy.transform.CompileStatic
@CompileStatic
@Singleton
class MusicianAssistant {
private static final SystemMessage SYSTEM_MSG = SystemMessage.from("""
You are an expert in Jazz music.
Reply with only the names of the artists, albums, etc.
Be very concise.
If a list is given, separate the items with commas.""")
private final ChatModel model
MusicianAssistant(ChatModel model) { // (1)
this.model = model
}
Musician generateTopThreeAlbums(String name) {
List<ChatMessage> messages = generateTopThreeAlbumsMessages(name)
ChatResponse albums = model.chat(messages);
String topThreeAlbums = albums.aiMessage().text();
new Musician(name: name, albums: topThreeAlbums);
}
private static List<ChatMessage> generateTopThreeAlbumsMessages(String name) {
[
SYSTEM_MSG,
UserMessage.from(String.format("Only list the top 3 albums of %s", name))
]
}
}| 1 | Inject via constructor injection a bean of type ChatModel. |
package example.micronaut;
public record Musician(String name, String albums) {
}from dataclasses import dataclass
from micronaut.core.annotation import Introspected
@Introspected
@dataclass(frozen=True)
class Musician:
name: str
albums: strpackage example.micronaut
data class Musician(val name: String, val albums: String)package example.micronaut
import groovy.transform.CompileStatic
@CompileStatic
class Musician {
String name
String albums
}You can configure the chat model via configuration.
For example, you may want to configure OpenAI in the main classpath:
src/main/resources/application.properties
micronaut.application.name=micronaut-guide
langchain4j.open-ai.chat-model.log-requests=true
langchain4j.open-ai.chat-model.log-responses=true
langchain4j.open-ai.chat-model.timeout=60s
langchain4j.open-ai.chat-model.temperature=0.3
langchain4j.open-ai.chat-model.model-name=gpt-4.1And a local SLM (Small Language Model) such as Ollama in the test classpath:
src/test/resources/application-test.properties
langchain4j.open-ai.enabled=false
langchain4j.ollama.model-name=tinyllama
langchain4j.ollama.chat-model.timeout=5m
# Ollama 0.32.10 changed the default from 1.1 to 1.0; without the penalty tinyllama ignores the chat history
langchain4j.ollama.chat-model.repeat-penalty=1.1
langchain4j.ollama.chat-model.log-requests=true
langchain4j.ollama.chat-model.log-responses=trueMoreover, you can also register a bean of type BeanCreatedEventListener to configure the Chat Model builder programmatically if configuration is not enough.
package example.micronaut;
import dev.langchain4j.model.ollama.OllamaChatModel;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import org.jspecify.annotations.NonNull;
import jakarta.inject.Singleton;
@Singleton
class OllamaChatModelBuilderListener
implements BeanCreatedEventListener<OllamaChatModel.OllamaChatModelBuilder> {
@Override
public OllamaChatModel.OllamaChatModelBuilder onCreated(
@NonNull BeanCreatedEvent<OllamaChatModel.OllamaChatModelBuilder> event) {
OllamaChatModel.OllamaChatModelBuilder builder = event.getBean();
builder.temperature(0.0);
return builder;
}
}from dev.langchain4j.model.ollama import OllamaChatModel
from jakarta.inject import Singleton
from micronaut.context.event import BeanCreatedEvent, BeanCreatedEventListener
@Singleton
class OllamaChatModelBuilderListener(
BeanCreatedEventListener[OllamaChatModel.OllamaChatModelBuilder]
):
def onCreated(
self, event: BeanCreatedEvent[OllamaChatModel.OllamaChatModelBuilder]
) -> OllamaChatModel.OllamaChatModelBuilder:
builder = event.getBean()
builder.temperature(0.0)
return builderpackage example.micronaut
import dev.langchain4j.model.ollama.OllamaChatModel.OllamaChatModelBuilder
import io.micronaut.context.event.BeanCreatedEvent
import io.micronaut.context.event.BeanCreatedEventListener
import org.jspecify.annotations.NonNull
import jakarta.inject.Singleton
@Singleton
class OllamaChatModelBuilderListener : BeanCreatedEventListener<OllamaChatModelBuilder> {
override fun onCreated(event: @NonNull BeanCreatedEvent<OllamaChatModelBuilder>): OllamaChatModelBuilder {
val builder = event.bean
builder.temperature(0.0)
return builder
}
}package example.micronaut
import dev.langchain4j.model.ollama.OllamaChatModel
import io.micronaut.context.event.BeanCreatedEvent
import io.micronaut.context.event.BeanCreatedEventListener
import org.jspecify.annotations.NonNull
import jakarta.inject.Singleton
@Singleton
class OllamaChatModelBuilderListener
implements BeanCreatedEventListener<OllamaChatModel.OllamaChatModelBuilder> {
@Override
OllamaChatModel.OllamaChatModelBuilder onCreated(
@NonNull BeanCreatedEvent<OllamaChatModel.OllamaChatModelBuilder> event) {
OllamaChatModel.OllamaChatModelBuilder builder = event.getBean()
builder.temperature(0.0)
builder
}
}17.2 Chat Memory
Models are stateless by design. Chat memory serves as container for previous messages, helping you maintain context in a conversation, but the model itself is not aware of this memory; it relies on you to include the relevant messages in each request for coherent and contextually relevant responses.
Langchain4J provides an API ChatMemory to help you manage chat memory. You can provide your own implementation or use one of the provided implementations.
The default implementation of ChatMemory, dev.langchain4j.store.memory.chat.InMemoryChatMemoryStore, stores ChatMessage instances in memory.
To use the Redis implementation dev.langchain4j.community.store.memory.chat.redis.RedisChatMemoryStore, add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-redis")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-redis</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-redis",
]To use the Neo4J implementation dev.langchain4j.community.store.memory.chat.neo4j.Neo4jChatMemoryStore, add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-neo4j")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-neo4j</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-neo4j",
]To use the Cassandra implementation dev.langchain4j.store.memory.chat.cassandra.CassandraChatMemoryStore, add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-cassandra")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-cassandra</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-cassandra",
]To use the Oracle implementation dev.langchain4j.store.memory.chat.oracle.OracleChatMemoryStore, add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-oracle")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-oracle</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-oracle",
]Then configure a JDBC datasource and the chat memory store properties, for example:
datasources.default.dialect=oracle
langchain4j.chat-memory-store.oracle.default.enabled=true
langchain4j.chat-memory-store.oracle.default.table-name=CHAT_MEMORY
langchain4j.chat-memory-store.oracle.default.memory-id-column-name=MEMORY_ID
langchain4j.chat-memory-store.oracle.default.content-column-name=CONTENTdatasources.default.dialect: oracle
langchain4j.chat-memory-store.oracle.default.enabled: true
langchain4j.chat-memory-store.oracle.default.table-name: CHAT_MEMORY
langchain4j.chat-memory-store.oracle.default.memory-id-column-name: MEMORY_ID
langchain4j.chat-memory-store.oracle.default.content-column-name: CONTENT"datasources.default.dialect" = "oracle"
"langchain4j.chat-memory-store.oracle.default.enabled" = true
"langchain4j.chat-memory-store.oracle.default.table-name" = "CHAT_MEMORY"
"langchain4j.chat-memory-store.oracle.default.memory-id-column-name" = "MEMORY_ID"
"langchain4j.chat-memory-store.oracle.default.content-column-name" = "CONTENT"datasources.default.dialect = "oracle"
langchain4j.chatMemoryStore.oracle.default.enabled = true
langchain4j.chatMemoryStore.oracle.default.tableName = "CHAT_MEMORY"
langchain4j.chatMemoryStore.oracle.default.memoryIdColumnName = "MEMORY_ID"
langchain4j.chatMemoryStore.oracle.default.contentColumnName = "CONTENT"{
"datasources.default.dialect" = "oracle"
"langchain4j.chat-memory-store.oracle.default.enabled" = true
"langchain4j.chat-memory-store.oracle.default.table-name" = "CHAT_MEMORY"
"langchain4j.chat-memory-store.oracle.default.memory-id-column-name" = "MEMORY_ID"
"langchain4j.chat-memory-store.oracle.default.content-column-name" = "CONTENT"
}{
"datasources.default.dialect": "oracle",
"langchain4j.chat-memory-store.oracle.default.enabled": true,
"langchain4j.chat-memory-store.oracle.default.table-name": "CHAT_MEMORY",
"langchain4j.chat-memory-store.oracle.default.memory-id-column-name": "MEMORY_ID",
"langchain4j.chat-memory-store.oracle.default.content-column-name": "CONTENT"
}The table must already exist. By default, the expected schema is CHAT_MEMORY(MEMORY_ID, CONTENT).
The segment under oracle (for example default) maps to the datasource name.
For multiple datasources, configure multiple entries such as
langchain4j.chat-memory-store.oracle.reporting.*.
To use the JDBC implementation dev.langchain4j.community.store.memory.chat.sql.SQLChatMemoryStore (PostgreSQL, MySQL, MariaDB or H2), add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-jdbc")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-jdbc</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-jdbc",
]Then configure the chat memory store of a datasource. The SQL dialect is detected from the database, and auto-create-table creates the table if it does not exist:
langchain4j.chat-memory-store.jdbc.default.table-name=chat_memory
langchain4j.chat-memory-store.jdbc.default.auto-create-table=truelangchain4j.chat-memory-store.jdbc.default.table-name: chat_memory
langchain4j.chat-memory-store.jdbc.default.auto-create-table: true"langchain4j.chat-memory-store.jdbc.default.table-name" = "chat_memory"
"langchain4j.chat-memory-store.jdbc.default.auto-create-table" = truelangchain4j.chatMemoryStore.jdbc.default.tableName = "chat_memory"
langchain4j.chatMemoryStore.jdbc.default.autoCreateTable = true{
"langchain4j.chat-memory-store.jdbc.default.table-name" = "chat_memory"
"langchain4j.chat-memory-store.jdbc.default.auto-create-table" = true
}{
"langchain4j.chat-memory-store.jdbc.default.table-name": "chat_memory",
"langchain4j.chat-memory-store.jdbc.default.auto-create-table": true
}The segment under jdbc (for example default) is the name of the datasource.
For Oracle Database, use the dedicated Oracle chat memory store of micronaut-langchain4j-store-oracle described above rather than the JDBC store.
|
To use the MongoDB implementation dev.langchain4j.community.store.memory.chat.mongodb.MongoDbChatMemoryStore, add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-mongodb")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-mongodb</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-mongodb",
]Then configure the database and the collection, and optionally expire-after-seconds to delete conversations after some time. The store uses the default MongoDB client, or the one of the server named:
langchain4j.chat-memory-store.mongodb.default.database-name=chat
langchain4j.chat-memory-store.mongodb.default.collection-name=memorylangchain4j.chat-memory-store.mongodb.default.database-name: chat
langchain4j.chat-memory-store.mongodb.default.collection-name: memory"langchain4j.chat-memory-store.mongodb.default.database-name" = "chat"
"langchain4j.chat-memory-store.mongodb.default.collection-name" = "memory"langchain4j.chatMemoryStore.mongodb.default.databaseName = "chat"
langchain4j.chatMemoryStore.mongodb.default.collectionName = "memory"{
"langchain4j.chat-memory-store.mongodb.default.database-name" = "chat"
"langchain4j.chat-memory-store.mongodb.default.collection-name" = "memory"
}{
"langchain4j.chat-memory-store.mongodb.default.database-name": "chat",
"langchain4j.chat-memory-store.mongodb.default.collection-name": "memory"
}Chat Memory of the AI Services
The AI services whose methods have a @MemoryId parameter keep a memory per memory ID. Their default ChatMemoryProvider keeps the conversations in the chat memory store named by langchain4j.chat-memory.store, otherwise in the persistent chat memory store if the application has exactly one (Redis, Neo4j, Cassandra, Oracle, JDBC or MongoDB), otherwise in memory. A ChatMemoryProvider bean named after the service, or annotated with @Primary, replaces it.
By default the memory keeps the last 20 messages. A token window memory keeps the last messages that fit in a number of tokens instead, counted with the TokenCountEstimator bean of the application (for example OpenAiTokenCountEstimator):
langchain4j.chat-memory.type=token-window
langchain4j.chat-memory.max-tokens=4000langchain4j.chat-memory.type: token-window
langchain4j.chat-memory.max-tokens: 4000"langchain4j.chat-memory.type" = "token-window"
"langchain4j.chat-memory.max-tokens" = 4000langchain4j.chatMemory.type = "token-window"
langchain4j.chatMemory.maxTokens = 4000{
"langchain4j.chat-memory.type" = "token-window"
"langchain4j.chat-memory.max-tokens" = 4000
}{
"langchain4j.chat-memory.type": "token-window",
"langchain4j.chat-memory.max-tokens": 4000
}| Property | Type | Description | Default value |
|---|---|---|---|
|
boolean |
Whether the default chat memory provider is enabled. Defaults to {@code true}. |
|
|
java.lang.String |
The name of the chat memory store bean |
|
|
How the memory evicts old messages |
||
|
java.lang.Integer |
The maximum number of messages of a message window memory |
|
|
java.lang.Integer |
The maximum number of tokens of a token window memory |
The following example shows how to use the ChatMemory:
package example.micronaut;
import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.memory.ChatMemory;
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.response.ChatResponse;
import jakarta.inject.Singleton;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
@Singleton
public class AssistantWithMemory {
private final Map<String, ChatMemory> conversations = new ConcurrentHashMap<>();
private final ChatModel model;
private final MessageWindowChatMemory.Builder messageWindowChatMemoryBuilder;
public AssistantWithMemory(MessageWindowChatMemory.Builder messageWindowChatMemoryBuilder,
ChatModel model) {
this.messageWindowChatMemoryBuilder = messageWindowChatMemoryBuilder;
this.model = model;
}
public MemoryIdAndResponse chat(String conversationId, String message) {
ChatMemory chatMemory = conversations.get(conversationId);
if (chatMemory == null) {
throw new IllegalArgumentException("Unknown conversation: " + conversationId);
}
chatMemory.add(UserMessage.from(message));
ChatResponse chatResponse = model.chat(chatMemory.messages());
AiMessage answer = chatResponse.aiMessage();
chatMemory.add(answer);
return new MemoryIdAndResponse(conversationId, answer.text());
}
public MemoryIdAndResponse chat(String message) {
String conversationId = startConversation();
return chat(conversationId, message);
}
private String startConversation() {
String memoryId = generateChatMemoryId();
ChatMemory chatMemory = generateChatMemory(memoryId);
conversations.putIfAbsent(memoryId, chatMemory);
return memoryId;
}
private String generateChatMemoryId() {
return UUID.randomUUID().toString();
}
private ChatMemory generateChatMemory(String memoryId) {
return messageWindowChatMemoryBuilder
.id(memoryId)
.build();
}
}from dev.langchain4j.data.message import UserMessage
from dev.langchain4j.memory.chat import MessageWindowChatMemory
from dev.langchain4j.model.chat import ChatModel
from jakarta.inject import Singleton
from java.util import UUID
from java.util.concurrent import ConcurrentHashMap
from .MemoryIdAndResponse import MemoryIdAndResponse
@Singleton
class AssistantWithMemory:
def __init__(self, message_window_chat_memory_builder: MessageWindowChatMemory.Builder,
model: ChatModel):
self.conversations = ConcurrentHashMap()
self.message_window_chat_memory_builder = message_window_chat_memory_builder
self.model = model
def chat(self, message: str, conversation_id: str | None = None) -> MemoryIdAndResponse:
if conversation_id is None:
conversation_id = self.start_conversation()
chat_memory = self.conversations.get(conversation_id)
if chat_memory is None:
raise ValueError(f"Unknown conversation: {conversation_id}")
chat_memory.add(UserMessage.from_(message))
chat_response = self.model.chat(chat_memory.messages())
answer = chat_response.aiMessage()
chat_memory.add(answer)
return MemoryIdAndResponse(conversation_id, answer.text())
def start_conversation(self) -> str:
memory_id = self.generate_chat_memory_id()
chat_memory = self.generate_chat_memory(memory_id)
self.conversations.putIfAbsent(memory_id, chat_memory)
return memory_id
def generate_chat_memory_id(self) -> str:
return UUID.randomUUID().toString()
def generate_chat_memory(self, memory_id: str):
return (self.message_window_chat_memory_builder
.id(memory_id)
.build())package example.micronaut
import dev.langchain4j.data.message.UserMessage
import dev.langchain4j.memory.ChatMemory
import dev.langchain4j.memory.chat.MessageWindowChatMemory
import dev.langchain4j.model.chat.ChatModel
import jakarta.inject.Singleton
import java.util.*
import java.util.concurrent.ConcurrentHashMap
@Singleton
class AssistantWithMemory(
val messageWindowChatMemoryBuilder: MessageWindowChatMemory.Builder,
val model: ChatModel) {
private val conversations: MutableMap<String, ChatMemory> = ConcurrentHashMap<String, ChatMemory>()
fun chat(conversationId: String, message: String): MemoryIdAndResponse {
val chatMemory = requireNotNull(this.conversations[conversationId]) {
"Unknown conversation: $conversationId"
}
chatMemory.add(UserMessage.from(message))
val chatResponse = model.chat(chatMemory.messages())
val aiMessage = chatResponse.aiMessage()
chatMemory.add(aiMessage)
return MemoryIdAndResponse(conversationId, aiMessage.text())
}
fun chat(message: String): MemoryIdAndResponse {
val conversationId = startConversation()
return chat(conversationId, message)
}
private fun startConversation(): String {
val memoryId = generateChatMemoryId()
val chatMemory = generateChatMemory(memoryId)
conversations.putIfAbsent(memoryId, chatMemory)
return memoryId
}
private fun generateChatMemoryId(): String {
return UUID.randomUUID().toString()
}
private fun generateChatMemory(memoryId: String): ChatMemory {
return messageWindowChatMemoryBuilder
.id(memoryId)
.build()
}
}package example.micronaut
import dev.langchain4j.data.message.AiMessage
import dev.langchain4j.data.message.UserMessage
import dev.langchain4j.memory.ChatMemory
import dev.langchain4j.memory.chat.MessageWindowChatMemory
import dev.langchain4j.model.chat.ChatModel
import dev.langchain4j.model.chat.response.ChatResponse
import jakarta.inject.Singleton
import java.util.concurrent.ConcurrentHashMap
@Singleton
class AssistantWithMemory {
private final Map<String, ChatMemory> conversations = new ConcurrentHashMap<>()
private final MessageWindowChatMemory.Builder messageWindowChatMemoryBuilder
private final ChatModel model
AssistantWithMemory(MessageWindowChatMemory.Builder messageWindowChatMemoryBuilder,
ChatModel model) {
this.model = model
this.messageWindowChatMemoryBuilder = messageWindowChatMemoryBuilder
}
private String startConversation() {
String memoryId = generateChatMemoryId()
ChatMemory chatMemory = generateChatMemory(memoryId)
conversations.putIfAbsent(memoryId, chatMemory)
memoryId
}
private String generateChatMemoryId() {
UUID.randomUUID().toString()
}
private ChatMemory generateChatMemory(String memoryId) {
messageWindowChatMemoryBuilder
.id(memoryId)
.build()
}
MemoryIdAndResponse chat(String memoryId, String message) {
ChatMemory chatMemory = conversations.get(memoryId)
if (chatMemory == null) {
throw new IllegalArgumentException("Unknown conversation: " + memoryId)
}
chatMemory.add(UserMessage.from(message))
ChatResponse chatResponse = model.chat(chatMemory.messages())
AiMessage aiMessage = chatResponse.aiMessage()
chatMemory.add(aiMessage)
new MemoryIdAndResponse(memoryId: memoryId, response: aiMessage.text())
}
MemoryIdAndResponse chat(String message) {
String conversationId = startConversation()
chat(conversationId, message)
}
}You could invoke the previous class as illustrated in the following test:
@Test
void chatWithMemory(AssistantWithMemory assistant) {
MemoryIdAndResponse johnConversation = assistant.chat("Let me introduce myself. My name is John");
String johnConversationId = johnConversation.memoryId();
assertNotNull(johnConversationId);
MemoryIdAndResponse aegonConversation = assistant.chat("Let me introduce myself. My name is Dan");
String aegonConversationId = aegonConversation.memoryId();
assertNotNull(aegonConversationId);
MemoryIdAndResponse answer = assistant.chat(johnConversationId, "What's my name?");
assertTrue(answer.response().toLowerCase().contains("john"), answer.response());
answer = assistant.chat(aegonConversationId, "What's my name?");
assertTrue(answer.response().toLowerCase().contains("dan"), answer.response());
} @Test
def chat_with_memory(self):
john_conversation = self.assistant.chat("Let me introduce myself. My name is John")
john_conversation_id = john_conversation.memory_id
assert john_conversation_id is not None
aegon_conversation = self.assistant.chat("Let me introduce myself. My name is Dan")
aegon_conversation_id = aegon_conversation.memory_id
assert aegon_conversation_id is not None
answer = self.assistant.chat("What's my name?", john_conversation_id)
assert "john" in answer.response.lower(), answer.response
answer = self.assistant.chat("What's my name?", aegon_conversation_id)
assert "dan" in answer.response.lower(), answer.response @Test
fun chatWithMemory(assistant: AssistantWithMemory) {
val johnConversation = assistant.chat("Let me introduce myself. My name is John")
val johnConversationId = johnConversation.memoryId
assertNotNull(johnConversationId)
val aegonConversation = assistant.chat("Let me introduce myself. My name is Dan")
val aegonConversationId = aegonConversation.memoryId
assertNotNull(aegonConversationId)
var answer = assistant.chat(johnConversationId, "What's my name?")
assertTrue(answer.response.lowercase().contains("john"), answer.response)
answer = assistant.chat(aegonConversationId, "What's my name?")
assertTrue(answer.response.lowercase().contains("dan"), answer.response)
} @Test
void chatWithMemory(AssistantWithMemory assistant) {
MemoryIdAndResponse johnConversation = assistant.chat("Let me introduce myself. My name is John")
String johnConversationId = johnConversation.memoryId
assertNotNull(johnConversationId)
MemoryIdAndResponse aegonConversation = assistant.chat("Let me introduce myself. My name is Dan")
String aegonConversationId = aegonConversation.memoryId
assertNotNull(aegonConversationId)
MemoryIdAndResponse answer = assistant.chat(johnConversationId, "What's my name?")
assertTrue(answer.response.toLowerCase().contains("john"), answer.response)
answer = assistant.chat(aegonConversationId, "What's my name?")
assertTrue(answer.response.toLowerCase().contains("dan"), answer.response)
}17.3 Anthropic
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-anthropic")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-anthropic</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-anthropic",
]Then add the necessary configuration.
Example Configuration
langchain4j.anthropic.api-key=YOUR_KEYlangchain4j.anthropic.api-key: YOUR_KEY"langchain4j.anthropic.api-key" = "YOUR_KEY"langchain4j.anthropic.apiKey = "YOUR_KEY"{
"langchain4j.anthropic.api-key" = "YOUR_KEY"
}{
"langchain4j.anthropic.api-key": "YOUR_KEY"
}Extended thinking and prompt caching are configured on the models:
Example Configuration
langchain4j.anthropic.api-key=${ANTHROPIC_API_KEY}
langchain4j.anthropic.chat-model.model-name=claude-sonnet-4-5
langchain4j.anthropic.chat-model.thinking-type=enabled
langchain4j.anthropic.chat-model.thinking-budget-tokens=2048
langchain4j.anthropic.chat-model.return-thinking=true
langchain4j.anthropic.chat-model.cache-system-messages=true
langchain4j.anthropic.chat-model.cache-tools=truelangchain4j:
anthropic:
api-key: ${ANTHROPIC_API_KEY}
chat-model:
model-name: claude-sonnet-4-5
thinking-type: enabled
thinking-budget-tokens: 2048
return-thinking: true
cache-system-messages: true
cache-tools: true[langchain4j.anthropic]
api-key = "${ANTHROPIC_API_KEY}"
[langchain4j.anthropic.chat-model]
model-name = "claude-sonnet-4-5"
thinking-type = "enabled"
thinking-budget-tokens = 2048
return-thinking = true
cache-system-messages = true
cache-tools = truelangchain4j {
anthropic {
apiKey = "${ANTHROPIC_API_KEY}"
chatModel {
modelName = "claude-sonnet-4-5"
thinkingType = "enabled"
thinkingBudgetTokens = 2048
returnThinking = true
cacheSystemMessages = true
cacheTools = true
}
}
}{
langchain4j {
anthropic {
api-key = "${ANTHROPIC_API_KEY}"
chat-model {
model-name = "claude-sonnet-4-5"
thinking-type = "enabled"
thinking-budget-tokens = 2048
return-thinking = true
cache-system-messages = true
cache-tools = true
}
}
}
}{
"langchain4j": {
"anthropic": {
"api-key": "${ANTHROPIC_API_KEY}",
"chat-model": {
"model-name": "claude-sonnet-4-5",
"thinking-type": "enabled",
"thinking-budget-tokens": 2048,
"return-thinking": true,
"cache-system-messages": true,
"cache-tools": true
}
}
}
}17.4 Azure
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-azure")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-azure</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-azure",
]Then add the necessary configuration.
Example Configuration
langchain4j.azure-open-ai.api-key=YOUR_KEY
langchain4j.azure-open-ai.endpoint=YOUR_ENDPOINTlangchain4j.azure-open-ai.api-key: YOUR_KEY
langchain4j.azure-open-ai.endpoint: YOUR_ENDPOINT"langchain4j.azure-open-ai.api-key" = "YOUR_KEY"
"langchain4j.azure-open-ai.endpoint" = "YOUR_ENDPOINT"langchain4j.azureOpenAi.apiKey = "YOUR_KEY"
langchain4j.azureOpenAi.endpoint = "YOUR_ENDPOINT"{
"langchain4j.azure-open-ai.api-key" = "YOUR_KEY"
"langchain4j.azure-open-ai.endpoint" = "YOUR_ENDPOINT"
}{
"langchain4j.azure-open-ai.api-key": "YOUR_KEY",
"langchain4j.azure-open-ai.endpoint": "YOUR_ENDPOINT"
}You will additionally need to define a bean of type TokenCredentials.
One way to do this is to include the Azure SDK module.
The audio transcription model (dev.langchain4j.model.audio.AudioTranscriptionModel) is only created when it is configured, for example with langchain4j.azure-open-ai.audio-transcription-model.deployment-name.
17.5 Bedrock
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-bedrock")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-bedrock</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-bedrock",
]Then add the necessary configuration.
Example Configuration
langchain4j.bedrock.region=us-east-1
langchain4j.bedrock.chat-model.model-id=anthropic.claude-3-haiku-20240307-v1:0langchain4j.bedrock.region: us-east-1
langchain4j.bedrock.chat-model.model-id: anthropic.claude-3-haiku-20240307-v1:0"langchain4j.bedrock.region" = "us-east-1"
"langchain4j.bedrock.chat-model.model-id" = "anthropic.claude-3-haiku-20240307-v1:0"langchain4j.bedrock.region = "us-east-1"
langchain4j.bedrock.chatModel.modelId = "anthropic.claude-3-haiku-20240307-v1:0"{
"langchain4j.bedrock.region" = "us-east-1"
"langchain4j.bedrock.chat-model.model-id" = "anthropic.claude-3-haiku-20240307-v1:0"
}{
"langchain4j.bedrock.region": "us-east-1",
"langchain4j.bedrock.chat-model.model-id": "anthropic.claude-3-haiku-20240307-v1:0"
}The model-id is the Amazon Bedrock model ID or inference profile ID to invoke. There is no default value, choose a model that is enabled for your account and region.
You will additionally need to define a bean of type AwsCredentialsProvider.
One way to do this is to include the AWS SDK module.
17.6 HuggingFace
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-hugging-face")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-hugging-face</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-hugging-face",
]The models use the Micronaut HTTP client. Add a Micronaut HTTP client implementation to your application, for example the default Netty implementation:
runtimeOnly("io.micronaut:micronaut-http-client")<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-http-client</artifactId>
<scope>runtime</scope>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut:micronaut-http-client",
]Then add the necessary configuration.
Example Configuration
langchain4j.hugging-face.access-token=YOUR_ACCESS_TOKENlangchain4j.hugging-face.access-token: YOUR_ACCESS_TOKEN"langchain4j.hugging-face.access-token" = "YOUR_ACCESS_TOKEN"langchain4j.huggingFace.accessToken = "YOUR_ACCESS_TOKEN"{
"langchain4j.hugging-face.access-token" = "YOUR_ACCESS_TOKEN"
}{
"langchain4j.hugging-face.access-token": "YOUR_ACCESS_TOKEN"
}17.7 Jlama
Jlama runs the models in the JVM, without a model server. Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-jlama")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-jlama</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-jlama",
]Then configure the model, which Jlama downloads from Hugging Face into its working directory on the first use:
Example Configuration
langchain4j.jlama.chat-model.model-name=tjake/Llama-3.2-1B-Instruct-JQ4
langchain4j.jlama.embedding-model.model-name=intfloat/e5-small-v2langchain4j:
jlama:
chat-model:
model-name: tjake/Llama-3.2-1B-Instruct-JQ4
embedding-model:
model-name: intfloat/e5-small-v2[langchain4j.jlama.chat-model]
model-name = "tjake/Llama-3.2-1B-Instruct-JQ4"
[langchain4j.jlama.embedding-model]
model-name = "intfloat/e5-small-v2"langchain4j {
jlama {
chatModel {
modelName = "tjake/Llama-3.2-1B-Instruct-JQ4"
}
embeddingModel {
modelName = "intfloat/e5-small-v2"
}
}
}{
langchain4j {
jlama {
chat-model {
model-name = "tjake/Llama-3.2-1B-Instruct-JQ4"
}
embedding-model {
model-name = "intfloat/e5-small-v2"
}
}
}
}{
"langchain4j": {
"jlama": {
"chat-model": {
"model-name": "tjake/Llama-3.2-1B-Instruct-JQ4"
},
"embedding-model": {
"model-name": "intfloat/e5-small-v2"
}
}
}
}
Jlama uses the incubating Vector API of the JDK: run the application with --add-modules jdk.incubator.vector --enable-preview. Jlama is not supported in a GraalVM native image.
|
17.8 MistralAi
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-mistralai")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-mistralai</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-mistralai",
]Then add the necessary configuration.
Example Configuration
langchain4j.mistral-ai.api-key=YOUR_KEYlangchain4j.mistral-ai.api-key: YOUR_KEY"langchain4j.mistral-ai.api-key" = "YOUR_KEY"langchain4j.mistralAi.apiKey = "YOUR_KEY"{
"langchain4j.mistral-ai.api-key" = "YOUR_KEY"
}{
"langchain4j.mistral-ai.api-key": "YOUR_KEY"
}17.9 Ollama
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-ollama")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-ollama</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-ollama",
]Then add the necessary configuration.
Example Configuration
langchain4j.ollama.base-url=YOUR_URLlangchain4j.ollama.base-url: YOUR_URL"langchain4j.ollama.base-url" = "YOUR_URL"langchain4j.ollama.baseUrl = "YOUR_URL"{
"langchain4j.ollama.base-url" = "YOUR_URL"
}{
"langchain4j.ollama.base-url": "YOUR_URL"
}17.10 Oracle Cloud GenAI
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-oci-genai")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-oci-genai</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-oci-genai",
]Setup a supported OCI authentication method.
Then add the necessary configuration to configure a chat model.
Example Configuration
langchain4j.oci-gen-ai.chat-model.model-name=orca-mini
langchain4j.oci-gen-ai.compartment-id=your-compartmentlangchain4j.oci-gen-ai.chat-model.model-name: orca-mini
langchain4j.oci-gen-ai.compartment-id: your-compartment"langchain4j.oci-gen-ai.chat-model.model-name" = "orca-mini"
"langchain4j.oci-gen-ai.compartment-id" = "your-compartment"langchain4j.ociGenAi.chatModel.modelName = "orca-mini"
langchain4j.ociGenAi.compartmentId = "your-compartment"{
"langchain4j.oci-gen-ai.chat-model.model-name" = "orca-mini"
"langchain4j.oci-gen-ai.compartment-id" = "your-compartment"
}{
"langchain4j.oci-gen-ai.chat-model.model-name": "orca-mini",
"langchain4j.oci-gen-ai.compartment-id": "your-compartment"
}17.11 OpenAi
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-openai")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-openai</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-openai",
]Provider modules use Micronaut’s HTTP client API and exclude LangChain4j’s JDK HTTP client. Add a concrete Micronaut HTTP client implementation to your application, for example the default Netty implementation:
runtimeOnly("io.micronaut:micronaut-http-client")<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-http-client</artifactId>
<scope>runtime</scope>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut:micronaut-http-client",
]For tests that instantiate OpenAI models, add the same implementation to the test runtime classpath:
testRuntimeOnly("io.micronaut:micronaut-http-client")<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-http-client</artifactId>
<scope>test</scope>
</dependency>[tool.pyronaut.dependencies]
test = [
"io.micronaut:micronaut-http-client",
]Then add the necessary configuration.
Example Configuration
langchain4j.open-ai.api-key=YOUR_KEYlangchain4j.open-ai.api-key: YOUR_KEY"langchain4j.open-ai.api-key" = "YOUR_KEY"langchain4j.openAi.apiKey = "YOUR_KEY"{
"langchain4j.open-ai.api-key" = "YOUR_KEY"
}{
"langchain4j.open-ai.api-key": "YOUR_KEY"
}Audio
The audio transcription model (dev.langchain4j.model.audio.AudioTranscriptionModel) and the text-to-speech model (dev.langchain4j.model.audio.TextToSpeechModel) are only created when they are configured:
Example Configuration
langchain4j.open-ai.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.audio-transcription-model.model-name=gpt-4o-mini-transcribe
langchain4j.open-ai.text-to-speech-model.model-name=gpt-4o-mini-tts
langchain4j.open-ai.text-to-speech-model.voice=alloylangchain4j:
open-ai:
api-key: ${OPENAI_API_KEY}
audio-transcription-model:
model-name: gpt-4o-mini-transcribe
text-to-speech-model:
model-name: gpt-4o-mini-tts
voice: alloy[langchain4j.open-ai]
api-key = "${OPENAI_API_KEY}"
[langchain4j.open-ai.audio-transcription-model]
model-name = "gpt-4o-mini-transcribe"
[langchain4j.open-ai.text-to-speech-model]
model-name = "gpt-4o-mini-tts"
voice = "alloy"langchain4j {
openAi {
apiKey = "${OPENAI_API_KEY}"
audioTranscriptionModel {
modelName = "gpt-4o-mini-transcribe"
}
textToSpeechModel {
modelName = "gpt-4o-mini-tts"
voice = "alloy"
}
}
}{
langchain4j {
open-ai {
api-key = "${OPENAI_API_KEY}"
audio-transcription-model {
model-name = "gpt-4o-mini-transcribe"
}
text-to-speech-model {
model-name = "gpt-4o-mini-tts"
voice = "alloy"
}
}
}
}{
"langchain4j": {
"open-ai": {
"api-key": "${OPENAI_API_KEY}",
"audio-transcription-model": {
"model-name": "gpt-4o-mini-transcribe"
},
"text-to-speech-model": {
"model-name": "gpt-4o-mini-tts",
"voice": "alloy"
}
}
}
}Responses API
The chat models of the Responses API are configured under langchain4j.open-ai-responses, and share the API key and other common settings of langchain4j.open-ai. Since the Chat Completions model is the default OpenAI chat model, configure the Responses models with a name, and select them with the name, for example with @AiService(named = "assistant"):
Example Configuration
langchain4j.open-ai.api-key=${OPENAI_API_KEY}
langchain4j.open-ai-responses.chat-models.assistant.model-name=gpt-5-mini
langchain4j.open-ai-responses.streaming-chat-models.assistant.model-name=gpt-5-minilangchain4j:
open-ai:
api-key: ${OPENAI_API_KEY}
open-ai-responses:
chat-models:
assistant:
model-name: gpt-5-mini
streaming-chat-models:
assistant:
model-name: gpt-5-mini[langchain4j.open-ai]
api-key = "${OPENAI_API_KEY}"
[langchain4j.open-ai-responses.chat-models.assistant]
model-name = "gpt-5-mini"
[langchain4j.open-ai-responses.streaming-chat-models.assistant]
model-name = "gpt-5-mini"langchain4j {
openAi {
apiKey = "${OPENAI_API_KEY}"
}
openAiResponses {
chatModels {
assistant {
modelName = "gpt-5-mini"
}
}
streamingChatModels {
assistant {
modelName = "gpt-5-mini"
}
}
}
}{
langchain4j {
open-ai {
api-key = "${OPENAI_API_KEY}"
}
open-ai-responses {
chat-models {
assistant {
model-name = "gpt-5-mini"
}
}
streaming-chat-models {
assistant {
model-name = "gpt-5-mini"
}
}
}
}
}{
"langchain4j": {
"open-ai": {
"api-key": "${OPENAI_API_KEY}"
},
"open-ai-responses": {
"chat-models": {
"assistant": {
"model-name": "gpt-5-mini"
}
},
"streaming-chat-models": {
"assistant": {
"model-name": "gpt-5-mini"
}
}
}
}
}
The Responses API client of LangChain4j appends /responses to the base URL. The default base URL, https://api.openai.com/v1, suits both APIs; set a custom base-url (of langchain4j.open-ai or of the Responses models) without a trailing slash.
|
17.12 Google AI Gemini
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-googleai-gemini")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-googleai-gemini</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-googleai-gemini",
]Then add the necessary configuration.
Example Configuration
langchain4j.google-ai-gemini.api-key=YOUR_API_KEYlangchain4j.google-ai-gemini.api-key: YOUR_API_KEY"langchain4j.google-ai-gemini.api-key" = "YOUR_API_KEY"langchain4j.googleAiGemini.apiKey = "YOUR_API_KEY"{
"langchain4j.google-ai-gemini.api-key" = "YOUR_API_KEY"
}{
"langchain4j.google-ai-gemini.api-key": "YOUR_API_KEY"
}17.13 Google Gen AI
The Google Gen AI module uses the Google Gen AI SDK, which works with both the Gemini API and Vertex AI. It provides a chat model, a streaming chat model, an embedding model and an image model. Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-google-genai")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-google-genai</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-google-genai",
]To use the Gemini API, configure an API key:
Example Configuration
langchain4j.google-gen-ai.api-key=${GEMINI_API_KEY}
langchain4j.google-gen-ai.chat-model.model-name=gemini-3.8-flashlangchain4j:
google-gen-ai:
api-key: ${GEMINI_API_KEY}
chat-model:
model-name: gemini-3.8-flash[langchain4j.google-gen-ai]
api-key = "${GEMINI_API_KEY}"
[langchain4j.google-gen-ai.chat-model]
model-name = "gemini-3.8-flash"langchain4j {
googleGenAi {
apiKey = "${GEMINI_API_KEY}"
chatModel {
modelName = "gemini-3.8-flash"
}
}
}{
langchain4j {
google-gen-ai {
api-key = "${GEMINI_API_KEY}"
chat-model {
model-name = "gemini-3.8-flash"
}
}
}
}{
"langchain4j": {
"google-gen-ai": {
"api-key": "${GEMINI_API_KEY}",
"chat-model": {
"model-name": "gemini-3.8-flash"
}
}
}
}To use Vertex AI, configure the project and the location. The requests are authenticated with the GoogleCredentials bean of the application if there is one, otherwise with the application default credentials:
Example Configuration
langchain4j.google-gen-ai.vertex-ai=true
langchain4j.google-gen-ai.project-id=my-project
langchain4j.google-gen-ai.location=us-central1langchain4j:
google-gen-ai:
vertex-ai: true
project-id: my-project
location: us-central1[langchain4j.google-gen-ai]
vertex-ai = true
project-id = "my-project"
location = "us-central1"langchain4j {
googleGenAi {
vertexAi = true
projectId = "my-project"
location = "us-central1"
}
}{
langchain4j {
google-gen-ai {
vertex-ai = true
project-id = "my-project"
location = "us-central1"
}
}
}{
"langchain4j": {
"google-gen-ai": {
"vertex-ai": true,
"project-id": "my-project",
"location": "us-central1"
}
}
}The models share a com.google.genai.Client bean configured under langchain4j.google-gen-ai, including the connection pool of its HTTP client:
| Property | Type | Description | Default value |
|---|---|---|---|
|
java.lang.String |
The API key of the Gemini API |
|
|
boolean |
Whether Vertex AI is used instead of the Gemini API |
|
|
java.lang.String |
The Google Cloud project of Vertex AI |
|
|
java.lang.String |
The location of Vertex AI |
|
|
java.lang.String |
The base URL of the API |
|
|
java.lang.String |
The version of the API |
|
|
java.time.Duration |
The timeout of the requests |
|
|
java.lang.Integer |
The maximum number of connections of the HTTP client |
|
|
java.lang.Integer |
The maximum number of connections of the HTTP client per host |
The SDK uses OkHttp. To customize the HTTP client, for example to add interceptors, register a BeanCreatedEventListener for ClientOptions.Builder (or HttpOptions.Builder):
package example.micronaut.googlegenai;
import com.google.genai.types.ClientOptions;
import io.micronaut.context.event.BeanCreatedEvent;
import io.micronaut.context.event.BeanCreatedEventListener;
import jakarta.inject.Singleton;
import okhttp3.OkHttpClient;
import java.time.Duration;
@Singleton
public class HttpClientCustomizer implements BeanCreatedEventListener<ClientOptions.Builder> { // (1)
@Override
public ClientOptions.Builder onCreated(BeanCreatedEvent<ClientOptions.Builder> event) {
OkHttpClient httpClient = new OkHttpClient.Builder()
.callTimeout(Duration.ofMinutes(2))
.build();
return event.getBean().customHttpClient(httpClient); // (2)
}
}from com.google.genai.types import ClientOptions
from jakarta.inject import Singleton
from java.time import Duration
from micronaut.context.event import BeanCreatedEvent, BeanCreatedEventListener
from okhttp3 import OkHttpClient
@Singleton
class HttpClientCustomizer(BeanCreatedEventListener[ClientOptions.Builder]): # (1)
def onCreated(self, event: BeanCreatedEvent[ClientOptions.Builder]) -> ClientOptions.Builder:
http_client = OkHttpClient.Builder() \
.callTimeout(Duration.ofMinutes(2)) \
.build()
return event.getBean().customHttpClient(http_client) # (2)package example.micronaut.googlegenai
import com.google.genai.types.ClientOptions
import io.micronaut.context.event.BeanCreatedEvent
import io.micronaut.context.event.BeanCreatedEventListener
import jakarta.inject.Singleton
import okhttp3.OkHttpClient
import java.time.Duration
@Singleton
class HttpClientCustomizer : BeanCreatedEventListener<ClientOptions.Builder> { // (1)
override fun onCreated(event: BeanCreatedEvent<ClientOptions.Builder>): ClientOptions.Builder {
val httpClient = OkHttpClient.Builder()
.callTimeout(Duration.ofMinutes(2))
.build()
return event.bean.customHttpClient(httpClient) // (2)
}
}package example.micronaut.googlegenai
import com.google.genai.types.ClientOptions
import io.micronaut.context.event.BeanCreatedEvent
import io.micronaut.context.event.BeanCreatedEventListener
import jakarta.inject.Singleton
import okhttp3.OkHttpClient
import java.time.Duration
@Singleton
class HttpClientCustomizer implements BeanCreatedEventListener<ClientOptions.Builder> { // (1)
@Override
ClientOptions.Builder onCreated(BeanCreatedEvent<ClientOptions.Builder> event) {
OkHttpClient httpClient = new OkHttpClient.Builder()
.callTimeout(Duration.ofMinutes(2))
.build()
event.bean.customHttpClient(httpClient) // (2)
}
}| 1 | The listener is notified when the client options of the Google Gen AI client are created. |
| 2 | It sets the OkHttp client that the SDK uses. |
The labels of the requests are not configurable, because they are only supported by Vertex AI. Set them with a BeanCreatedEventListener for the model builder, for example GoogleGenAiChatModel.Builder.
|
17.14 VertexAi
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-vertexai")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-vertexai</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-vertexai",
]Then add the necessary configuration.
To provide explicit Google Cloud credentials, register a GoogleCredentials bean. When no such bean is present, the Vertex AI client uses Application Default Credentials.
Example Configuration
langchain4j.vertex-ai.endpoint=YOUR_ENDPOINT
langchain4j.vertex-ai.model-name=YOUR_MODEL
langchain4j.vertex-ai.project=YOUR_PROJECT
langchain4j.vertex-ai.location=YOUR_LOCATION
langchain4j.vertex-ai.publisher=YOUR_PUBLISHERlangchain4j.vertex-ai.endpoint: YOUR_ENDPOINT
langchain4j.vertex-ai.model-name: YOUR_MODEL
langchain4j.vertex-ai.project: YOUR_PROJECT
langchain4j.vertex-ai.location: YOUR_LOCATION
langchain4j.vertex-ai.publisher: YOUR_PUBLISHER"langchain4j.vertex-ai.endpoint" = "YOUR_ENDPOINT"
"langchain4j.vertex-ai.model-name" = "YOUR_MODEL"
"langchain4j.vertex-ai.project" = "YOUR_PROJECT"
"langchain4j.vertex-ai.location" = "YOUR_LOCATION"
"langchain4j.vertex-ai.publisher" = "YOUR_PUBLISHER"langchain4j.vertexAi.endpoint = "YOUR_ENDPOINT"
langchain4j.vertexAi.modelName = "YOUR_MODEL"
langchain4j.vertexAi.project = "YOUR_PROJECT"
langchain4j.vertexAi.location = "YOUR_LOCATION"
langchain4j.vertexAi.publisher = "YOUR_PUBLISHER"{
"langchain4j.vertex-ai.endpoint" = "YOUR_ENDPOINT"
"langchain4j.vertex-ai.model-name" = "YOUR_MODEL"
"langchain4j.vertex-ai.project" = "YOUR_PROJECT"
"langchain4j.vertex-ai.location" = "YOUR_LOCATION"
"langchain4j.vertex-ai.publisher" = "YOUR_PUBLISHER"
}{
"langchain4j.vertex-ai.endpoint": "YOUR_ENDPOINT",
"langchain4j.vertex-ai.model-name": "YOUR_MODEL",
"langchain4j.vertex-ai.project": "YOUR_PROJECT",
"langchain4j.vertex-ai.location": "YOUR_LOCATION",
"langchain4j.vertex-ai.publisher": "YOUR_PUBLISHER"
}17.15 VertexAi Gemini
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-vertexai-gemini")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-vertexai-gemini</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-vertexai-gemini",
]Then add the necessary configuration.
To provide explicit Google Cloud credentials, register a GoogleCredentials bean. When no such bean is present, the Vertex AI Gemini client uses Application Default Credentials.
Example Configuration
langchain4j.vertex-ai-gemini.model-name=YOUR_MODEL
langchain4j.vertex-ai-gemini.project=YOUR_PROJECT
langchain4j.vertex-ai-gemini.location=YOUR_LOCATIONlangchain4j.vertex-ai-gemini.model-name: YOUR_MODEL
langchain4j.vertex-ai-gemini.project: YOUR_PROJECT
langchain4j.vertex-ai-gemini.location: YOUR_LOCATION"langchain4j.vertex-ai-gemini.model-name" = "YOUR_MODEL"
"langchain4j.vertex-ai-gemini.project" = "YOUR_PROJECT"
"langchain4j.vertex-ai-gemini.location" = "YOUR_LOCATION"langchain4j.vertexAiGemini.modelName = "YOUR_MODEL"
langchain4j.vertexAiGemini.project = "YOUR_PROJECT"
langchain4j.vertexAiGemini.location = "YOUR_LOCATION"{
"langchain4j.vertex-ai-gemini.model-name" = "YOUR_MODEL"
"langchain4j.vertex-ai-gemini.project" = "YOUR_PROJECT"
"langchain4j.vertex-ai-gemini.location" = "YOUR_LOCATION"
}{
"langchain4j.vertex-ai-gemini.model-name": "YOUR_MODEL",
"langchain4j.vertex-ai-gemini.project": "YOUR_PROJECT",
"langchain4j.vertex-ai-gemini.location": "YOUR_LOCATION"
}18 Scoring Models
Scoring models (also called re-ranking models) score the relevance of text segments to a query. They are used to re-rank the content retrieved for retrieval augmented generation, see Configuring Retrieval.
Each module configures a dev.langchain4j.model.scoring.ScoringModel bean under langchain4j.<provider>.scoring-model (and named ones under langchain4j.<provider>.scoring-models.<name>), and an EmbeddingModel bean under langchain4j.<provider>.embedding-model. The models use the Micronaut HTTP client.
18.1 Cohere
Add the following dependency to use the Cohere models:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-cohere")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-cohere</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-cohere",
]Example Configuration
langchain4j.cohere.api-key=${COHERE_API_KEY}
langchain4j.cohere.scoring-model.model-name=rerank-v3.5
langchain4j.cohere.embedding-model.model-name=embed-v4.0langchain4j:
cohere:
api-key: ${COHERE_API_KEY}
scoring-model:
model-name: rerank-v3.5
embedding-model:
model-name: embed-v4.0[langchain4j.cohere]
api-key = "${COHERE_API_KEY}"
[langchain4j.cohere.scoring-model]
model-name = "rerank-v3.5"
[langchain4j.cohere.embedding-model]
model-name = "embed-v4.0"langchain4j {
cohere {
apiKey = "${COHERE_API_KEY}"
scoringModel {
modelName = "rerank-v3.5"
}
embeddingModel {
modelName = "embed-v4.0"
}
}
}{
langchain4j {
cohere {
api-key = "${COHERE_API_KEY}"
scoring-model {
model-name = "rerank-v3.5"
}
embedding-model {
model-name = "embed-v4.0"
}
}
}
}{
"langchain4j": {
"cohere": {
"api-key": "${COHERE_API_KEY}",
"scoring-model": {
"model-name": "rerank-v3.5"
},
"embedding-model": {
"model-name": "embed-v4.0"
}
}
}
}| Property | Type | Description | Default value |
|---|---|---|---|
|
boolean |
true |
|
|
java.lang.String |
||
|
java.lang.String |
||
|
java.time.Duration |
||
|
java.lang.Boolean |
false |
|
|
java.lang.Boolean |
false |
18.2 Jina
Add the following dependency to use the Jina models:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-jina")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-jina</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-jina",
]Example Configuration
langchain4j.jina.api-key=${JINA_API_KEY}
langchain4j.jina.scoring-model.model-name=jina-reranker-v2-base-multilingual
langchain4j.jina.embedding-model.model-name=jina-embeddings-v3langchain4j:
jina:
api-key: ${JINA_API_KEY}
scoring-model:
model-name: jina-reranker-v2-base-multilingual
embedding-model:
model-name: jina-embeddings-v3[langchain4j.jina]
api-key = "${JINA_API_KEY}"
[langchain4j.jina.scoring-model]
model-name = "jina-reranker-v2-base-multilingual"
[langchain4j.jina.embedding-model]
model-name = "jina-embeddings-v3"langchain4j {
jina {
apiKey = "${JINA_API_KEY}"
scoringModel {
modelName = "jina-reranker-v2-base-multilingual"
}
embeddingModel {
modelName = "jina-embeddings-v3"
}
}
}{
langchain4j {
jina {
api-key = "${JINA_API_KEY}"
scoring-model {
model-name = "jina-reranker-v2-base-multilingual"
}
embedding-model {
model-name = "jina-embeddings-v3"
}
}
}
}{
"langchain4j": {
"jina": {
"api-key": "${JINA_API_KEY}",
"scoring-model": {
"model-name": "jina-reranker-v2-base-multilingual"
},
"embedding-model": {
"model-name": "jina-embeddings-v3"
}
}
}
}| Property | Type | Description | Default value |
|---|---|---|---|
|
boolean |
true |
|
|
java.lang.String |
||
|
java.lang.String |
||
|
java.time.Duration |
||
|
java.lang.Boolean |
false |
|
|
java.lang.Boolean |
false |
18.3 Voyage AI
Add the following dependency to use the Voyage AI models:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-voyageai")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-voyageai</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-voyageai",
]Example Configuration
langchain4j.voyage-ai.api-key=${VOYAGE_API_KEY}
langchain4j.voyage-ai.scoring-model.model-name=rerank-2.5
langchain4j.voyage-ai.embedding-model.model-name=voyage-3.5langchain4j:
voyage-ai:
api-key: ${VOYAGE_API_KEY}
scoring-model:
model-name: rerank-2.5
embedding-model:
model-name: voyage-3.5[langchain4j.voyage-ai]
api-key = "${VOYAGE_API_KEY}"
[langchain4j.voyage-ai.scoring-model]
model-name = "rerank-2.5"
[langchain4j.voyage-ai.embedding-model]
model-name = "voyage-3.5"langchain4j {
voyageAi {
apiKey = "${VOYAGE_API_KEY}"
scoringModel {
modelName = "rerank-2.5"
}
embeddingModel {
modelName = "voyage-3.5"
}
}
}{
langchain4j {
voyage-ai {
api-key = "${VOYAGE_API_KEY}"
scoring-model {
model-name = "rerank-2.5"
}
embedding-model {
model-name = "voyage-3.5"
}
}
}
}{
"langchain4j": {
"voyage-ai": {
"api-key": "${VOYAGE_API_KEY}",
"scoring-model": {
"model-name": "rerank-2.5"
},
"embedding-model": {
"model-name": "voyage-3.5"
}
}
}
}| Property | Type | Description | Default value |
|---|---|---|---|
|
boolean |
true |
|
|
java.lang.String |
||
|
java.lang.String |
||
|
java.time.Duration |
||
|
java.lang.Boolean |
false |
|
|
java.lang.Boolean |
false |
19 Embedding Stores
19.1 In-Memory
An in-memory embedding store is enabled by default, set the following property
langchain4j.in-memory.embedding-store.enabled with value false to disable it.
The store is not attached to AI services automatically. See the Retrieval Augmented Generation section of the AI Service guide to declare a ContentRetriever bean.
19.2 Chroma
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-chroma")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-chroma</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-chroma",
]Example Configuration
langchain4j.chroma.embedding-store.base-url=http://localhost:8000
langchain4j.chroma.embedding-store.collection-name=documents
langchain4j.chroma.embedding-store.api-version=V2langchain4j.chroma.embedding-store.base-url: http://localhost:8000
langchain4j.chroma.embedding-store.collection-name: documents
langchain4j.chroma.embedding-store.api-version: V2"langchain4j.chroma.embedding-store.base-url" = "http://localhost:8000"
"langchain4j.chroma.embedding-store.collection-name" = "documents"
"langchain4j.chroma.embedding-store.api-version" = "V2"langchain4j.chroma.embeddingStore.baseUrl = "http://localhost:8000"
langchain4j.chroma.embeddingStore.collectionName = "documents"
langchain4j.chroma.embeddingStore.apiVersion = "V2"{
"langchain4j.chroma.embedding-store.base-url" = "http://localhost:8000"
"langchain4j.chroma.embedding-store.collection-name" = "documents"
"langchain4j.chroma.embedding-store.api-version" = "V2"
}{
"langchain4j.chroma.embedding-store.base-url": "http://localhost:8000",
"langchain4j.chroma.embedding-store.collection-name": "documents",
"langchain4j.chroma.embedding-store.api-version": "V2"
}19.3 Elastic Search
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-elasticsearch")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-elasticsearch</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-elasticsearch",
]Example Configuration
elasticsearch.httpHosts=http://localhost:9200,http://127.0.0.2:9200
langchain4j.elasticsearch.embedding-stores.default.dimension=384elasticsearch.httpHosts: "http://localhost:9200,http://127.0.0.2:9200"
langchain4j.elasticsearch.embedding-stores.default.dimension: 384"elasticsearch.httpHosts" = "http://localhost:9200,http://127.0.0.2:9200"
"langchain4j.elasticsearch.embedding-stores.default.dimension" = 384elasticsearch.httpHosts = "http://localhost:9200,http://127.0.0.2:9200"
langchain4j.elasticsearch.embeddingStores.default.dimension = 384{
"elasticsearch.httpHosts" = "http://localhost:9200,http://127.0.0.2:9200"
"langchain4j.elasticsearch.embedding-stores.default.dimension" = 384
}{
"elasticsearch.httpHosts": "http://localhost:9200,http://127.0.0.2:9200",
"langchain4j.elasticsearch.embedding-stores.default.dimension": 384
}19.4 Milvus
Add the following dependency to use the Milvus embedding store, based on LangChain4j’s Milvus V2 integration:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-milvus")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-milvus</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-milvus",
]Example Configuration
langchain4j.milvus-v2.embedding-store.uri=http://localhost:19530
langchain4j.milvus-v2.embedding-store.collection-name=documents
langchain4j.milvus-v2.embedding-store.dimension=384
langchain4j.milvus-v2.embedding-store.consistency-level=STRONGlangchain4j:
milvus-v2:
embedding-store:
uri: http://localhost:19530
collection-name: documents
dimension: 384
consistency-level: STRONG[langchain4j.milvus-v2.embedding-store]
uri = "http://localhost:19530"
collection-name = "documents"
dimension = 384
consistency-level = "STRONG"langchain4j {
milvusV2 {
embeddingStore {
uri = "http://localhost:19530"
collectionName = "documents"
dimension = 384
consistencyLevel = "STRONG"
}
}
}{
langchain4j {
milvus-v2 {
embedding-store {
uri = "http://localhost:19530"
collection-name = "documents"
dimension = 384
consistency-level = "STRONG"
}
}
}
}{
"langchain4j": {
"milvus-v2": {
"embedding-store": {
"uri": "http://localhost:19530",
"collection-name": "documents",
"dimension": 384,
"consistency-level": "STRONG"
}
}
}
}The dimension must match the dimension of the embeddings of your embedding model. Milvus is eventually consistent by default: set consistency-level to STRONG to search the embeddings right after they are added.
| Property | Type | Description | Default value |
|---|---|---|---|
|
io.milvus.v2.client.MilvusClientV2 |
||
|
java.lang.String |
||
|
java.lang.Integer |
||
|
java.lang.String |
||
|
java.lang.Integer |
||
|
io.milvus.v2.common.IndexParam$IndexType |
||
|
io.milvus.v2.common.IndexParam$MetricType |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
io.milvus.v2.common.ConsistencyLevel |
||
|
java.lang.Boolean |
||
|
java.lang.Boolean |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
dev.langchain4j.store.embedding.milvus.v2.MilvusV2Ranker |
||
|
io.milvus.v2.common.IndexParam$MetricType |
||
|
io.milvus.v2.common.IndexParam$IndexType |
||
|
dev.langchain4j.store.embedding.milvus.v2.MilvusV2EmbeddingStore$MilvusSparseMode |
||
|
dev.langchain4j.store.embedding.milvus.v2.MilvusV2EmbeddingStore$SearchMode |
||
|
io.milvus.v2.client.MilvusClientV2 |
||
|
java.lang.String |
||
|
java.lang.Integer |
||
|
java.lang.String |
||
|
java.lang.Integer |
||
|
io.milvus.v2.common.IndexParam$IndexType |
||
|
io.milvus.v2.common.IndexParam$MetricType |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
io.milvus.v2.common.ConsistencyLevel |
||
|
java.lang.Boolean |
||
|
java.lang.Boolean |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
dev.langchain4j.store.embedding.milvus.v2.MilvusV2Ranker |
||
|
io.milvus.v2.common.IndexParam$MetricType |
||
|
io.milvus.v2.common.IndexParam$IndexType |
||
|
dev.langchain4j.store.embedding.milvus.v2.MilvusV2EmbeddingStore$MilvusSparseMode |
||
|
dev.langchain4j.store.embedding.milvus.v2.MilvusV2EmbeddingStore$SearchMode |
19.5 MongoDB
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-mongodb-atlas")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-mongodb-atlas</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-mongodb-atlas",
]Configuring a MongoDB server
mongodb.servers.default.uri: mongodb://username:password@localhost:27017/databaseNameExample Configuration
langchain4j.mongodb-atlas.embedding-stores.default.database-name=testdb
langchain4j.mongodb-atlas.embedding-stores.default.collection-name=testcol
langchain4j.mongodb-atlas.embedding-stores.default.index-name=testindexlangchain4j.mongodb-atlas.embedding-stores.default.database-name: testdb
langchain4j.mongodb-atlas.embedding-stores.default.collection-name: testcol
langchain4j.mongodb-atlas.embedding-stores.default.index-name: testindex"langchain4j.mongodb-atlas.embedding-stores.default.database-name" = "testdb"
"langchain4j.mongodb-atlas.embedding-stores.default.collection-name" = "testcol"
"langchain4j.mongodb-atlas.embedding-stores.default.index-name" = "testindex"langchain4j.mongodbAtlas.embeddingStores.default.databaseName = "testdb"
langchain4j.mongodbAtlas.embeddingStores.default.collectionName = "testcol"
langchain4j.mongodbAtlas.embeddingStores.default.indexName = "testindex"{
"langchain4j.mongodb-atlas.embedding-stores.default.database-name" = "testdb"
"langchain4j.mongodb-atlas.embedding-stores.default.collection-name" = "testcol"
"langchain4j.mongodb-atlas.embedding-stores.default.index-name" = "testindex"
}{
"langchain4j.mongodb-atlas.embedding-stores.default.database-name": "testdb",
"langchain4j.mongodb-atlas.embedding-stores.default.collection-name": "testcol",
"langchain4j.mongodb-atlas.embedding-stores.default.index-name": "testindex"
}19.6 Neo4j
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-neo4j")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-neo4j</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-neo4j",
]Example Configuration
neo4j.uri=bolt://localhost
langchain4j.neo4j.embedding-stores.default.dimension=384neo4j.uri: bolt://localhost
langchain4j.neo4j.embedding-stores.default.dimension: 384"neo4j.uri" = "bolt://localhost"
"langchain4j.neo4j.embedding-stores.default.dimension" = 384neo4j.uri = "bolt://localhost"
langchain4j.neo4j.embeddingStores.default.dimension = 384{
"neo4j.uri" = "bolt://localhost"
"langchain4j.neo4j.embedding-stores.default.dimension" = 384
}{
"neo4j.uri": "bolt://localhost",
"langchain4j.neo4j.embedding-stores.default.dimension": 384
}19.7 Oracle
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-oracle")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-oracle</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-oracle",
]Then add one of the supported JDBC connection pools, for example Hikari:
runtimeOnly("io.micronaut.sql:micronaut-jdbc-hikari")<dependency>
<groupId>io.micronaut.sql</groupId>
<artifactId>micronaut-jdbc-hikari</artifactId>
<scope>runtime</scope>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.sql:micronaut-jdbc-hikari",
]Example Configuration
datasources.default.dialect=oracle
langchain4j.oracle.embedding-stores.default.table=test
langchain4j.oracle.embedding-stores.default.table.create-option=create_if_not_existsdatasources.default.dialect: oracle
langchain4j.oracle.embedding-stores.default.table: test
langchain4j.oracle.embedding-stores.default.table.create-option: create_if_not_exists"datasources.default.dialect" = "oracle"
"langchain4j.oracle.embedding-stores.default.table" = "test"
"langchain4j.oracle.embedding-stores.default.table.create-option" = "create_if_not_exists"datasources.default.dialect = "oracle"
langchain4j.oracle.embeddingStores.default.table = "test"
langchain4j.oracle.embeddingStores.default.table.createOption = "create_if_not_exists"{
"datasources.default.dialect" = "oracle"
"langchain4j.oracle.embedding-stores.default.table" = "test"
"langchain4j.oracle.embedding-stores.default.table.create-option" = "create_if_not_exists"
}{
"datasources.default.dialect": "oracle",
"langchain4j.oracle.embedding-stores.default.table": "test",
"langchain4j.oracle.embedding-stores.default.table.create-option": "create_if_not_exists"
}19.8 Open Search
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-opensearch")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-opensearch</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-opensearch",
]Example Configuration
micronaut.opensearch.rest-client.http-hosts=http://localhost:9200,http://127.0.0.2:9200
langchain4j.opensearch.embedding-stores.default.dimension=384micronaut.opensearch.rest-client.http-hosts: "http://localhost:9200,http://127.0.0.2:9200"
langchain4j.opensearch.embedding-stores.default.dimension: 384"micronaut.opensearch.rest-client.http-hosts" = "http://localhost:9200,http://127.0.0.2:9200"
"langchain4j.opensearch.embedding-stores.default.dimension" = 384micronaut.opensearch.restClient.httpHosts = "http://localhost:9200,http://127.0.0.2:9200"
langchain4j.opensearch.embeddingStores.default.dimension = 384{
"micronaut.opensearch.rest-client.http-hosts" = "http://localhost:9200,http://127.0.0.2:9200"
"langchain4j.opensearch.embedding-stores.default.dimension" = 384
}{
"micronaut.opensearch.rest-client.http-hosts": "http://localhost:9200,http://127.0.0.2:9200",
"langchain4j.opensearch.embedding-stores.default.dimension": 384
}19.9 PGVector
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-pgvector")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-pgvector</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-pgvector",
]Then add one of the supported JDBC connection pools, for example Hikari:
runtimeOnly("io.micronaut.sql:micronaut-jdbc-hikari")<dependency>
<groupId>io.micronaut.sql</groupId>
<artifactId>micronaut-jdbc-hikari</artifactId>
<scope>runtime</scope>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.sql:micronaut-jdbc-hikari",
]Example Configuration
datasources.default.dialect=postgres
langchain4j.pgvector.embedding-stores.default.table=mytable
langchain4j.pgvector.embedding-stores.default.dimension=384
test-resources.containers.postgres.image-name=pgvector/pgvector:pg16datasources.default.dialect: postgres
langchain4j.pgvector.embedding-stores.default.table: "mytable"
langchain4j.pgvector.embedding-stores.default.dimension: 384
# Add this if you plan to use testresources
test-resources.containers.postgres.image-name: pgvector/pgvector:pg16"datasources.default.dialect" = "postgres"
"langchain4j.pgvector.embedding-stores.default.table" = "mytable"
"langchain4j.pgvector.embedding-stores.default.dimension" = 384
"test-resources.containers.postgres.image-name" = "pgvector/pgvector:pg16"datasources.default.dialect = "postgres"
langchain4j.pgvector.embeddingStores.default.table = "mytable"
langchain4j.pgvector.embeddingStores.default.dimension = 384
testResources.containers.postgres.imageName = "pgvector/pgvector:pg16"{
"datasources.default.dialect" = "postgres"
"langchain4j.pgvector.embedding-stores.default.table" = "mytable"
"langchain4j.pgvector.embedding-stores.default.dimension" = 384
"test-resources.containers.postgres.image-name" = "pgvector/pgvector:pg16"
}{
"datasources.default.dialect": "postgres",
"langchain4j.pgvector.embedding-stores.default.table": "mytable",
"langchain4j.pgvector.embedding-stores.default.dimension": 384,
"test-resources.containers.postgres.image-name": "pgvector/pgvector:pg16"
}19.10 Redis
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-redis")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-redis</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-redis",
]Example Configuration
langchain4j.redis.embedding-store.host=localhost
langchain4j.redis.embedding-store.port=6379
langchain4j.redis.embedding-stores.default.dimension=384langchain4j.redis.embedding-store.host: localhost
langchain4j.redis.embedding-store.port: 6379
langchain4j.redis.embedding-stores.default.dimension: 384"langchain4j.redis.embedding-store.host" = "localhost"
"langchain4j.redis.embedding-store.port" = 6379
"langchain4j.redis.embedding-stores.default.dimension" = 384langchain4j.redis.embeddingStore.host = "localhost"
langchain4j.redis.embeddingStore.port = 6379
langchain4j.redis.embeddingStores.default.dimension = 384{
"langchain4j.redis.embedding-store.host" = "localhost"
"langchain4j.redis.embedding-store.port" = 6379
"langchain4j.redis.embedding-stores.default.dimension" = 384
}{
"langchain4j.redis.embedding-store.host": "localhost",
"langchain4j.redis.embedding-store.port": 6379,
"langchain4j.redis.embedding-stores.default.dimension": 384
}19.11 Qdrant
Add the following dependency:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-qdrant")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-qdrant</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-qdrant",
]To use Testcontainers & Test Resources add the following dependency:
testResourcesService("io.micronaut.langchain4j:micronaut-langchain4j-qdrant-testresource")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-qdrant-testresource</artifactId>
<scope>testResourcesService</scope>
</dependency>[tool.pyronaut.dependencies]
test = [
"io.micronaut.langchain4j:micronaut-langchain4j-qdrant-testresource",
]Example Configuration
langchain4j.qdrant.embedding-store.host=localhost
langchain4j.qdrant.embedding-store.port=6334
langchain4j.qdrant.embedding-store.collection-name=mycollection# Omitt the following 2 properties if you use Test resources
langchain4j.qdrant.embedding-store.host: localhost
langchain4j.qdrant.embedding-store.port: 6334
# Minimal configuration required for Test resources
langchain4j.qdrant.embedding-store.collection-name: mycollection"langchain4j.qdrant.embedding-store.host" = "localhost"
"langchain4j.qdrant.embedding-store.port" = 6334
"langchain4j.qdrant.embedding-store.collection-name" = "mycollection"langchain4j.qdrant.embeddingStore.host = "localhost"
langchain4j.qdrant.embeddingStore.port = 6334
langchain4j.qdrant.embeddingStore.collectionName = "mycollection"{
"langchain4j.qdrant.embedding-store.host" = "localhost"
"langchain4j.qdrant.embedding-store.port" = 6334
"langchain4j.qdrant.embedding-store.collection-name" = "mycollection"
}{
"langchain4j.qdrant.embedding-store.host": "localhost",
"langchain4j.qdrant.embedding-store.port": 6334,
"langchain4j.qdrant.embedding-store.collection-name": "mycollection"
}19.12 Weaviate
Add the following dependency to use the Weaviate embedding store:
implementation("io.micronaut.langchain4j:micronaut-langchain4j-store-weaviate")<dependency>
<groupId>io.micronaut.langchain4j</groupId>
<artifactId>micronaut-langchain4j-store-weaviate</artifactId>
</dependency>[tool.pyronaut.dependencies]
runtime = [
"io.micronaut.langchain4j:micronaut-langchain4j-store-weaviate",
]Example Configuration
langchain4j.weaviate.embedding-store.scheme=https
langchain4j.weaviate.embedding-store.host=my-cluster.weaviate.network
langchain4j.weaviate.embedding-store.api-key=${WEAVIATE_API_KEY}
langchain4j.weaviate.embedding-store.object-class=Documentlangchain4j:
weaviate:
embedding-store:
scheme: https
host: my-cluster.weaviate.network
api-key: ${WEAVIATE_API_KEY}
object-class: Document[langchain4j.weaviate.embedding-store]
scheme = "https"
host = "my-cluster.weaviate.network"
api-key = "${WEAVIATE_API_KEY}"
object-class = "Document"langchain4j {
weaviate {
embeddingStore {
scheme = "https"
host = "my-cluster.weaviate.network"
apiKey = "${WEAVIATE_API_KEY}"
objectClass = "Document"
}
}
}{
langchain4j {
weaviate {
embedding-store {
scheme = "https"
host = "my-cluster.weaviate.network"
api-key = "${WEAVIATE_API_KEY}"
object-class = "Document"
}
}
}
}{
"langchain4j": {
"weaviate": {
"embedding-store": {
"scheme": "https",
"host": "my-cluster.weaviate.network",
"api-key": "${WEAVIATE_API_KEY}",
"object-class": "Document"
}
}
}
}| Property | Type | Description | Default value |
|---|---|---|---|
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.Integer |
||
|
java.lang.Boolean |
||
|
java.lang.Boolean |
||
|
java.lang.Integer |
||
|
java.lang.String |
||
|
java.lang.Boolean |
||
|
java.lang.String |
||
|
java.util.Collection |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.String |
||
|
java.lang.Integer |
||
|
java.lang.Boolean |
||
|
java.lang.Boolean |
||
|
java.lang.Integer |
||
|
java.lang.String |
||
|
java.lang.Boolean |
||
|
java.lang.String |
||
|
java.util.Collection |
||
|
java.lang.String |
||
|
java.lang.String |
20 Breaking Changes
This section documents breaking changes between Micronaut LangChain4j versions.
2.4.0
Updated default model names
The default model names of several providers referenced models that are retired or about to be retired. They now default to current models:
| Provider | Model type | Previous default | New default |
|---|---|---|---|
Anthropic |
Chat |
|
|
Mistral AI |
Chat |
|
|
OpenAI |
Chat |
|
|
OpenAI |
Embedding |
|
|
OpenAI |
Image |
|
|
Vertex AI Gemini |
Chat |
|
|
Set the model-name property of the model to keep using a previous model. Note that embeddings produced by different embedding models are not compatible, so applications that stored embeddings created with the previous OpenAI default must either set model-name: text-embedding-ada-002 or re-embed their content.
Bedrock model property removed
The langchain4j.bedrock.model property was never applied to the Bedrock models. It has been removed. Configure the model with langchain4j.bedrock.chat-model.model-id (and the equivalent property of the streaming chat and embedding models) instead.
Chat memory of the AI services
The default chat memory provider of the AI services kept the conversations in memory, even when a persistent chat memory store (Redis, Neo4j, Cassandra, Oracle) was configured. It now keeps them in the persistent chat memory store if the application has exactly one. Set langchain4j.chat-memory.store: inMemory to keep them in memory.
Hugging Face uses the Micronaut HTTP client
The Hugging Face models use the Micronaut HTTP client instead of LangChain4j’s JDK HTTP client, like the OpenAI, Anthropic, Ollama, Mistral AI and Google AI Gemini models. Add a Micronaut HTTP client implementation, such as io.micronaut:micronaut-http-client, to the runtime classpath.
LangChain4j Community 1.22
The LangChain4j community modules (Redis, Neo4j, OCI GenAI) have been upgraded from 1.16.0-beta26 to 1.22.0-beta32, aligned with the LangChain4j 1.22.0 BOM. Refer to the LangChain4j Community release notes for changes.
Nullness annotations
Every package of Micronaut LangChain4j is now annotated with JSpecify @NullMarked: the types of the API are non-null, unless annotated with @Nullable. Kotlin reads these annotations, so Kotlin code sees non-null and nullable types where it previously saw platform types. For example, the getters of the optional properties of PgVectorEmbeddingStoreConfig now return nullable types.
21 Repository
You can find the source code of this project in this repository:
22 Release History
For this project, you can find a list of releases (with release notes) here: