Skip to content

Conversation

@cescoffier
Copy link
Collaborator

Introduces GuardrailMetricsObserver to collect guardrail metrics with enhanced tags including aiservice name, operation, guardrail class, type (input/output), and outcome (success/failure/reprompt).

Deprecates the previous annotation-based metrics collection in GuardrailObservabilityProcessor.

@cescoffier cescoffier requested a review from a team as a code owner November 28, 2025 15:12
@cescoffier cescoffier added the enhancement New feature or request label Nov 28, 2025
Copy link
Collaborator

@geoand geoand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💪🏽

  Introduces GuardrailMetricsObserver to collect guardrail metrics with enhanced
  tags including aiservice name, operation, guardrail class, type (input/output),
  and outcome (success/failure/reprompt).

  Deprecates the previous annotation-based metrics collection in GuardrailObservabilityProcessor.
@cescoffier
Copy link
Collaborator Author

I've reorganized the commits.

@quarkus-bot
Copy link

quarkus-bot bot commented Nov 28, 2025

Status for workflow Build (on pull request)

This is the status report for running Build (on pull request) on commit b79d501.

Warning

Unable to include the stracktraces as the report was too long. See annotations below for the details.

Warning

Unable to include the failure links as the report was too long. See annotations below for the details.

Failing Jobs

Status Name Step Failures Logs Raw logs
JVM tests - agentic - Java 17 Run tests of agentic with JDK 17 Failures Logs Raw logs
JVM tests - agentic - Java 21 Run tests of agentic with JDK 21 Failures Logs Raw logs
JVM tests - agentic - Java 24 Run tests of agentic with JDK 24 Failures Logs Raw logs
JVM tests - core - Java 17 Run tests of core with JDK 17 Failures Logs Raw logs
JVM tests - core - Java 21 Run tests of core with JDK 21 Failures Logs Raw logs
JVM tests - core - Java 24 Run tests of core with JDK 24 Failures Logs Raw logs
JVM tests - docs - Java 17 Run tests of docs with JDK 17 Failures Logs Raw logs
JVM tests - docs - Java 21 Run tests of docs with JDK 21 Failures Logs Raw logs
JVM tests - docs - Java 24 Run tests of docs with JDK 24 Failures Logs Raw logs
JVM tests - embedding-stores - Java 17 Run tests of embedding-stores with JDK 17 Failures Logs Raw logs
JVM tests - embedding-stores - Java 21 Run tests of embedding-stores with JDK 21 Failures Logs Raw logs
JVM tests - embedding-stores - Java 24 Run tests of embedding-stores with JDK 24 Failures Logs Raw logs
JVM tests - integration-tests - Java 17 Run tests of integration-tests with JDK 17 Failures Logs Raw logs
JVM tests - integration-tests - Java 21 Run tests of integration-tests with JDK 21 Failures Logs Raw logs
JVM tests - integration-tests - Java 24 Run tests of integration-tests with JDK 24 Failures Logs Raw logs
JVM tests - mcp - Java 17 Run tests of mcp with JDK 17 Failures Logs Raw logs
JVM tests - mcp - Java 21 Run tests of mcp with JDK 21 Failures Logs Raw logs
JVM tests - mcp - Java 24 Run tests of mcp with JDK 24 Failures Logs Raw logs
JVM tests - memory-stores - Java 17 Run tests of memory-stores with JDK 17 Failures Logs Raw logs
JVM tests - memory-stores - Java 21 Run tests of memory-stores with JDK 21 Failures Logs Raw logs
JVM tests - memory-stores - Java 24 Run tests of memory-stores with JDK 24 Failures Logs Raw logs
JVM tests - model-providers - Java 17 Run tests of model-providers with JDK 17 Failures Logs Raw logs
JVM tests - model-providers - Java 21 Run tests of model-providers with JDK 21 Failures Logs Raw logs
JVM tests - model-providers - Java 24 Run tests of model-providers with JDK 24 Failures Logs Raw logs
JVM tests - rag - Java 17 Run tests of rag with JDK 17 Failures Logs Raw logs
JVM tests - rag - Java 21 Run tests of rag with JDK 21 Failures Logs Raw logs
JVM tests - rag - Java 24 Run tests of rag with JDK 24 Failures Logs Raw logs
JVM tests - tools - Java 17 Run tests of tools with JDK 17 Failures Logs Raw logs
JVM tests - tools - Java 21 Run tests of tools with JDK 21 Failures Logs Raw logs
JVM tests - tools - Java 24 Run tests of tools with JDK 24 Failures Logs Raw logs
Native tests - azure-openai Run integration tests azure-openai ⚠️ Check → Logs Raw logs
Native tests - devui Run integration tests devui Failures Logs Raw logs
Native tests - devui-multiple-embedding-models Run integration tests devui-multiple-embedding-models Failures Logs Raw logs
Native tests - hugging-face Run integration tests hugging-face ⚠️ Check → Logs Raw logs
Native tests - mcp Run integration tests mcp ⚠️ Check → Logs Raw logs
Native tests - multiple-providers Run integration tests multiple-providers Failures Logs Raw logs
Native tests - ollama Run integration tests ollama ⚠️ Check → Logs Raw logs
Native tests - rag Run integration tests rag ⚠️ Check → Logs Raw logs
Native tests - rag-pgvector Run integration tests rag-pgvector ⚠️ Check → Logs Raw logs
Native tests - rag-pgvector-flyway Run integration tests rag-pgvector-flyway ⚠️ Check → Logs Raw logs
Native tests - secure-mcp Run integration tests secure-mcp ⚠️ Check → Logs Raw logs
Native tests - simple-ollama Run integration tests simple-ollama ⚠️ Check → Logs Raw logs
Native tests - tools Run integration tests tools ⚠️ Check → Logs Raw logs
In process embedding model tests - embed-all-minilm-l6-v2 Run integration tests embed-all-minilm-l6-v2 ⚠️ Check → Logs Raw logs
In process embedding model tests - embed-all-minilm-l6-v2-q Run integration tests embed-all-minilm-l6-v2-q ⚠️ Check → Logs Raw logs
In process embedding model tests - embed-bge-small-en Run integration tests embed-bge-small-en ⚠️ Check → Logs Raw logs
In process embedding model tests - embed-bge-small-en-q Run integration tests embed-bge-small-en-q ⚠️ Check → Logs Raw logs
In process embedding model tests - embed-bge-small-en-v15 Run integration tests embed-bge-small-en-v15 ⚠️ Check → Logs Raw logs
In process embedding model tests - embed-bge-small-en-v15-and-ollama Run integration tests embed-bge-small-en-v15-and-ollama Failures Logs Raw logs
In process embedding model tests - embed-bge-small-en-v15-q Run integration tests embed-bge-small-en-v15-q ⚠️ Check → Logs Raw logs
In process embedding model tests - embed-e5-small-v2 Run integration tests embed-e5-small-v2 ⚠️ Check → Logs Raw logs
In process embedding model tests - embed-e5-small-v2-q Run integration tests embed-e5-small-v2-q ⚠️ Check → Logs Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ JVM tests - agentic - Java 17 #

📦 agentic/deployment

io.quarkiverse.langchain4j.agentic.deployment.AgentBeanSmokeTest.

io.quarkiverse.langchain4j.agentic.deployment.WorkflowTest.

io.quarkiverse.langchain4j.agentic.deployment.validation.SupervisorAgentShouldWorkTest.


⚙️ JVM tests - agentic - Java 21 #

📦 agentic/deployment

io.quarkiverse.langchain4j.agentic.deployment.AgentBeanSmokeTest.

io.quarkiverse.langchain4j.agentic.deployment.WorkflowTest.

io.quarkiverse.langchain4j.agentic.deployment.validation.SupervisorAgentShouldWorkTest.


⚙️ JVM tests - agentic - Java 24 #

📦 agentic/deployment

io.quarkiverse.langchain4j.agentic.deployment.AgentBeanSmokeTest.

io.quarkiverse.langchain4j.agentic.deployment.WorkflowTest.

io.quarkiverse.langchain4j.agentic.deployment.validation.SupervisorAgentShouldWorkTest.


⚙️ JVM tests - core - Java 17 #

📦 core/opentelemetry-tests

io.quarkiverse.langchain4j.opentelemetry.test.ListenersProcessorOnlySpanChatModelListenerTest.

io.quarkiverse.langchain4j.opentelemetry.test.ListenersProcessorSingleChatModelSpanContributorTest.

io.quarkiverse.langchain4j.opentelemetry.test.ListenersProcessorTwoChatModelSpanContributorsTest.


⚙️ JVM tests - core - Java 21 #

📦 core/opentelemetry-tests

io.quarkiverse.langchain4j.opentelemetry.test.ListenersProcessorOnlySpanChatModelListenerTest.

io.quarkiverse.langchain4j.opentelemetry.test.ListenersProcessorSingleChatModelSpanContributorTest.

io.quarkiverse.langchain4j.opentelemetry.test.ListenersProcessorTwoChatModelSpanContributorsTest.


⚙️ JVM tests - core - Java 24 #

📦 core/opentelemetry-tests

io.quarkiverse.langchain4j.opentelemetry.test.ListenersProcessorOnlySpanChatModelListenerTest.

io.quarkiverse.langchain4j.opentelemetry.test.ListenersProcessorSingleChatModelSpanContributorTest.

io.quarkiverse.langchain4j.opentelemetry.test.ListenersProcessorTwoChatModelSpanContributorsTest.


⚙️ JVM tests - docs - Java 17 #

📦 docs

io.quarkiverse.langchain4j.docs.it.AntoraTest.antoraSite


⚙️ JVM tests - docs - Java 21 #

📦 docs

io.quarkiverse.langchain4j.docs.it.AntoraTest.antoraSite


⚙️ JVM tests - docs - Java 24 #

📦 docs

io.quarkiverse.langchain4j.docs.it.AntoraTest.antoraSite


⚙️ JVM tests - embedding-stores - Java 17 #

📦 embedding-stores/chroma/deployment

io.quarkiverse.langchain4j.chroma.deployment.ChromaEmbeddingStoreV1Test.

io.quarkiverse.langchain4j.chroma.deployment.ChromaEmbeddingStoreV2Test.


⚙️ JVM tests - embedding-stores - Java 21 #

📦 embedding-stores/chroma/deployment

io.quarkiverse.langchain4j.chroma.deployment.ChromaEmbeddingStoreV1Test.

io.quarkiverse.langchain4j.chroma.deployment.ChromaEmbeddingStoreV2Test.


⚙️ JVM tests - embedding-stores - Java 24 #

📦 embedding-stores/chroma/deployment

io.quarkiverse.langchain4j.chroma.deployment.ChromaEmbeddingStoreV1Test.

io.quarkiverse.langchain4j.chroma.deployment.ChromaEmbeddingStoreV2Test.


⚙️ JVM tests - integration-tests - Java 17 #

📦 integration-tests/openai

org.acme.example.openai.aiservices.AssistantResourceWithGuardrailsAndObservabilityTest.guardrailMetricsAvailable line 49


⚙️ JVM tests - integration-tests - Java 21 #

📦 integration-tests/openai

org.acme.example.openai.aiservices.AssistantResourceWithGuardrailsAndObservabilityTest.guardrailMetricsAvailable line 49


⚙️ JVM tests - integration-tests - Java 24 #

📦 integration-tests/openai

org.acme.example.openai.aiservices.AssistantResourceWithGuardrailsAndObservabilityTest.guardrailMetricsAvailable line 49


⚙️ JVM tests - mcp - Java 17 #

📦 mcp/deployment

io.quarkiverse.langchain4j.mcp.test.AgentMcpClientTest.

io.quarkiverse.langchain4j.mcp.test.McpAccessTokenTest.

io.quarkiverse.langchain4j.mcp.test.McpClaudeConfigTest.

io.quarkiverse.langchain4j.mcp.test.McpClientAndToolProviderCDITest.

io.quarkiverse.langchain4j.mcp.test.McpHealthCheckTest.

io.quarkiverse.langchain4j.mcp.test.McpOverHttpTransportTest.

io.quarkiverse.langchain4j.mcp.test.McpWithResourcesAsToolsTest.

io.quarkiverse.langchain4j.mcp.test.MultipleMcpClientsTest.

io.quarkiverse.langchain4j.mcp.test.NoAutomaticToolProviderTest.

io.quarkiverse.langchain4j.mcp.test.registryclient.McpRegistryClientTest.


⚙️ JVM tests - mcp - Java 21 #

📦 mcp/deployment

io.quarkiverse.langchain4j.mcp.test.AgentMcpClientTest.

io.quarkiverse.langchain4j.mcp.test.McpAccessTokenTest.

io.quarkiverse.langchain4j.mcp.test.McpClaudeConfigTest.

io.quarkiverse.langchain4j.mcp.test.McpClientAndToolProviderCDITest.

io.quarkiverse.langchain4j.mcp.test.McpHealthCheckTest.

io.quarkiverse.langchain4j.mcp.test.McpOverHttpTransportTest.

io.quarkiverse.langchain4j.mcp.test.McpWithResourcesAsToolsTest.

io.quarkiverse.langchain4j.mcp.test.MultipleMcpClientsTest.

io.quarkiverse.langchain4j.mcp.test.NoAutomaticToolProviderTest.

io.quarkiverse.langchain4j.mcp.test.registryclient.McpRegistryClientTest.


⚙️ JVM tests - mcp - Java 24 #

📦 mcp/deployment

io.quarkiverse.langchain4j.mcp.test.AgentMcpClientTest.

io.quarkiverse.langchain4j.mcp.test.McpAccessTokenTest.

io.quarkiverse.langchain4j.mcp.test.McpClaudeConfigTest.

io.quarkiverse.langchain4j.mcp.test.McpClientAndToolProviderCDITest.

io.quarkiverse.langchain4j.mcp.test.McpHealthCheckTest.

io.quarkiverse.langchain4j.mcp.test.McpOverHttpTransportTest.

io.quarkiverse.langchain4j.mcp.test.McpWithResourcesAsToolsTest.

io.quarkiverse.langchain4j.mcp.test.MultipleMcpClientsTest.

io.quarkiverse.langchain4j.mcp.test.NoAutomaticToolProviderTest.

io.quarkiverse.langchain4j.mcp.test.registryclient.McpRegistryClientTest.


⚙️ JVM tests - memory-stores - Java 17 #

📦 memory-stores/memory-store-redis/deployment

io.quarkiverse.langchain4j.memorystore.redis.test.RedisMemoryToolBoxTest.


⚙️ JVM tests - memory-stores - Java 21 #

📦 memory-stores/memory-store-redis/deployment

io.quarkiverse.langchain4j.memorystore.redis.test.RedisMemoryToolBoxTest.


⚙️ JVM tests - memory-stores - Java 24 #

📦 memory-stores/memory-store-redis/deployment

io.quarkiverse.langchain4j.memorystore.redis.test.RedisMemoryToolBoxTest.


⚙️ JVM tests - model-providers - Java 17 #

📦 model-providers/anthropic/deployment

io.quarkiverse.langchain4j.anthropic.deployment.AnthropicChatLanguageModelSmokeTest.

io.quarkiverse.langchain4j.anthropic.deployment.AnthropicDisabledLanguageModelSmokeTest.

io.quarkiverse.langchain4j.anthropic.deployment.AnthropicStreamingChatLanguageModelSmokeTest.


⚙️ JVM tests - model-providers - Java 21 #

📦 model-providers/anthropic/deployment

io.quarkiverse.langchain4j.anthropic.deployment.AnthropicChatLanguageModelSmokeTest.

io.quarkiverse.langchain4j.anthropic.deployment.AnthropicDisabledLanguageModelSmokeTest.

io.quarkiverse.langchain4j.anthropic.deployment.AnthropicStreamingChatLanguageModelSmokeTest.


⚙️ JVM tests - model-providers - Java 24 #

📦 model-providers/anthropic/deployment

io.quarkiverse.langchain4j.anthropic.deployment.AnthropicChatLanguageModelSmokeTest.

io.quarkiverse.langchain4j.anthropic.deployment.AnthropicDisabledLanguageModelSmokeTest.

io.quarkiverse.langchain4j.anthropic.deployment.AnthropicStreamingChatLanguageModelSmokeTest.


⚙️ JVM tests - rag - Java 17 #

📦 rag/easy-rag/deployment

io.quarkiverse.langchain4j.test.EasyRagManualTriggerOnClasspathTest.

io.quarkiverse.langchain4j.test.EasyRagManualTriggerTest.

io.quarkiverse.langchain4j.test.EasyRagNotRecursiveOnClasspathTest. line 50

io.quarkiverse.langchain4j.test.EasyRagNotRecursiveTest. line 46

io.quarkiverse.langchain4j.test.EasyRagOnClasspathTest. line 60

io.quarkiverse.langchain4j.test.EasyRagPathMatcherOnClasspathTest. line 51

io.quarkiverse.langchain4j.test.EasyRagPathMatcherTest. line 45

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsAlreadyExistTest. line 53

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsDontAlreadyExistTest.

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsFileNotSetTest.

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsOnClasspathAlreadyExistTest. line 55

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsOnClasspathDontAlreadyExistTest. line 38

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsOnClasspathFileNotSetTest. line 30

io.quarkiverse.langchain4j.test.EasyRagTest. line 42

io.quarkiverse.langchain4j.test.EasyRagWithExplicitAugmentorOnClasspathTest. line 50

io.quarkiverse.langchain4j.test.EasyRagWithExplicitAugmentorTest. line 46


⚙️ JVM tests - rag - Java 21 #

📦 rag/easy-rag/deployment

io.quarkiverse.langchain4j.test.EasyRagManualTriggerOnClasspathTest.

io.quarkiverse.langchain4j.test.EasyRagManualTriggerTest.

io.quarkiverse.langchain4j.test.EasyRagNotRecursiveOnClasspathTest. line 50

io.quarkiverse.langchain4j.test.EasyRagNotRecursiveTest. line 46

io.quarkiverse.langchain4j.test.EasyRagOnClasspathTest. line 60

io.quarkiverse.langchain4j.test.EasyRagPathMatcherOnClasspathTest. line 51

io.quarkiverse.langchain4j.test.EasyRagPathMatcherTest. line 45

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsAlreadyExistTest. line 53

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsDontAlreadyExistTest.

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsFileNotSetTest.

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsOnClasspathAlreadyExistTest. line 55

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsOnClasspathDontAlreadyExistTest. line 38

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsOnClasspathFileNotSetTest. line 30

io.quarkiverse.langchain4j.test.EasyRagTest. line 42

io.quarkiverse.langchain4j.test.EasyRagWithExplicitAugmentorOnClasspathTest. line 50

io.quarkiverse.langchain4j.test.EasyRagWithExplicitAugmentorTest. line 46


⚙️ JVM tests - rag - Java 24 #

📦 rag/easy-rag/deployment

io.quarkiverse.langchain4j.test.EasyRagManualTriggerOnClasspathTest.

io.quarkiverse.langchain4j.test.EasyRagManualTriggerTest.

io.quarkiverse.langchain4j.test.EasyRagNotRecursiveOnClasspathTest. line 50

io.quarkiverse.langchain4j.test.EasyRagNotRecursiveTest. line 46

io.quarkiverse.langchain4j.test.EasyRagOnClasspathTest. line 60

io.quarkiverse.langchain4j.test.EasyRagPathMatcherOnClasspathTest. line 51

io.quarkiverse.langchain4j.test.EasyRagPathMatcherTest. line 45

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsAlreadyExistTest. line 53

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsDontAlreadyExistTest.

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsFileNotSetTest.

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsOnClasspathAlreadyExistTest. line 55

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsOnClasspathDontAlreadyExistTest. line 38

io.quarkiverse.langchain4j.test.EasyRagReuseEmbeddingsOnClasspathFileNotSetTest. line 30

io.quarkiverse.langchain4j.test.EasyRagTest. line 42

io.quarkiverse.langchain4j.test.EasyRagWithExplicitAugmentorOnClasspathTest. line 50

io.quarkiverse.langchain4j.test.EasyRagWithExplicitAugmentorTest. line 46


⚙️ JVM tests - tools - Java 17 #

📦 tools/tavily/deployment

io.quarkiverse.langchain4j.tavily.test.TavilyTest.


⚙️ JVM tests - tools - Java 21 #

📦 tools/tavily/deployment

io.quarkiverse.langchain4j.tavily.test.TavilyTest.


⚙️ JVM tests - tools - Java 24 #

📦 tools/tavily/deployment

io.quarkiverse.langchain4j.tavily.test.TavilyTest.


⚙️ Native tests - devui #

📦 integration-tests/devui

devui.LangChain4jDevUIJsonRpcTest.testAddAndSearchEmbedding


⚙️ Native tests - devui-multiple-embedding-models #

📦 integration-tests/devui-multiple-embedding-models

io.quarkiverse.langchain4j.test.devui.MultipleEmbeddingModelsDevUITest.test


⚙️ Native tests - multiple-providers #

📦 integration-tests/multiple-providers

org.acme.example.multiple.MultipleChatProvidersTest.seventhNamedModel


⚙️ In process embedding model tests - embed-bge-small-en-v15-and-ollama #

📦 integration-tests/in-process-embedding-models/embed-bge-small-en-v15-and-ollama

org.acme.test.ResourceTest.test

@cescoffier
Copy link
Collaborator Author

Hum, I need to implement some sort of conditional inclusion of the bean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Development

Successfully merging this pull request may close these issues.

2 participants