Architecture

Understand how Avaliar processes and analyzes your LLM traces

Overview

Avaliar captures, stores, and analyzes LLM traces through two integration paths. Both paths converge on the same detection pipeline so that every trace — regardless of how it was submitted — is analyzed consistently and appears in the same dashboard.

Data Flow

SDK Path

1. Your App Calls a @traceable Function

   A function decorated with @traceable executes in your application. The SDK automatically captures inputs, outputs, timing, token counts, and any metadata you pass in.

2. Trace Submitted to Avaliar

   The SDK submits the trace to Avaliar asynchronously in the background. Tracing never adds latency to your application — if Avaliar is unreachable, your app continues running normally.

3. Detection Runs

   If cloud detection is enabled, Avaliar analyzes the trace for safety issues — prompt injection, jailbreaks, toxicity, PII, bias, and hallucination. Detection runs asynchronously after the trace is stored.

4. Results Appear in the Dashboard

   The trace, along with any detected issues, becomes available in the Trace Explorer. Detection results typically appear within a few seconds.

Proxy Path

1. Request Sent Through Avaliar Proxy

   Your application sends its LLM request to the Avaliar Proxy instead of directly to the provider. No code changes are needed beyond updating the base URL.

2. API Key Validated

   The proxy validates your API key and checks that your account is authorized to use the target model and provider.

3. Synchronous Prompt Inspection

   Before the request ever reaches the LLM, the proxy runs prompt injection detection on the input. If a threat is detected, the request is blocked and never forwarded to the provider.

4. Request Forwarded to LLM Provider

   If the prompt is clean, the proxy forwards the request to the target provider and captures the full response including latency and token counts.

5. Detection Runs on the Response

   The response is analyzed asynchronously for all remaining detectors — toxicity, PII, bias, hallucination, and more.

6. Results Appear in the Dashboard

   The full trace and any detected issues appear in the Trace Explorer, typically within a few seconds of the request completing.

Async Detection

Detection runs asynchronously after a trace is stored. This means:

• Your application is never blocked waiting for detection to complete
• Traces appear immediately in the Trace Explorer, with detection results added shortly after
• Prompt inspection via the Proxy is the one exception — it runs synchronously before forwarding, because blocking a malicious prompt requires an answer before the LLM call proceeds

Detection results typically appear within a few seconds. On the Trace Explorer, a trace that has not yet been analyzed will show a “Pending” status until detection completes.

Authentication

Avaliar uses two authentication mechanisms depending on how you’re accessing the platform.

Method	Use Case	How to use it
Session	Dashboard, analytics, settings, team management	Log in at app.avaliar.ai
API Key	SDK and Proxy integrations	Set AVALIAR_API_KEY or pass in the X-API-Key header

API keys are scoped — a key created for the SDK cannot be used for the Proxy, and vice versa. You can create and manage keys from app.avaliar.ai/api-keys.

Next Steps

Detection Overview

Learn how the built-in detectors analyze your traces.

API Keys

Create and manage the API keys that authenticate your integrations.