Proxy Overview

Route LLM API calls through the Avaliar Proxy for automatic monitoring

What is the Avaliar Proxy?

The Avaliar Proxy is a transparent gateway that sits between your AI agent and LLM providers. Every request flowing through the proxy is automatically captured, analyzed for safety issues, and stored as a trace on the Avaliar platform — with minimal changes to your existing setup.

Instead of instrumenting your codebase with decorators or SDK calls, you replace your LLM provider’s endpoint URL with the Avaliar Proxy URL and add your Avaliar API key to the request. The proxy handles everything else: tracing, detection, cost tracking, and alerting.

Supported Providers

Provider	Models
OpenAI	GPT-4o, GPT-4o-mini, GPT-4.1, GPT-4.1-mini, GPT-4.1-nano, GPT-5, GPT-5-mini, and more
Anthropic	Claude Opus 4.6, Claude Sonnet 4.6, Claude Opus 4.5, Claude Sonnet 4.5, Claude Haiku 4.5, and more
Google Gemini	Gemini 2.5 Pro, Gemini 2.5 Flash, Gemini 1.5 Pro, Gemini 1.5 Flash, and more

See the full Supported Models list for all available model IDs.

Benefits

• Minimal integration — Swap a URL and add an API key. No SDK, no decorators, no new dependencies
• Automatic tracing — Every request and response is captured and stored as a trace on Avaliar
• Safety detection — All traces are analyzed for prompt injection, jailbreaks, toxicity, PII, and more
• Prompt injection blocking — Harmful prompts are blocked before they reach the LLM provider
• Multi-provider support — Works with OpenAI, Anthropic, and Google Gemini
• Confidential mode — Opt specific requests out of logging for sensitive data

Proxy integration requires a Pro plan. Upgrade your plan to get started.

How It Works

The proxy adds a single hop between your AI agent and the LLM provider:

Your AI Agent  →  Avaliar Proxy  →  LLM Provider (OpenAI / Anthropic / Gemini)
      ↑                 |
      └─────────────────┘
        Response returned

For each request, the proxy:

1. Validates your Avaliar API key and checks for prompt injection
2. Forwards the request to the LLM provider
3. Captures the response and runs asynchronous safety detection
4. Returns the response to your agent — latency impact is minimal

SDK vs Proxy Integration

Feature	SDK	Proxy
Integration effort	Add @traceable decorator	Swap URL + add API key
Detection modes	Local + Cloud	Cloud only
Prompt blocking	No	Yes
Overhead tracking	No	Yes (proxy_overhead_ms)
Plan required	Free	Pro

Proxy Setup

Integrate the proxy in minutes — no code changes beyond a URL swap.

How the Proxy Works

Understand the 3-stage pipeline that powers every proxy request.