Quickstart

Start monitoring your LLM calls with Avaliar in 5 minutes — via the Proxy API or the Python SDK

There are two ways to send your LLM traffic to Avaliar. Both capture every request and response as a trace and run safety detection — pick the one that fits your stack.

Proxy API

Route your LLM calls through Avaliar’s endpoint. No SDK, no decorators — swap your provider URL and add your Avaliar key. Works with any language.
Best for: quick setup, non-Python apps, blocking prompt injection.

Python SDK

Wrap your LLM functions with the @traceable decorator. Code-native and gives you local or cloud detection.
Best for: Python apps, the free plan, fine-grained tracing.

Which should I use?

	Proxy API	Python SDK
How you integrate	Change your endpoint URL	Add a @traceable decorator
Language	Any (HTTP)	Python
Code changes	None beyond the URL	Decorate your LLM functions
Detection	Cloud	Local or Cloud
Blocks prompt injection	Yes (before it reaches the LLM)	No
Plan required	Pro	Free

Not sure? If you’re in Python and on the free plan, start with the SDK. If you want zero code changes or you’re not on Python, use the Proxy API.

---

Set up your method

Both paths start with an API key from the Avaliar Dashboard under Settings → API Keys — the only difference is the scope you pick.

Proxy API

1. Create a Proxy API key

   1. Open Settings → API Keys in the dashboard.
   2. Click Create Key and select the proxy scope.
   3. Copy the key — you won’t be able to see it again.

   Use your Avaliar key only in server-side code. Never expose it in a browser or client app.

2. Point your requests at the proxy

   Replace your provider’s endpoint with the matching Avaliar Proxy URL and add your api_key to the request body. Your existing provider key (OpenAI, Anthropic, …) stays in the body exactly where it was.

   Provider	Proxy Endpoint
   OpenAI	POST https://proxy.avaliar.ai/api/openai_service/chat_completion
   Anthropic	POST https://proxy.avaliar.ai/api/anthropic_service/chat_completion
   Google Gemini	POST https://proxy.avaliar.ai/api/gemini_service/chat_completion

   OpenAI

   import requests
   
   response = requests.post(
       "https://proxy.avaliar.ai/api/openai_service/chat_completion",
       json={
           "api_key": "your-avaliar-api-key",    # Avaliar key — new
           "openai_api_key": "your-openai-key",   # Your existing OpenAI key
           "model": "gpt-4o",
           "messages": [{"role": "user", "content": "Hello!"}],
       },
   )
   print(response.json())
   

   Detection runs automatically on every proxy request — there’s nothing extra to enable. See Proxy Setup for all parameters, confidential requests, and error handling.

3. View in the dashboard

   Open app.avaliar.ai/traces to see your traces, detected issues, latency, and cost in real time — a trace appears within a few seconds of the call.

Python SDK

1. Install the SDK

   pip install avaliar-python-sdk
   
2. Create an SDK key

   1. Open Settings → API Keys in the dashboard.
   2. Click Create Key and select the sdk scope.
   3. Copy the generated key.

   Store your API key securely. Treat it like a password and never commit it to source control.

3. Set the environment variable

   Export your key so the SDK authenticates automatically.

   macOS / Linux

   export AVALIAR_API_KEY="your-key"
   

   Add this to your shell profile (.bashrc, .zshrc, …) so it persists across sessions.

4. Add tracing

   Wrap any LLM call with the @traceable decorator. The SDK captures the input, output, model, provider, latency, and token usage automatically.

   from avaliar import traceable
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI()
   
   @traceable("llm", model="gpt-4o", provider="openai")
   async def generate(prompt: str) -> str:
       response = await client.chat.completions.create(
           model="gpt-4o",
           messages=[{"role": "user", "content": prompt}],
       )
       return response.choices[0].message.content
   

   Every call to generate() now creates a trace in Avaliar.

5. Enable detection

   Turn on safety detection by adding detection=True and listing the detectors you want to run.

   from avaliar import traceable
   from avaliar.detectors import DetectorType
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI()
   
   @traceable(
       "llm",
       model="gpt-4o",
       provider="openai",
       detection=True,
       detectors=[
           DetectorType.PROMPT_INJECTION,
           DetectorType.JAILBREAK,
           DetectorType.TOXICITY,
           DetectorType.PII,
           DetectorType.BIAS,
           DetectorType.HALLUCINATION,
       ],
   )
   async def generate(prompt: str) -> str:
       response = await client.chat.completions.create(
           model="gpt-4o",
           messages=[{"role": "user", "content": prompt}],
       )
       return response.choices[0].message.content
   

   Each detector runs independently — enable as many or as few as you need. See the Detection overview for details.

6. View in the dashboard

   Open app.avaliar.ai/traces to see your traces, detected issues, and latency metrics in real time.

   Traces typically appear within a few seconds of the LLM call completing.

Next Steps

Proxy Integration

Full proxy reference — supported models, confidential mode, and the request pipeline.

@traceable Decorator

All decorator options: span types, custom metadata, and nested traces.

Detection

Configure detectors, choose local vs cloud mode, and set severity thresholds.