Skip to main content

​
Portkey AI Gateway Tracing & Observability

Portkey is an AI Gateway and Control Panel that provides production-ready features for AI applications including observability, reliability, and cost management. Learn how to instrument the Portkey SDK using the openinference-instrumentation-portkey package for comprehensive LLM tracing and monitoring.

​
Quick Start: Portkey Python Integration

​
Installation & Setup

Install the required packages for Portkey AI Gateway tracing: 
pip install openinference-instrumentation-portkey portkey-ai arize-otel

​
Instrumentation Setup

Configure the PortkeyInstrumentor and tracer to send traces to Arize for LLM observability: 
# Import open-telemetry dependencies
from arize.otel import register

# Setup OTel via our convenience function
tracer_provider = register(
    space_id = "your-space-id", # in app space settings page
    api_key = "your-api-key", # in app space settings page
    project_name = "your-project-name", # name this to whatever you would like
)

# Import openinference instrumentor to map Portkey traces to a standard format
from openinference.instrumentation.portkey import PortkeyInstrumentor

# Turn on the instrumentor
PortkeyInstrumentor().instrument(tracer_provider=tracer_provider)

​
Example: Basic Portkey AI Gateway Usage

Test your Portkey integration with this example code and observe traces in Arize: 
import os
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

# Set up your API keys
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
os.environ["PORTKEY_API_KEY"] = "your-portkey-api-key"  # Optional for self-hosted

client = OpenAI(
    api_key=os.environ.get("OPENAI_API_KEY"),
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        provider="openai",
        api_key=os.environ.get("PORTKEY_API_KEY")
    )
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is artificial intelligence?"}]
)

print(response.choices[0].message.content)
Start using your LLM application and monitor traces in Arize.

​
What is covered by the Instrumentation

Arize provides comprehensive observability for Portkey’s AI Gateway capabilities, automatically tracing:

​
Multi-Provider LLM Management

  • Multiple Provider Calls: Track requests across different LLM providers (OpenAI, Anthropic, Cohere) through Portkey’s unified interface
  • Provider Switching: Monitor seamless switching between AI providers
  • Cost Optimization: Track usage and costs across different LLM providers

​
Reliability & Performance Monitoring

  • Fallback and Retry Logic: Monitor automatic fallbacks and retry attempts when primary services fail
  • Load Balancing: Observe how requests are distributed across multiple models or providers
  • Latency Tracking: Monitor response times and performance metrics

​
Intelligent Caching & Optimization

  • Semantic Caching: See cache hits and misses for semantic caching to optimize costs
  • Request Deduplication: Track duplicate request handling
  • Performance Optimization: Identify bottlenecks and optimization opportunities