Vertex AI

Documentation Index

Fetch the complete documentation index at: https://arize-ax.mintlify.dev/docs/llms.txt

Use this file to discover all available pages before exploring further.

​

Prerequisites

• Python 3.10+
• An Arize AX account (sign up)
• A Google Cloud project with the Vertex AI API enabled
• A service account or user with the roles/aiplatform.user IAM role
• Authenticated Application Default Credentials (gcloud auth application-default login) or a service account JSON file referenced by GOOGLE_APPLICATION_CREDENTIALS

​

Launch Arize AX

1. Sign in to your Arize AX account.
2. From Space Settings, copy your Space ID and API Key. You will set them as ARIZE_SPACE_ID and ARIZE_API_KEY below.

​

Install

pip install arize-otel openinference-instrumentation-google-genai google-genai

​

Configure credentials

# Recommended for local dev — uses your gcloud user credentials.
gcloud auth application-default login

# Or, with a service account JSON file:
# export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"

export ARIZE_SPACE_ID="<your-space-id>"
export ARIZE_API_KEY="<your-api-key>"
export ARIZE_PROJECT_NAME="vertexai-tracing-example"
export GOOGLE_CLOUD_PROJECT="<your-gcp-project-id>"
export GOOGLE_CLOUD_LOCATION="us-central1"  # optional; the SDK defaults to us-central1

​

Setup tracing

# instrumentation.py
import os

from arize.otel import register
from openinference.instrumentation.google_genai import GoogleGenAIInstrumentor

tracer_provider = register(
    space_id=os.environ["ARIZE_SPACE_ID"],
    api_key=os.environ["ARIZE_API_KEY"],
    project_name=os.environ["ARIZE_PROJECT_NAME"],
)

GoogleGenAIInstrumentor().instrument(tracer_provider=tracer_provider)
print("Arize AX tracing initialized for Vertex AI.")

​

Run Vertex AI

# example.py

# Importing instrumentation first ensures tracing is set up
# before `google.genai` is imported.
from instrumentation import tracer_provider

from google import genai

# `vertexai=True` flips the client into Vertex mode. Project and location
# come from GOOGLE_CLOUD_PROJECT / GOOGLE_CLOUD_LOCATION env vars, or pass
# them explicitly: genai.Client(vertexai=True, project="...", location="...")
client = genai.Client(vertexai=True)

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Why is the ocean salty? Answer in two sentences.",
)

print(response.text)

​

Expected output

Arize AX tracing initialized for Vertex AI.
The ocean is salty because rivers continuously dissolve mineral salts from rocks and soil and carry them to the sea, where they accumulate over millions of years. Water leaves the ocean through evaporation but the salts remain, steadily concentrating until reaching today's roughly 3.5% salinity.

​

Verify in Arize AX

1. Open your Arize AX space and select project vertexai-tracing-example.
2. You should see a new trace within ~30 seconds containing a GenerateContent LLM span with the prompt, response, and token usage attached.
3. If no traces appear, see Troubleshooting.

​

Troubleshooting

• No traces in Arize AX. Confirm ARIZE_SPACE_ID and ARIZE_API_KEY are set in the same shell that runs example.py. Enable OpenTelemetry debug logs with export OTEL_LOG_LEVEL=debug and re-run.
• Vertex spans missing but other spans present. GoogleGenAIInstrumentor().instrument(...) must run before any from google import genai import. Make sure instrumentation.py is the first import in your entry point.
• 401 UNAUTHENTICATED. API keys are not supported by this API. ADC isn’t set up. Run gcloud auth application-default login (interactive) or set GOOGLE_APPLICATION_CREDENTIALS to a service account JSON. Vertex AI accepts only IAM-based auth, not API keys.
• 403 PERMISSION_DENIED. Permission denied on resource project ... The principal in your ADC doesn’t have roles/aiplatform.user (or finer-grained Vertex permissions) on the project, or you authenticated with end-user credentials that have no quota project. Grant the role in the IAM console, then run gcloud auth application-default set-quota-project <PROJECT_ID>.
• Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. This warning means gcloud auth application-default login succeeded but you haven’t set a quota project. Calls work but may hit 403 later. Fix with gcloud auth application-default set-quota-project <PROJECT_ID>.
• 404 NOT_FOUND for the model. The model isn’t available in the region you set for GOOGLE_CLOUD_LOCATION. Check the Vertex AI generative model availability matrix and swap regions accordingly.
• Using the Gemini API instead of Vertex. Drop vertexai=True and set GEMINI_API_KEY — see the Google GenAI tracing doc for the API-key path. The same instrumentor captures both.
• Using the legacy vertexai SDK (vertexai.GenerativeModel). That SDK is deprecated as of June 24, 2025 and will be removed June 24, 2026. Migrate to google-genai with genai.Client(vertexai=True, ...) as shown above. The OpenInference instrumentor for the legacy SDK (openinference-instrumentation-vertexai) still works against gapic but is no longer the recommended path.

​

Resources