> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Audio Transcription And Evaluation With Gemini Flash

> Prompt Gemini Flash with an audio file, trace the calls to Arize AX, and evaluate the transcript sentiment with LLM as a judge.

In this case, you'll use a [sound recording](https://www.jfklibrary.org/asset-viewer/archives/jfkwha-006) of President John F. Kennedy’s 1961 State of the Union address.

This guide performs the following tasks:

1. Prompt Gemini to generate a transcript of the audio recording.
2. Trace Gemini API calls and send the traces to the Arize AX platform with links to audio file for playback.
3. Evaluate the transcription output from Gemini for sentiment analysis using Phoenix Evals and Gemini LLM (LLM as a Judge).

## Install dependencies

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
pip install -U google-genai "arize-phoenix-evals<3" "arize<8" opentelemetry-api \
  opentelemetry-sdk openinference-semantic-conventions arize-otel
```

## Set your credentials

Copy your **Space ID** and **API key** from your Arize AX Space Settings page, and set them alongside your Gemini API key. If you don't have a Gemini key yet, create one in [Google AI Studio](https://aistudio.google.com/apikey).

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
export ARIZE_SPACE_ID="<your-space-id>"
export ARIZE_API_KEY="<your-api-key>"
export GEMINI_API_KEY="<your-gemini-api-key>"
export GOOGLE_CLOUD_PROJECT="<your-google-cloud-project-id>"
```

`GOOGLE_CLOUD_PROJECT` is used later by the Gemini judge, which reaches the model through Vertex AI rather than the Gemini API.

## Load an audio file sample and set the URL

Download the recording locally. The URL is also passed to Arize AX later so you can play the audio back next to the trace.

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
curl -sL "https://storage.googleapis.com/generativeai-downloads/data/State_of_the_Union_Address_30_January_1961.mp3" \
  -o sample.mp3
```

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
import os

from google import genai

## Audio file url --> allows you to play audio in UI
URL = "https://storage.googleapis.com/generativeai-downloads/data/State_of_the_Union_Address_30_January_1961.mp3"

gemini_client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

your_file = gemini_client.files.upload(file="sample.mp3")
```

## Tracing setup

You'll need to set Arize AX variables (Space id, API key and Developer Key) below to send traces to the Arize AX Platform. Sign up for free [here](https://app.arize.com/auth/join).

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
from opentelemetry import trace
from arize.otel import register
from opentelemetry.trace import Status, StatusCode
from opentelemetry.semconv.trace import SpanAttributes


PROJECT_NAME = "gemini-audio"  # Set this to any name you'd like for your app

# Setup OTel via our convenience function
tracer_provider = register(
    space_id=os.environ["ARIZE_SPACE_ID"],  # in app space settings page
    api_key=os.environ["ARIZE_API_KEY"],  # in app space settings page
    project_name=PROJECT_NAME,
)

trace.set_tracer_provider(tracer_provider)
tracer = trace.get_tracer(__name__)
```

### Configure prompt

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
prompt = "Provide a transcript of the speech from 01:00 to 01:30."
```

## Call Gemini

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
# Call Gemini

with tracer.start_as_current_span(
    "process_audio",
    openinference_span_kind="llm",
) as span:
  span.set_attribute("input.audio.url", URL)
  span.set_attribute("llm.prompts", prompt)
  span.set_attribute("input.value", prompt)
  response = gemini_client.models.generate_content(
    model='gemini-3.5-flash',
    contents=[
      prompt,
      your_file,
    ]
  )
  span.set_attribute("input.audio.transcript", response.text)
  span.set_attribute("output.value", response.text)
  span.set_status(Status(StatusCode.OK))

response.text

```

## Evaluate Gemini's output transcript for sentiment analysis

First, export spans from Arize AX that contain transcript output from Arize AX

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
pip install "arize[Tracing]>=7.1.0,<8" packaging
```

```python expandable theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
# Note: This example uses Python SDK v7
import os
from datetime import datetime, timezone, timedelta

from arize.exporter import ArizeExportClient
from arize.utils.types import Environments

client = ArizeExportClient()

print('#### Exporting your primary dataset into a dataframe.')

primary_df = client.export_model_to_df(
    space_id=os.environ["ARIZE_SPACE_ID"],
    model_id=PROJECT_NAME,
    where="name = 'process_audio'", #Just pull the spans with name = "process_audio"
    environment=Environments.TRACING,
    start_time = datetime.now(timezone.utc) - timedelta(days=1),
    end_time = datetime.now(timezone.utc) #pull traces for the last 24 hours
)

#set the column in the dataframe to match the variable name used in our eval template
primary_df["output"] = primary_df["attributes.output.value"]

```

### Evaluation Template

```python expandable theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
SENTIMENT_EVAL_TEMPLATE = """

You are a helpful AI bot that checks for the sentiment in the output text. Your task is to evaluate the sentiment of the given output and categorize it as positive, neutral, or negative.

Here is the data:
[BEGIN DATA]
============
[Output]: {attributes.output.value}
============
[END DATA]

Determine the sentiment of the output based on the content and context provided. Your response should be ONLY a single word, either "positive", "neutral", or "negative", and should not contain any text or characters aside from that word.

Then write out in a step by step manner an EXPLANATION to show how you determined the sentiment of the output.  Do not include any text or characters aside from the EXPLANATION.

Your response should follow the format of the example response below. Provide a single LABEL and a single EXPLANATION. Do not include any special characters in the response. Do not include special characters such as "#" in your response.

Example response:

EXPLANATION: An explanation of your reasoning for why the label is "positive", "neutral", or "negative"
LABEL: "positive" or "neutral" or "negative"

"""
```

#### Evaluate transcriptions using Gemini as a LLM as a Judge

`GeminiModel` reaches Gemini through Vertex AI, so it needs Google Cloud credentials and a project. Run these once, using a project that has the Vertex AI API enabled.

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
gcloud auth application-default login
gcloud config set project <your-google-cloud-project-id>
```

```python expandable theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
#Gemini as LLM as a Judge - LLM Classify
import os

import pandas as pd
from phoenix.evals import (GeminiModel, llm_classify)

#We will use Gemini 2.5 Pro to evaluate the text transcription
project_id = os.environ["GOOGLE_CLOUD_PROJECT"] # Set this to your google project id
gemini_model = GeminiModel(model="gemini-2.5-pro", project=project_id)

rails = ["positive", "neutral", "negative"]

evals_df = llm_classify(
    data=primary_df,
    template=SENTIMENT_EVAL_TEMPLATE,
    model=gemini_model,
    rails=rails,
    provide_explanation=True
)

#set eval labels
evals_df["eval.sentiment.label"] = evals_df["label"]
evals_df["eval.sentiment.explanation"] = evals_df["explanation"]
evals_df["context.span_id"] = primary_df["context.span_id"]

evals_df.head()
```

### Send evaluations to Arize AX

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
# Note: This example uses Python SDK v7
import os

from arize.pandas.logger import Client


# Initialize Arize AX client using the model_id and version you used previously
arize_client = Client(
    space_id=os.environ["ARIZE_SPACE_ID"],
    api_key=os.environ["ARIZE_API_KEY"],
)

# send the evaluation results to Arize AX
arize_client.log_evaluations_sync(evals_df, "gemini-audio")
```

## Next Steps

#### Useful API references:

More details about Gemini API's [vision capabilities](https://ai.google.dev/gemini-api/docs/vision) in the documentation.

If you want to know about the File API, check its [API reference](https://ai.google.dev/api/files) or the [File API](https://github.com/google-gemini/cookbook/blob/main/quickstarts/File_API.ipynb) quickstart.

#### Related examples

Check this example using the audio files to give you more ideas on what the gemini API can do with them:

* Share [Voice memos](https://github.com/google-gemini/cookbook/blob/main/examples/Voice_memos.ipynb) with Gemini API and brainstorm ideas

#### Continue your discovery of the Gemini API

Have a look at the Audio quickstart to learn about another type of media file, then learn more about [prompting with media files](https://ai.google.dev/tutorials/prompting_with_media) in the docs, including the supported formats and maximum length for audio files. .