Managing LLM integrations with GraphQL

AI provider integrations connect Arize AX to an LLM provider so you can run prompt playground sessions, LLM-as-a-judge evaluations and managed agents without pasting a key into every tool. This guide covers the mutations for creating and rotating those integrations, and the separate, narrower credential records the schema also exposes.

Find the IDs you need

LLM integrations hang off an account, an organization, or a space. Start from viewer for the space ID and account for the account and organization IDs. See using global node IDs for how these opaque IDs work.

query FindAccountOrgAndSpaceIds {
  account {
    id
    name
    organizations(first: 10) {
      edges {
        node { id name }
      }
    }
  }
  viewer {
    spaces(first: 20) {
      edges {
        node { id name }
      }
    }
  }
}

{}

Understand the three resource types

LlmIntegration: backs the AI Providers UI, prompt playground, evals and agent runtimes. It carries its own apiKey, baseUrl and modelNames, is scoped via scopings, and is listed on Account.llmIntegrations and Space.llmIntegrations.
ExternalLlmApiKey and CustomLlmEndpoint: narrower records scoped by accountOrganizationId directly (no further scoping, unlike LlmIntegration’s scopings), listed on AccountOrganization.externalLlmApiKeys and .customLlmEndpoints. Nothing links either back to an LlmIntegration; creating one does not create a usable playground integration.
GoogleCloudIntegration: an organization-scoped record that verifies access to a Google Cloud project. It has no query field, so the only way to see one again is the ID the mutation returned.

For most automation, create an LlmIntegration directly with createLlmIntegration. Reach for ExternalLlmApiKey or CustomLlmEndpoint only if you specifically need that narrower record.

List configured LLM integrations

List the integrations available to the account and to a space. Space-level results include integrations scoped to that space, its organization, or the whole account.

query ListLlmIntegrations($spaceId: ID!) {
  account {
    llmIntegrations {
      id
      name
      provider
      hasApiKey
    }
  }
  node(id: $spaceId) {
    ... on Space {
      llmIntegrations {
        id
        name
        provider
        allowUseInAgents
      }
    }
  }
}

{
  "spaceId": "U3BhY2U6MTIz"
}

Never select apiKey in a listing query; use hasApiKey to check whether a key is set without round-tripping the secret.

Add a provider API key and connect it as an LLM integration

An ExternalLlmApiKey only stores a raw credential at the organization level. To actually use the provider in the playground, evals or agents, create an LlmIntegration as a separate step, passing the key straight to it.

mutation AddExternalApiKey($input: CreateExternalLlmApiKeyInput!) {
  createExternalLlmApiKey(input: $input) {
    externalLlmApiKey { id provider createdAt }
  }
}

mutation CreateIntegrationFromKey($input: CreateLlmIntegrationInput!) {
  createLlmIntegration(input: $input) {
    llmIntegration { id name provider hasApiKey }
  }
}

{
  "input": {
    "accountOrganizationId": "QWNjb3VudE9yZ2FuaXphdGlvbjo0",
    "apiKey": "sk-ant-REPLACE_ME",
    "provider": "anthropic"
  }
}

The second mutation’s input reuses the same key:

{ "accountId": "QWNjb3VudDo5", "provider": "anthropic", "name": "Team Anthropic key", "apiKey": "sk-ant-REPLACE_ME", "enableDefaultModels": true, "scopings": [{ "spaceId": "U3BhY2U6MTIz" }] }

. Omit scopings for an account-wide integration. For AWS or GCP, add providerMetadata (AWS: roleArn, externalId; GCP: projectId, location, projectAccessLabel). Reference: createExternalLlmApiKey, createLlmIntegration.

Register a custom OpenAI-compatible endpoint

Point Arize at a self-hosted or third-party OpenAI-compatible model server with createCustomLlmEndpoint, or use createLlmIntegration with provider: "custom" and a baseUrl if you want it usable directly in the playground.

mutation RegisterCustomEndpoint($input: CreateCustomLlmEndpointInput!) {
  createCustomLlmEndpoint(input: $input) {
    customLlmEndpoint { id name modelName baseUrl hasApiKey }
  }
}

{
  "input": {
    "accountOrganizationId": "QWNjb3VudE9yZ2FuaXphdGlvbjo0",
    "name": "Internal Llama endpoint",
    "modelName": "llama-3.3-70b",
    "baseUrl": "https://my-llm-proxy.example.com/v1",
    "apiKey": "REPLACE_ME",
    "compatibleFormat": "openai"
  }
}

Enter the base URL including the version path (for example, /v1) but without an endpoint path like /chat/completions; Arize appends that automatically. Add headers (a list of { key, value } pairs) for anything else your proxy requires. Reference: createCustomLlmEndpoint.

Set up a Google Cloud (Vertex) integration

Verify access to a Google Cloud project before using Vertex AI models. Set an arize-integration-key label on the GCP project first, then register that same value as projectAccessLabel.

mutation SetUpGoogleCloud($input: CreateGoogleCloudIntegrationInput!) {
  createGoogleCloudIntegration(input: $input) {
    googleCloudIntegration { id projectId location }
  }
}

{
  "input": {
    "accountOrganizationId": "QWNjb3VudE9yZ2FuaXphdGlvbjo0",
    "projectId": "my-gcp-project",
    "location": "us-central1",
    "projectAccessLabel": "my-arize-integration-key-value"
  }
}

GoogleCloudIntegration has no list query and does not implement Node, so it cannot be refetched by ID either. Save the id this mutation returns. To use Vertex models in the playground or evals, also create an LlmIntegration with provider: "googleCloud" and the same projectId/location/projectAccessLabel in providerMetadata.

Reference: createGoogleCloudIntegration.

Rotate or remove a key

Update the stored key on an LlmIntegration without recreating it, or delete the integration outright once it is no longer in use.

mutation RotateIntegrationKey($input: UpdateLlmIntegrationInput!) {
  updateLlmIntegration(input: $input) {
    llmIntegration { id hasApiKey updatedAt }
  }
}

mutation RemoveIntegration($input: DeleteLlmIntegrationInput!) {
  deleteLlmIntegration(input: $input) {
    success
  }
}

{
  "input": {
    "integrationId": "TGxtSW50ZWdyYXRpb246NTU=",
    "apiKey": "sk-proj-NEW_KEY"
  }
}

Passing null or an empty string for apiKey on updateLlmIntegration removes the stored key instead of rotating it. updateExternalLlmApiKey, deleteExternalLlmApiKey, deleteCustomLlmEndpoint and deleteGoogleCloudIntegration follow the same shape for the narrower resource types. Reference: updateLlmIntegration, deleteLlmIntegration.

Gotchas and behavior notes

The input provider enum and the output provider enum do not match

createLlmIntegration and updateLlmIntegration take provider: LlmProvider (lowercase values like openai, azureopenai, aws, googleCloud). Reading an integration back returns provider: LLMIntegrationProvider, a differently-cased and differently-spelled enum (openAI, azureOpenAI, awsBedrock, vertexAI). Do not echo a value you read straight back into a mutation.

oauthConfig is required when enabling OAuth, and omitting it keeps the existing config

On updateLlmIntegration, switching authType to oauth2_client_credentials requires oauthConfig with tokenUrl, clientId and clientSecret; switching away from it deletes the stored config. Omitting oauthConfig while authType is unchanged keeps whatever is already stored.

compatibleFormat uses a narrower provider enum

createCustomLlmEndpoint and createLlmIntegration both type compatibleFormat as ExternalLlmApiKeyProvider, which has only 8 values and does not include custom, googleCloud, aws, cursor or typeSafeAi. It defaults to openai.

LLM integration mutations

Full argument and return-type reference for all twelve mutations.

All mutations

Index of every GraphQL mutation grouped by domain.

API explorer

Run queries and mutations interactively against your own account.

Skills

MCP server

API reference

API clients

Managing LLM integrations with GraphQL

Find the IDs you need

Understand the three resource types

List configured LLM integrations

Add a provider API key and connect it as an LLM integration

Register a custom OpenAI-compatible endpoint

Set up a Google Cloud (Vertex) integration

Rotate or remove a key

Gotchas and behavior notes

LLM integration mutations

All mutations

API explorer

​Find the IDs you need

​Understand the three resource types

​List configured LLM integrations

​Add a provider API key and connect it as an LLM integration

​Register a custom OpenAI-compatible endpoint

​Set up a Google Cloud (Vertex) integration

​Rotate or remove a key

​Gotchas and behavior notes

LLM integration mutations

All mutations

API explorer

Find the IDs you need

Understand the three resource types

List configured LLM integrations

Add a provider API key and connect it as an LLM integration

Register a custom OpenAI-compatible endpoint

Set up a Google Cloud (Vertex) integration

Rotate or remove a key

Gotchas and behavior notes