Evaluators

The evaluators client methods are currently in BETA. The API may change without notice. A one-time warning is emitted on first use.

An evaluator scores spans or examples. Each evaluator has one or more immutable versions; a version is a template (LLM-as-judge), a code (managed built-in or user-supplied Python), or a remote (customer-hosted HTTP endpoint backed by an EVALUATOR integration) configuration. The Get, Update, Delete, ListVersions, DeleteVersions, and CreateVersion methods accept either an evaluator name or an ID — when a name is passed, the parent Space (name or ID) is also required so the SDK can resolve the name to a unique ID. GetVersion takes a strict version ID. Set exactly one of VersionConfig.Template, VersionConfig.Code, or VersionConfig.Remote on Create and CreateVersion; within Code, set exactly one of Managed or Custom. Conflicting configurations return evaluators.ErrConflictingVersionConfig or evaluators.ErrConflictingCodeConfig without contacting the server. Typed convenience methods (CreateTemplateEvaluator, CreateCodeEvaluator, CreateRemoteEvaluator and the matching CreateTemplateVersion, CreateCodeVersion, CreateRemoteVersion) create one evaluator kind directly without assembling a VersionConfig.

List Evaluators

List returns a paginated list of evaluators. Defaults to a page size of 50. Space, when non-empty, filters by space (substring match on space name, or exact match if a base64 resource ID is passed). Signature:

func (c *Client) List(ctx context.Context, req ListRequest) (*ListEvaluators, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    resp, err := client.Evaluators.List(context.Background(), evaluators.ListRequest{
        Space: "your-space-name-or-id",
        Limit: 25,
    })
    if err != nil {
        var unauthorized *arize.UnauthorizedError
        if errors.As(err, &unauthorized) {
            log.Fatalf("unauthorized: %v", unauthorized)
        }
        log.Fatal(err)
    }

    for _, ev := range resp.Evaluators {
        fmt.Printf("%s: %s (type=%s)\n", ev.Id, ev.Name, ev.Type)
    }
}

Get an Evaluator

Get returns a single evaluator (with one of its versions), resolving by name or ID. When VersionID is set, the response carries that specific version; otherwise the latest version is returned. Signature:

func (c *Client) Get(ctx context.Context, req GetRequest) (*EvaluatorWithVersion, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    ev, err := client.Evaluators.Get(
        context.Background(),
        evaluators.GetRequest{
            Evaluator: "your-evaluator-name-or-id",
            Space:     "your-space-name-or-id",
        },
    )
    if err != nil {
        var notFound *arize.NotFoundError
        if errors.As(err, &notFound) {
            log.Fatalf("evaluator not found: %v", notFound)
        }
        log.Fatal(err)
    }

    fmt.Printf("evaluator %s: %s (type=%s)\n", ev.Id, ev.Name, ev.Type)
}

Create an Evaluator

Create creates a new evaluator together with its initial version, resolving the parent space by name or ID. The evaluator’s type is derived from Version — set Version.Template for a template (LLM-based) evaluator or Version.Code for a code evaluator (with exactly one of Code.Managed or Code.Custom). Signature:

func (c *Client) Create(ctx context.Context, req CreateRequest) (*EvaluatorWithVersion, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    ev, err := client.Evaluators.Create(
        context.Background(),
        evaluators.CreateRequest{
            Space:       "your-space-name-or-id",
            Name:        "hallucination-judge",
            Description: "scores response groundedness against retrieved context",
            Version: evaluators.VersionConfig{
                CommitMessage: "initial version",
                Template: &evaluators.TemplateConfigInput{
                    // Template configuration (prompt, LLM config, etc.)
                },
            },
        },
    )
    if err != nil {
        if errors.Is(err, evaluators.ErrConflictingVersionConfig) {
            log.Fatal("set exactly one of Version.Template or Version.Code")
        }
        var conflict *arize.ConflictError
        if errors.As(err, &conflict) {
            log.Fatalf("evaluator already exists: %v", conflict)
        }
        log.Fatal(err)
    }

    fmt.Printf("created evaluator %s\n", ev.Id)
}

Update an Evaluator

Update patches an existing evaluator’s metadata, resolving the target by name or ID, and returns the updated evaluator. Name and Description are pointers — nil preserves the existing value. Returns evaluators.ErrNoUpdateFields without contacting the server when no patch field is set. Signature:

func (c *Client) Update(ctx context.Context, req UpdateRequest) (*Evaluator, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    newName := "renamed-evaluator"
    ev, err := client.Evaluators.Update(
        context.Background(),
        evaluators.UpdateRequest{
            Evaluator: "your-evaluator-name-or-id",
            Space:     "your-space-name-or-id",
            Name:      &newName,
        },
    )
    if err != nil {
        if errors.Is(err, evaluators.ErrNoUpdateFields) {
            log.Fatal("no fields to update")
        }
        var notFound *arize.NotFoundError
        if errors.As(err, &notFound) {
            log.Fatalf("evaluator not found: %v", notFound)
        }
        log.Fatal(err)
    }

    fmt.Printf("updated evaluator %s: %s\n", ev.Id, ev.Name)
}

Delete an Evaluator

Delete removes an evaluator, resolving by name or ID. It returns only an error. Signature:

func (c *Client) Delete(ctx context.Context, req DeleteRequest) error

Usage Example:

package main

import (
    "context"
    "errors"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    err = client.Evaluators.Delete(
        context.Background(),
        evaluators.DeleteRequest{
            Evaluator: "your-evaluator-name-or-id",
            Space:     "your-space-name-or-id",
        },
    )
    if err != nil {
        var notFound *arize.NotFoundError
        if errors.As(err, &notFound) {
            log.Printf("no evaluator to remove: %v", notFound)
            return
        }
        log.Fatal(err)
    }
}

List Versions

ListVersions returns a paginated list of versions for an evaluator, resolving the evaluator by name or ID. Defaults to a page size of 50. Signature:

func (c *Client) ListVersions(
    ctx context.Context,
    req ListVersionsRequest,
) (*ListEvaluatorVersions, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    resp, err := client.Evaluators.ListVersions(
        context.Background(),
        evaluators.ListVersionsRequest{
            Evaluator: "your-evaluator-name-or-id",
            Space:     "your-space-name-or-id",
            Limit:     25,
        },
    )
    if err != nil {
        var notFound *arize.NotFoundError
        if errors.As(err, &notFound) {
            log.Fatalf("evaluator not found: %v", notFound)
        }
        log.Fatal(err)
    }

    for _, v := range resp.EvaluatorVersions {
        if d, err := v.Discriminator(); err == nil {
            fmt.Printf("version (type=%s)\n", d)
        }
    }
}

Delete Versions

DeleteVersions deletes a batch of versions (1–100 per request) belonging to an evaluator, resolving the evaluator by name or ID. The delete is partial-tolerant: version IDs that exist and belong to the evaluator are deleted, and requested IDs that were not deleted are returned in NotDeletedVersionIds. Completed reports that the request finished — it does not mean every requested ID was found and deleted. Signature:

func (c *Client) DeleteVersions(
    ctx context.Context,
    req DeleteVersionsRequest,
) (*DeleteEvaluatorVersions, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    result, err := client.Evaluators.DeleteVersions(
        context.Background(),
        evaluators.DeleteVersionsRequest{
            Evaluator:  "your-evaluator-name-or-id",
            Space:      "your-space-name-or-id",
            VersionIDs: []string{"version-id-1", "version-id-2"},
        },
    )
    if err != nil {
        var notFound *arize.NotFoundError
        if errors.As(err, &notFound) {
            log.Fatalf("evaluator not found: %v", notFound)
        }
        log.Fatal(err)
    }

    fmt.Printf("delete completed=%t, %d version IDs not deleted\n",
        result.Completed,
        len(result.NotDeletedVersionIds),
    )
}

Create a Version

CreateVersion appends a new version to an existing evaluator, resolving the evaluator by name or ID. The new version’s kind (set via Version.Template, Version.Code, or Version.Remote) must match the parent evaluator’s type. Signature:

func (c *Client) CreateVersion(
    ctx context.Context,
    req CreateVersionRequest,
) (*EvaluatorVersion, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    v, err := client.Evaluators.CreateVersion(
        context.Background(),
        evaluators.CreateVersionRequest{
            Evaluator: "your-evaluator-name-or-id",
            Space:     "your-space-name-or-id",
            Version: evaluators.VersionConfig{
                CommitMessage: "tighten judging rubric",
                Template: &evaluators.TemplateConfigInput{
                    // Template configuration (prompt, LLM config, etc.)
                },
            },
        },
    )
    if err != nil {
        if errors.Is(err, evaluators.ErrConflictingVersionConfig) {
            log.Fatal("set exactly one of Version.Template or Version.Code")
        }
        var badRequest *arize.BadRequestError
        if errors.As(err, &badRequest) {
            log.Fatalf("invalid version: %v", badRequest)
        }
        log.Fatal(err)
    }

    fmt.Printf("created evaluator version\n")
    if tpl, err := v.AsEvaluatorVersionTemplate(); err == nil {
        fmt.Printf("template commit: %s\n", tpl.CommitMessage)
    }
}

Get a Version

GetVersion returns a single evaluator version by its ID. Version IDs are strict IDs — no name resolution is performed. The returned EvaluatorVersion is a discriminated union — call v.AsEvaluatorVersionTemplate() or v.AsEvaluatorVersionCode() to extract the typed variant after checking the discriminator. Signature:

func (c *Client) GetVersion(ctx context.Context, req GetVersionRequest) (*EvaluatorVersion, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    v, err := client.Evaluators.GetVersion(
        context.Background(),
        evaluators.GetVersionRequest{VersionID: "your-version-id"},
    )
    if err != nil {
        var notFound *arize.NotFoundError
        if errors.As(err, &notFound) {
            log.Fatalf("version not found: %v", notFound)
        }
        log.Fatal(err)
    }

    d, err := v.Discriminator()
    if err != nil {
        log.Fatal(err)
    }
    switch d {
    case string(evaluators.EvaluatorTypeTemplate):
        tpl, err := v.AsEvaluatorVersionTemplate()
        if err != nil {
            log.Fatal(err)
        }
        fmt.Printf("template version: %s\n", tpl.CommitMessage)
    case string(evaluators.EvaluatorTypeCode):
        code, err := v.AsEvaluatorVersionCode()
        if err != nil {
            log.Fatal(err)
        }
        fmt.Printf("code version: %s\n", code.CommitMessage)
    }
}

Create a Template Evaluator

CreateTemplateEvaluator creates a TEMPLATE (LLM-as-judge) evaluator together with its initial version in one call. It is equivalent to Create with Version.Template set, without assembling a VersionConfig. Signature:

func (c *Client) CreateTemplateEvaluator(
    ctx context.Context,
    req CreateTemplateEvaluatorRequest,
) (*EvaluatorWithVersion, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    ev, err := client.Evaluators.CreateTemplateEvaluator(
        context.Background(),
        evaluators.CreateTemplateEvaluatorRequest{
            Space:         "your-space-name-or-id",
            Name:          "hallucination-judge",
            Description:   "scores response groundedness against retrieved context",
            CommitMessage: "initial version",
            Config: evaluators.TemplateConfigInput{
                // Template configuration (prompt, LLM config, etc.)
            },
        },
    )
    if err != nil {
        var conflict *arize.ConflictError
        if errors.As(err, &conflict) {
            log.Fatalf("evaluator already exists: %v", conflict)
        }
        log.Fatal(err)
    }

    fmt.Printf("created template evaluator %s\n", ev.Id)
}

Create a Code Evaluator

CreateCodeEvaluator creates a CODE evaluator together with its initial version in one call. It is equivalent to Create with Version.Code set. Set exactly one of Config.Managed (a built-in managed evaluator) or Config.Custom (user-supplied Python); a conflicting configuration returns evaluators.ErrConflictingCodeConfig without contacting the server. Signature:

func (c *Client) CreateCodeEvaluator(
    ctx context.Context,
    req CreateCodeEvaluatorRequest,
) (*EvaluatorWithVersion, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    ev, err := client.Evaluators.CreateCodeEvaluator(
        context.Background(),
        evaluators.CreateCodeEvaluatorRequest{
            Space:         "your-space-name-or-id",
            Name:          "exact-match",
            CommitMessage: "initial version",
            Config: evaluators.CodeConfig{
                Custom: &evaluators.CustomCodeConfig{
                    // Custom code configuration (name, code, variables, etc.)
                },
            },
        },
    )
    if err != nil {
        if errors.Is(err, evaluators.ErrConflictingCodeConfig) {
            log.Fatal("set exactly one of Config.Managed or Config.Custom")
        }
        log.Fatal(err)
    }

    fmt.Printf("created code evaluator %s\n", ev.Id)
}

Create a Remote Evaluator

CreateRemoteEvaluator creates a REMOTE evaluator backed by an existing EVALUATOR integration, together with its initial version. It is equivalent to Create with Version.Remote set. This method requires the enableRemoteEvalTasks feature flag on the account. Signature:

func (c *Client) CreateRemoteEvaluator(
    ctx context.Context,
    req CreateRemoteEvaluatorRequest,
) (*EvaluatorWithVersion, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    ev, err := client.Evaluators.CreateRemoteEvaluator(
        context.Background(),
        evaluators.CreateRemoteEvaluatorRequest{
            Space:         "your-space-name-or-id",
            Name:          "remote-judge",
            CommitMessage: "initial version",
            IntegrationID: "your-evaluator-integration-id",
        },
    )
    if err != nil {
        var badRequest *arize.BadRequestError
        if errors.As(err, &badRequest) {
            log.Fatalf("invalid remote evaluator: %v", badRequest)
        }
        log.Fatal(err)
    }

    fmt.Printf("created remote evaluator %s\n", ev.Id)
}

Create a Template Version

CreateTemplateVersion appends a new TEMPLATE version to an existing evaluator, resolving the evaluator by name or ID. It is equivalent to CreateVersion with Version.Template set; the parent evaluator must be a template evaluator. Signature:

func (c *Client) CreateTemplateVersion(
    ctx context.Context,
    req CreateTemplateVersionRequest,
) (*EvaluatorVersion, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    v, err := client.Evaluators.CreateTemplateVersion(
        context.Background(),
        evaluators.CreateTemplateVersionRequest{
            Evaluator:     "your-evaluator-name-or-id",
            Space:         "your-space-name-or-id",
            CommitMessage: "tighten judging rubric",
            Config: evaluators.TemplateConfigInput{
                // Template configuration (prompt, LLM config, etc.)
            },
        },
    )
    if err != nil {
        var badRequest *arize.BadRequestError
        if errors.As(err, &badRequest) {
            log.Fatalf("invalid version: %v", badRequest)
        }
        log.Fatal(err)
    }

    if tpl, err := v.AsEvaluatorVersionTemplate(); err == nil {
        fmt.Printf("created template version: %s\n", tpl.CommitMessage)
    }
}

Create a Code Version

CreateCodeVersion appends a new CODE version to an existing evaluator, resolving the evaluator by name or ID. It is equivalent to CreateVersion with Version.Code set. Set exactly one of Config.Managed or Config.Custom; a conflicting configuration returns evaluators.ErrConflictingCodeConfig without contacting the server. Signature:

func (c *Client) CreateCodeVersion(
    ctx context.Context,
    req CreateCodeVersionRequest,
) (*EvaluatorVersion, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    v, err := client.Evaluators.CreateCodeVersion(
        context.Background(),
        evaluators.CreateCodeVersionRequest{
            Evaluator:     "your-evaluator-name-or-id",
            Space:         "your-space-name-or-id",
            CommitMessage: "add tolerance to comparison",
            Config: evaluators.CodeConfig{
                Custom: &evaluators.CustomCodeConfig{
                    // Custom code configuration (name, code, variables, etc.)
                },
            },
        },
    )
    if err != nil {
        if errors.Is(err, evaluators.ErrConflictingCodeConfig) {
            log.Fatal("set exactly one of Config.Managed or Config.Custom")
        }
        log.Fatal(err)
    }

    if code, err := v.AsEvaluatorVersionCode(); err == nil {
        fmt.Printf("created code version: %s\n", code.CommitMessage)
    }
}

Create a Remote Version

CreateRemoteVersion appends a new REMOTE version to an existing evaluator, resolving the evaluator by name or ID, optionally switching to a different EVALUATOR integration. It is equivalent to CreateVersion with Version.Remote set. Signature:

func (c *Client) CreateRemoteVersion(
    ctx context.Context,
    req CreateRemoteVersionRequest,
) (*EvaluatorVersion, error)

Usage Example:

package main

import (
    "context"
    "errors"
    "fmt"
    "log"

    "github.com/Arize-ai/client-go-v2/arize"
    "github.com/Arize-ai/client-go-v2/arize/evaluators"
)

func main() {
    client, err := arize.NewClient(arize.Config{APIKey: "your-api-key"})
    if err != nil {
        log.Fatal(err)
    }

    v, err := client.Evaluators.CreateRemoteVersion(
        context.Background(),
        evaluators.CreateRemoteVersionRequest{
            Evaluator:     "your-evaluator-name-or-id",
            Space:         "your-space-name-or-id",
            CommitMessage: "switch backing integration",
            IntegrationID: "your-evaluator-integration-id",
        },
    )
    if err != nil {
        var badRequest *arize.BadRequestError
        if errors.As(err, &badRequest) {
            log.Fatalf("invalid version: %v", badRequest)
        }
        log.Fatal(err)
    }

    fmt.Printf("created remote evaluator version\n")
    _ = v
}

Skills

MCP server

API reference

API clients

List Evaluators

Get an Evaluator

Create an Evaluator

Update an Evaluator

Delete an Evaluator

List Versions

Delete Versions

Create a Version

Get a Version

Create a Template Evaluator

Create a Code Evaluator

Create a Remote Evaluator

Create a Template Version

Create a Code Version

Create a Remote Version

​List Evaluators

​Get an Evaluator

​Create an Evaluator

​Update an Evaluator

​Delete an Evaluator

​List Versions

​Delete Versions

​Create a Version

​Get a Version

​Create a Template Evaluator

​Create a Code Evaluator

​Create a Remote Evaluator

​Create a Template Version

​Create a Code Version

​Create a Remote Version

List Evaluators

Get an Evaluator

Create an Evaluator

Update an Evaluator

Delete an Evaluator

List Versions

Delete Versions

Create a Version

Get a Version

Create a Template Evaluator

Create a Code Evaluator

Create a Remote Evaluator

Create a Template Version

Create a Code Version

Create a Remote Version