Skip to main content
POST
/
v2
/
annotation-queues
Create an annotation queue
curl --request POST \
  --url https://api.arize.com/v2/annotation-queues \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": [
    "ac_abc123"
  ],
  "annotator_emails": [
    "annotator1@example.com"
  ],
  "assignment_method": "all"
}
'
{
  "id": "aq_abc123",
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "instructions": "Review each response for accuracy and helpfulness",
  "annotation_configs": [
    {
      "id": "ac_001",
      "name": "accuracy",
      "type": "continuous",
      "space_id": "spc_xyz789",
      "minimum_score": 0,
      "maximum_score": 10,
      "optimization_direction": "maximize",
      "created_at": "2024-01-10T08:00:00Z"
    },
    {
      "id": "ac_002",
      "name": "sentiment",
      "type": "categorical",
      "space_id": "spc_xyz789",
      "values": [
        {
          "label": "positive",
          "score": 1
        },
        {
          "label": "neutral",
          "score": 0
        },
        {
          "label": "negative",
          "score": -1
        }
      ],
      "optimization_direction": "maximize",
      "created_at": "2024-01-10T08:00:00Z"
    }
  ],
  "annotators": [
    {
      "id": "usr_001",
      "email": "reviewer@example.com"
    },
    {
      "id": "usr_002",
      "email": "annotator@example.com"
    }
  ],
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-15T10:30:00Z"
}

Authorizations

​
Authorization
string
header
required

Most Arize AI endpoints require authentication. For those endpoints that require authentication, include your API key in the request header using the format

Body

application/json

Body containing annotation queue creation parameters

​
name
string
required

The name of the annotation queue. Must be unique within the space for active queues.

Maximum string length: 255
Example:

"Quality Review Queue"

​
space_id
string
required

The space ID that the annotation queue belongs to

Example:

"spc_xyz789"

​
annotation_config_ids
string[]
required

IDs of annotation configs to associate with this queue. All configs must belong to the same space.

Minimum array length: 1
Example:
["ac_abc123", "ac_def456"]
​
annotator_emails
string<email>[]
required

Email addresses of annotators to assign to the queue. Emails are resolved to user IDs server-side.

Minimum array length: 1

An email address

Example:
[
  "annotator1@example.com",
  "annotator2@example.com"
]
​
instructions
string

Instructions for annotators working on this queue

Maximum string length: 5000
Example:

"Review each response for accuracy and helpfulness"

​
assignment_method
enum<string>
default:all

How records are assigned to annotators. Defaults to "all".

  • all: Every annotator is assigned to every record.
  • random: Each record is randomly assigned to one annotator.
Available options:
all,
random
​
record_sources
object[]

Record sources to add to the annotation queue on creation. At most 2 record sources (projects or datasets) may be provided in a single create request. Additional records from other sources can be added after creation.

Maximum array length: 2

Response

Returns the created annotation queue

​
id
string
required

The unique identifier for the annotation queue

Example:

"aq_abc123"

​
name
string
required

The name of the annotation queue

Example:

"Quality Review Queue"

​
space_id
string
required

The space id the annotation queue belongs to

Example:

"spc_xyz789"

​
annotators
object[]
required

Users assigned as annotators to this queue

​
created_at
string<date-time>
required

The timestamp for when the annotation queue was created

Example:

"2024-01-15T10:30:00Z"

​
updated_at
string<date-time>
required

The timestamp for when the annotation queue was last updated

Example:

"2024-01-20T14:45:00Z"

​
instructions
string | null

The instructions for the annotation queue

Example:

"Review each response for accuracy and helpfulness"

​
annotation_configs
object[]

The annotation configs associated with this queue