Import metrics from arize.pandas.generative.nlp_metrics
Metrics
bleu
BLEU score is typically used to evaluate the quality of machine-translated text from one natural language to another. BLEU calculates scores for individual translated segments, typically sentences, by comparing them to a set of high-quality reference translations. These scores are then averaged over the entire corpus to obtain an estimate of the overall quality of the translation.
Argument
Expected Type
Description
response_col
pd.Series
[Required] Pandas Series containing translations (as strings) to score
references_col
pd.Series
[Required] Pandas Series containing a reference, or list of several references, per prediction
max_order
int
[Optional] Maximum n-gram order to use when computing BLEU score. Defaults to 4
smooth
bool
[Optional] Whether or not to apply Lin et al. 2004 smoothing. Defaults to False
Code Example
bleu_scores = bleu( response_col=df['summary'], references_col=df['reference_summary'], max_order=5, # optional field smooth=True # optional field )
sacre_bleu
A hassle-free computation of shareable, comparable, and reproducible BLEU scores. Inspired by Rico Sennrichās multi-bleu-detok.perl, it produces the official Workshop on Machine Translation (WMT) scores but works with plain text.
Argument
Expected Type
Description
response_col
pd.Series
[Required] Pandas Series containing translations (as strings) to score
references_col
pd.Series
[Required] Pandas Series containing a reference, or list of several references, per prediction Note: There must be the same number of references for each prediction (i.e. all sub-lists must be of the same length)
smooth_method
str
[Optional] The smoothing method to use, defaults to 'exp'. Possible values are: - 'none': no smoothing - 'floor': increment zero counts - 'add-k': increment num/denom by k for n>1 - 'exp': exponential decay
smooth_value
float
[Optional] The smoothing value. Defaults to Nonesmooth_method='floor', smooth_value defaults to 0.1smooth_method='add-k', smooth_value defaults to 1
lowercase
bool
[Optional] If True, lowercases the input, enabling case-insensitivity. Defaults to False
force
bool
[Optional] If True, insists that your tokenized input is actually de-tokenized. Defaults to False
use_effective_order
bool
[Optional] If True, stops including n-gram orders for which precision is 0. This should be True, if sentence-level BLEU will be computed. Defaults to False
Code Example
sacre_bleu_scores = sacre_bleu( response_col=df['summary'], references_col=df['reference_summary'], smooth_method='floor', # optional field smooth_value=0.1, # defaulted to 0.1 since smooth_method is 'floor' lowercase=True # optional field )
google_bleu
BLEU score is typically used as a corpus measure, and it has some limitations when applied to single sentences. To overcome this issue in RL experiments, there exists a variation called the GLEU score. The GLEU score is the minimum of recall and precision.
Argument
Expected Type
Description
response_col
pd.Series
Pandas Series containing translations (as strings) to score
references_col
pd.Series
Pandas Series containing a reference, or list of several references, for each translation
min_len
int
The minimum order of n-gram this function should extract. Defaults to 1
max_len
int
The maximum order of n-gram this function should extract. Defaults to 4
Code Example
google_bleu_scores = google_bleu( response_col=df['summary'], references_col=df['reference_summary'], min_len=2, # optional field max_len=5 # optional field )
rouge
A software package and a set of metrics commonly used to evaluate machine translation and automatic summarization software in natural language processing. These metrics involve comparing a machine-produced summary or translation with a reference or set of references that have been human-produced.
Argument
Expected Type
Description
response_col
pd.Series
[Required] Pandas Series containing predictions (as strings) to score
references_col
pd.Series
[Required] Pandas Series containing a reference, or list of several references, per prediction
rogue_types
List[str]
[Optional] A list of rouge types to calculate. Defaults to ['rougeL']. Valid rogue types: -rogue1: unigram (1-gram) based scoring -rogue2: bigram (2-gram) based scoring-rogueL: longest common subsequence based scoring -rogueLSum: splits text using ā/nā
use_stemmer
bool
[Optional] If True, uses Porter stemmer to strip word suffixes. Defaults to False.
An automatic metric typically used to evaluate machine translation, which is based on a generalized concept of unigram matching between the machine-produced translation and the reference human-produced translations.
Argument
Expected Type
Description
response_col
pd.Series
[Required] Pandas Series containing predictions (as strings) to score
references_col
pd.Series
[Required] Pandas Series containing a reference, or list of several references, per prediction
alpha
float
[Optional] Parameter for controlling relative weights of precision and recall. Default is 0.9
beta
float
[Optional] Parameter for controlling shape of penalty as a function of fragmentation. Default is 3
gamma
float
[Optional] The relative weight assigned to fragmentation penalty. Default is 0.5
Code Example
meteor_scores = meteor( response_col=df['summary'], references_col=df['reference_summary'], alpha=0.8, # optional field beta=4, # optional field gamma=0.4 # optional field )
Was this page helpful?
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"Arize AX Docs","colors":{"primary":"#EA338A","light":"#FF3CA8","dark":"#FF3CA8"},"logo":{"light":"https://mintcdn.com/arize-ax/V-uvingrGLOCK4VD/logo/light.svg?fit=max\u0026auto=format\u0026n=V-uvingrGLOCK4VD\u0026q=85\u0026s=cc58ee9388d5b5253d39b65cc662908f","dark":"https://mintcdn.com/arize-ax/V-uvingrGLOCK4VD/logo/dark.svg?fit=max\u0026auto=format\u0026n=V-uvingrGLOCK4VD\u0026q=85\u0026s=b16a0266f5913934131301a1b60af51e","href":"https://arize.com"},"favicon":"https://mintcdn.com/arize-ax/1gcABHkqP5TIGbru/favicon.ico?fit=max\u0026auto=format\u0026n=1gcABHkqP5TIGbru\u0026q=85\u0026s=aef538e87b54a3c819583bc40acfc2f8","appearance":{"default":"dark"},"background":{"color":{"light":"#F8F8F6","dark":"#0E0C0B"}},"navbar":{"links":[],"primary":{"type":"button","label":"Get started","href":"https://app.arize.com/auth/join"}},"footer":{"socials":{"x":"https://x.com/arizeai","bluesky":"https://bsky.app/profile/arize.bsky.social","threads":"https://www.threads.com/@arize.ai","instagram":"https://www.instagram.com/arize_ai","linkedin":"https://www.linkedin.com/company/arizeai","youtube":"https://youtube.com/@arizeai","slack":"https://arize-ai.slack.com/ssb/redirect","website":"https://fosstodon.org/@arizeai"},"links":[{"header":"Support","items":[{"label":"Chat Us On Slack","href":"https://arize-ai.slack.com/ssb/redirect"},{"label":"support@arize.com","href":"mailto:support@arize.com"},{"label":"System Status","href":"https://status.arize.com"}]},{"header":"Get Started","items":[{"label":"Signup For Free","href":"https://arize.com/join"},{"label":"Book A Demo","href":"https://arize.com/request-a-demo"}]},{"header":"Resources","items":[{"label":"Blog","href":"https://arize.com/blog"}]}]},"seo":{"metatags":{"canonical":"https://arize.com/docs"},"indexing":"all"},"fonts":{"heading":{"family":"Geist","weight":600},"body":{"family":"Geist"}},"styling":{"codeblocks":{"theme":{"light":"github-light-default","dark":"github-dark-default"}},"latex":true},"integrations":{"gtm":{"tagId":"GTM-W96XM35"}},"contextual":{"options":["copy","view","chatgpt","claude"],"display":"header"},"interaction":{"drilldown":false},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"arize-ax","hasAudioNarration":false,"actualSubdomain":"arize-ax","customDomains":[],"gitSource":{"type":"github","owner":"arize-ai","repo":"docs","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"feedback":{"thumbs":true,"contextualFeedback":{"allowEmail":true,"emailDisclaimer":"We will only use your email to follow up with you about this feedback."}},"entitlements":{"AI_CHAT":{"status":"ENABLED"},"REMOVE_BRANDING":{"status":"ENABLED"},"CONTEXTUAL_FEEDBACK":{"status":"ENABLED"},"CODE_SNIPPET_FEEDBACK":{"status":"ENABLED"},"WEB_MCP":{"status":"ENABLED"},"NARRATION":{"status":"ENABLED"}},"buildId":"6b034b209ec73fbdf8ac7a63","clientVersion":"0.0.4271","searchFilterCounts":[{"language":"default","tag":"APIs, SDKs, Skills \u0026 MCP","count":375},{"language":"default","tag":"Get Started","count":256},{"language":"default","tag":"API reference","count":220},{"language":"default","tag":"REST","count":187},{"language":"default","tag":"Endpoints","count":184},{"language":"default","tag":"API clients","count":143},{"language":"default","tag":"Machine Learning","count":139},{"language":"default","tag":"ML observability","count":139},{"language":"default","tag":"Integrations","count":129},{"language":"default","tag":"Python SDK","count":67}]},"assistantConfig":{"enableAskAiButton":true,"enableStarterQuestions":true,"enableAskAiSearchBar":true,"deflection":{"enabled":true,"email":"support@arize.com"}},"basePath":"/docs","hasChatPermissions":true,"hasWebMcpPermissions":true,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"site-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"treatment","enabled":true,"payload":{"type":"string","value":"treatment"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"starterQuestions":[{"_id":"6944513fe59e82150b5700b3","deploymentId":"6894ef048eafbd2a7c20796e","questionText":"Need more help? Ask our team in #arize-ax-platform-support on community Slack.","status":"active","generatedBy":"manual","createdAt":"2025-12-18T19:08:47.802Z","updatedAt":"2025-12-18T19:15:01.436Z","__v":0}],"googleFontsInlined":true},"customCss":[{"filePath":"style.css","content":"/* cyrillic-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrodmgPn.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrMdmgPn.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* symbols2 */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFg08vz7ehw.woff2) format('woff2');\n unicode-range: U+2000-2001, U+2004-2008, U+200A, U+23B8-23BD, U+2500-259F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrgdmgPn.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrkdmgPn.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrcdmg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrodmgPn.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrMdmgPn.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* symbols2 */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFg08vz7ehw.woff2) format('woff2');\n unicode-range: U+2000-2001, U+2004-2008, U+200A, U+23B8-23BD, U+2500-259F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrgdmgPn.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrkdmgPn.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrcdmg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrodmgPn.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrMdmgPn.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* symbols2 */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFg08vz7ehw.woff2) format('woff2');\n unicode-range: U+2000-2001, U+2004-2008, U+200A, U+23B8-23BD, U+2500-259F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrgdmgPn.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrkdmgPn.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrcdmg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrodmgPn.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrMdmgPn.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* symbols2 */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFg08vz7ehw.woff2) format('woff2');\n unicode-range: U+2000-2001, U+2004-2008, U+200A, U+23B8-23BD, U+2500-259F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrgdmgPn.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrkdmgPn.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Geist Mono';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/docs/_mintlify/assets/fonts/google/s/geistmono/v6/or3nQ6H-1_WfwkMZI_qYFrcdmg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\nhtml:root{--font-family-mono-custom: 'Geist Mono'}html:root{--gray-50: 248 248 246;--gray-100: 242 242 237;--gray-200: 230 230 223;--gray-300: 205 205 200;--gray-400: 179 179 191;--gray-500: 138 138 150;--gray-600: 115 115 140;--gray-700: 69 69 84;--gray-800: 35 34 58;--gray-900: 18 18 33;--gray-950: 18 18 33}html.dark{--gray-50: 252 251 247;--gray-100: 252 251 247;--gray-200: 207 206 221;--gray-300: 200 199 214;--gray-400: 166 165 188;--gray-500: 138 137 150;--gray-600: 92 88 92;--gray-700: 58 52 50;--gray-800: 36 31 29;--gray-900: 23 20 19;--gray-950: 14 12 11}html:root{--ax-radius-sm: 2px;--ax-radius-md: 6px;--ax-radius-lg: 8px;--ax-accent-bg: #FF3CA8;--ax-panel: #CDBEF7;--ax-panel-2: #E3DBFC;--ax-panel-code: #FFFFFF;--ax-panel-border: rgba(19, 22, 64, 0.22);--ax-panel-icon: #4827C1;--ax-panel-t1: #121221;--ax-panel-t2: #2A2A38;--ax-panel-accent: #6E1046;--ax-panel-code-kw: #3F4110;--ax-panel-code-flag: #5A3200;--ax-panel-code-url: #004A70;--ax-panel-selection: #FFE9F4}html:not(.dark){--ax-bg: #F8F8F6;--ax-surface: #FBFBF9;--ax-code-bg: #F5F5ED;--ax-chip: #EDEDE2;--ax-line: rgba(19, 22, 64, 0.15);--ax-t1: #121221;--ax-t2: #454554;--ax-t3: #73738C;--ax-accent: #EA338A;--ax-accent-line: rgba(234, 51, 138, 0.55);--ax-accent-soft: #FEEDF3;--ax-on-accent: #FFFFFF;--ax-note: #EFF4D2;--ax-note-line: rgba(103, 115, 0, 0.24);--ax-note-t: #677300;--ax-warn: #FFE9CC;--ax-warn-line: rgba(193, 100, 0, 0.24);--ax-warn-t: #C16400;--ax-blue: #E7F3FD;--ax-blue-line: rgba(0, 117, 177, 0.24);--ax-blue-t: #0075B1;--ax-danger: #FFEBE9;--ax-danger-line: rgba(207, 34, 46, 0.28);--ax-danger-t: #CF222E}html.dark{--ax-bg: #0E0C0B;--ax-surface: #171413;--ax-code-bg: #040302;--ax-chip: #241F1D;--ax-line: rgba(207, 206, 221, 0.16);--ax-t1: #FCFBF7;--ax-t2: #C8C7D6;--ax-t3: #A6A5BC;--ax-accent: #FF3CA8;--ax-accent-line: rgba(255, 60, 168, 0.45);--ax-accent-soft: #38001D;--ax-on-accent: #FFFFFF;--ax-note: #181A00;--ax-note-line: rgba(199, 221, 50, 0.30);--ax-note-t: #C7DD32;--ax-warn: #412100;--ax-warn-line: rgba(255, 194, 71, 0.30);--ax-warn-t: #FFC247;--ax-blue: #04283D;--ax-blue-line: rgba(57, 195, 255, 0.30);--ax-blue-t: #97D8FF;--ax-danger: rgba(207, 34, 46, 0.18);--ax-danger-line: rgba(207, 34, 46, 0.45);--ax-danger-t: #FFEBE9}#content{line-height:1.75}#page-title,#content h1{font-size:clamp(2rem, 1.4rem + 2.1vw, 2.875rem);line-height:1.12;letter-spacing:-0.02em;font-weight:600}#content h2,#content h3,#content h4{letter-spacing:-0.01em;font-weight:600}#content code,#content kbd,#content samp{font-feature-settings:'tnum' 1}#content a:not(.card){color:var(--ax-accent);text-decoration-thickness:1px;text-underline-offset:3px}:focus-visible{outline:2px solid var(--ax-accent-line);outline-offset:2px}html.dark :focus-visible{outline-color:var(--ax-accent-line)}::selection{background:var(--ax-accent-soft)}#navbar{background:var(--ax-bg);border-bottom:1px solid var(--ax-line)}#navbar div:has(>.nav-tabs){position:relative}#navbar div:has(>.nav-tabs)::before{content:'';position:absolute;top:0;left:50%;width:100vw;margin-left:-50vw;border-top:1px solid var(--ax-line);pointer-events:none}.nav-tabs-item[data-active]{box-shadow:inset 0 -2px 0 var(--ax-accent)}.ax-product-selector{position:relative;display:inline-flex;align-items:stretch;flex:0 0 auto}.ax-product-selector--desktop{margin-left:0.75rem;align-self:center}.page-context-dropdown__label{font-family:inherit;font-weight:400}.page-context-dropdown__chevron>span{display:inline-block;transition:transform 150ms ease}.page-context-dropdown__chevron svg{display:block;width:1rem;height:1rem}.page-context-dropdown__chevron[aria-expanded=\"true\"]>span{transform:rotate(180deg)}.page-context-dropdown__menu{position:absolute;z-index:60;top:calc(100% + 0.55rem);right:0;min-width:12.5rem;padding:0.4rem;background:var(--ax-bg);border:1px solid var(--ax-line);border-radius:var(--ax-radius-md);box-shadow:0 0.75rem 1.5rem rgb(18 18 33 / 0.18)}.ax-product-selector--desktop .page-context-dropdown__menu{right:auto;left:0}.page-context-dropdown__menu[hidden]{display:none}.page-context-dropdown__item{display:flex;align-items:center;justify-content:space-between;min-height:2.5rem;padding:0.55rem 0.65rem;color:var(--ax-t1);border-radius:var(--ax-radius-sm);font-size:0.9375rem;font-weight:400;text-decoration:none;transition:background-color 150ms ease, color 150ms ease}.page-context-dropdown__item:hover,.page-context-dropdown__item:focus-visible{color:var(--ax-accent);background:var(--ax-accent-soft)}.page-context-dropdown__item[data-selected=\"true\"]{color:var(--ax-accent);font-weight:500}.page-context-dropdown__item[data-selected=\"true\"]::after{content:'ā';margin-left:1.5rem;color:var(--ax-accent);font-size:0.875rem}@media (prefers-reduced-motion: reduce){.page-context-dropdown__chevron>span,.page-context-dropdown__item{transition:none}}@media (max-width: 767px){#navbar .ax-product-selector--desktop{display:none}#mobile-nav-content .ax-product-selector--mobile{margin:0.5rem 0 1rem}#mobile-nav-content .page-context-dropdown__label{flex:1 1 auto}#mobile-nav-content .page-context-dropdown__chevron{flex:0 0 auto}#mobile-nav-content .page-context-dropdown__menu{position:static;margin-top:0.4rem}}#topbar-cta-button{background:var(--ax-accent-bg);color:var(--ax-on-accent);border-radius:var(--ax-radius-md);border:none}#topbar-cta-button span,#topbar-cta-button svg{color:var(--ax-on-accent)!important}#search-bar-entry{background:var(--ax-surface);border:1px solid var(--ax-line);border-radius:var(--ax-radius-md)}#assistant-entry{background:var(--ax-surface);border:1px solid var(--ax-accent-line);border-radius:var(--ax-radius-md)}#assistant-entry svg{color:var(--ax-accent)}#page-context-menu,.page-context-dropdown{font-size:0.9375rem;align-items:stretch}#page-context-menu-button,.page-context-dropdown__label{display:inline-flex;align-items:center;padding:10px 14px;border-radius:var(--ax-radius-md) 0 0 var(--ax-radius-md);background:var(--ax-surface);border-color:var(--ax-line);color:var(--ax-t1);border:1px solid var(--ax-line);text-decoration:none}#page-context-menu-button>div{font-size:0.9375rem;gap:9px}#page-context-menu>button[aria-haspopup],.page-context-dropdown__chevron{height:auto;align-self:stretch;aspect-ratio:auto;padding:10px 11px;border-radius:0 var(--ax-radius-md) var(--ax-radius-md) 0;background:var(--ax-surface);border-color:var(--ax-line);color:var(--ax-t3);border:1px solid var(--ax-line);border-left:0;cursor:pointer}#sidebar-content li[data-active]>a{background:var(--ax-accent-soft);color:var(--ax-accent);border-radius:var(--ax-radius-md);font-weight:500}.sidebar-group-header{font-weight:600;color:var(--ax-t1)}.toc-item[data-active],.toc-item[data-active] a{color:var(--ax-accent);font-weight:400}[data-component-part=\"step-content\"] .card:has(a[href=\"/ax/set-up-with-ai-assistants\"]) .size-6[class*=\"fill-gray-800\"]{color:var(--ax-accent-bg)!important;fill:var(--ax-accent-bg)!important}[data-component-part=\"callout-icon\"] svg[style*=\"regular/zap.svg\"]{background-color:var(--ax-accent-bg)!important}.card.card{background:var(--ax-surface);border-color:var(--ax-line);border-radius:var(--ax-radius-lg);box-shadow:none}.frame.frame{background:var(--ax-surface);border:1px solid var(--ax-line);border-radius:var(--ax-radius-lg)}.accordion.accordion,.accordion-group.accordion-group{border-radius:var(--ax-radius-md);border-color:var(--ax-line)}.accordion.accordion{background:var(--ax-surface)}.code-block.code-block,.code-group.code-group{background:var(--ax-code-bg);border-color:var(--ax-line);border-radius:var(--ax-radius-md)}.code-block-background{background:var(--ax-code-bg)!important}html.dark .code-block-background{background:var(--ax-code-bg)!important}#content :not(pre)>code{background:var(--ax-chip);border-radius:var(--ax-radius-sm)}.code-block:has([data-floating-buttons]) [data-fade-overlay]{background-image:none}[data-floating-buttons] button{background:var(--ax-chip)}.callout .code-block.code-block,.callout .code-group.code-group,.callout .code-block-background,html.dark .callout .code-block-background{background:var(--ax-bg)!important}.callout{border-radius:var(--ax-radius-lg)}.callout[data-callout-type=\"callout\"],.callout[data-callout-type=\"info\"]{background:var(--ax-surface);border-color:var(--ax-line)}.callout[data-callout-type=\"callout\"] [data-component-part=\"callout-content\"],.callout[data-callout-type=\"info\"] [data-component-part=\"callout-content\"]{color:var(--ax-t2)!important}.callout[data-callout-type=\"tip\"],.callout[data-callout-type=\"check\"]{background:var(--ax-note);border-color:var(--ax-note-line)}.callout[data-callout-type=\"tip\"] [data-component-part=\"callout-content\"],.callout[data-callout-type=\"check\"] [data-component-part=\"callout-content\"]{color:var(--ax-note-t)!important}.callout[data-callout-type=\"warning\"]{background:var(--ax-warn);border-color:var(--ax-warn-line)}.callout[data-callout-type=\"warning\"] [data-component-part=\"callout-content\"]{color:var(--ax-warn-t)!important}.callout[data-callout-type=\"note\"]{background:var(--ax-blue);border-color:var(--ax-blue-line)}.callout[data-callout-type=\"note\"] [data-component-part=\"callout-content\"]{color:var(--ax-blue-t)!important}.callout[data-callout-type=\"danger\"]{background:var(--ax-danger);border-color:var(--ax-danger-line)}.callout[data-callout-type=\"danger\"] [data-component-part=\"callout-content\"]{color:var(--ax-danger-t)!important}[data-component-part=\"step-number\"]>div{background:var(--ax-accent-soft);border:1px solid var(--ax-accent-line);color:var(--ax-accent);font-weight:500}.update.update-container{flex-direction:column;gap:0.5rem;border-top:1px solid var(--ax-line)}.update.update-container:first-of-type{border-top:none}.update.update-container>div:first-child{width:100%;position:static}.update.update-container>div:last-child{width:100%;max-width:none}#content th{color:var(--ax-t1);font-weight:600;border-bottom:1px solid var(--ax-line)}#content td{border-bottom:1px solid var(--ax-line)}footer{border-top:1px solid var(--ax-line)}.ax-hero .callout{background:var(--ax-panel);border-color:var(--ax-panel-border);border-radius:var(--ax-radius-lg)}.ax-hero .callout [data-component-part=\"callout-content\"],.ax-hero .callout p,.ax-hero .callout li{color:var(--ax-panel-t2)!important}.ax-hero .callout h2,.ax-hero .callout h3,.ax-hero .callout strong{color:var(--ax-panel-t1)!important}.ax-hero .callout [data-component-part=\"callout-icon\"] svg{background:var(--ax-panel-icon)!important}.ax-hero .code-block,.ax-hero .code-group,.ax-hero .code-block-background{background:var(--ax-panel-code)!important;border-color:var(--ax-panel-border)}.ax-hero .code-block,.ax-hero .code-group{padding:0;overflow:hidden}html.dark .ax-hero .code-block,html.dark .ax-hero .code-group,html.dark .ax-hero .code-block-background,html.dark .ax-hero .shiki,html:not(.dark) .ax-hero .code-block,html:not(.dark) .ax-hero .code-group,html:not(.dark) .ax-hero .code-block-background{background:var(--ax-panel-code)!important}html.dark .ax-hero .shiki,html.dark .ax-hero .shiki span{color:var(--ax-panel-t1)!important}html.dark .ax-hero .shiki span[style*=\"#FFA657\"]{color:var(--ax-panel-code-kw)!important}html.dark .ax-hero .shiki span[style*=\"#79C0FF\"]{color:var(--ax-panel-code-flag)!important}html.dark .ax-hero .shiki span[style*=\"#A5D6FF\"]{color:var(--ax-panel-code-url)!important}.ax-hero .code-group>div:first-child,html.dark .ax-hero .code-group>div:first-child{background:var(--ax-panel-2)!important;border-bottom:1px solid var(--ax-panel-border)}.ax-hero .code-group [role=\"tab\"]{color:var(--ax-panel-t2)!important}.ax-hero .code-group [role=\"tab\"][aria-selected=\"true\"]{color:var(--ax-panel-accent)!important}.ax-hero .code-group [role=\"tab\"][aria-selected=\"true\"]>div:last-child{background:var(--ax-panel-accent)!important}.ax-hero .code-block svg,.ax-hero .code-group svg{color:var(--ax-panel-t2)}html.dark .ax-hero [data-fade-overlay]{background-image:linear-gradient(\n to right,\n rgb(255 255 255 / 0) 0,\n rgb(255 255 255 / 0.2) 10px,\n rgb(255 255 255 / 0.5) 25px,\n rgb(255 255 255 / 0.8) 35px,\n rgb(255 255 255) 45px\n )}.ax-hero .callout a{color:var(--ax-panel-accent)!important}.ax-hero::selection,.ax-hero ::selection{background:var(--ax-panel-selection)}.card-group .card img{max-height:197px!important}.card-group[class$=\"-2\"] .card img{height:160px!important}.card-group[class$=\"-3\"] .card img{height:150px!important}table{display:table!important}.step-title{font-size:1.5rem!important}"}]}
{"page":{"pagePath":"api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation","slug":"/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation","pageMetadata":{"title":"NLP_Metrics","sidebarTitle":"NLPĀ Metrics","description":"Install extra dependencies to compute LLM evaluation metrics","href":"/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Install extra dependencies to compute LLM evaluation metrics\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,..._provideComponents(),...props.components},{CodeBlock,Heading,OptimizedImage,Table,Warning}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!OptimizedImage)_missingMdxReference(`OptimizedImage`,true);if(!Table)_missingMdxReference(`Table`,true);if(!Warning)_missingMdxReference(`Warning`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[_jsx(OptimizedImage,{className:`mint-inline mint-m-0`,src:`https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/9e9834e4-image.jpeg`,inline:true}),` `,_jsx(_components.a,{href:`https://github.com/Arize-ai/client_python/blob/main/src/arize/pandas/generative/nlp_metrics/hf_metrics.py`,children:`View Source on GitHub`})]}),`\n`,_jsx(Warning,{children:_jsxs(_components.p,{children:[_jsx(OptimizedImage,{className:`mint-inline mint-m-0`,src:`https://storage.googleapis.com/arize-phoenix-assets/assets/images/arize-docs-images/7539e0ab-python-3_8-red.svg`,inline:true}),` minimum required for NLP Metrics`]})}),`\n`,_jsx(_components.p,{children:`Install extra dependencies in the SDK:`}),`\n`,_jsx(CodeBlock,{numberOfLines:`2`,language:`text`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`text`,children:_jsxs(_components.code,{language:`text`,numberOfLines:`2`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{children:`%pip install -q \"arize[NLP_Metrics]\u003c8.0.0\"`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{})})]})})}),`\n`,_jsxs(_components.p,{children:[`Import metrics from `,_jsx(_components.code,{children:`arize.pandas.generative.nlp_metrics`})]}),`\n`,_jsx(Heading,{level:`3`,id:`metrics`,children:`Metrics`}),`\n`,_jsx(Heading,{level:`4`,id:`bleu`,children:_jsx(_components.code,{children:`bleu`})}),`\n`,_jsx(_components.p,{children:`BLEU score is typically used to evaluate the quality of machine-translated text from one natural language to another. BLEU calculates scores for individual translated segments, typically sentences, by comparing them to a set of high-quality reference translations. These scores are then averaged over the entire corpus to obtain an estimate of the overall quality of the translation.`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Argument`}),_jsx(_components.th,{children:`Expected Type`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`response_col`})}),_jsx(_components.td,{children:`pd.Series`}),_jsx(_components.td,{children:`[Required] Pandas Series containing translations (as strings) to score`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`references_col`})}),_jsx(_components.td,{children:`pd.Series`}),_jsx(_components.td,{children:`[Required] Pandas Series containing a reference, or list of several references, per prediction`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`max_order`})}),_jsx(_components.td,{children:`int`}),_jsxs(_components.td,{children:[`[Optional] Maximum n-gram order to use when computing BLEU score. Defaults to `,_jsx(_components.code,{children:`4`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`smooth`})}),_jsx(_components.td,{children:`bool`}),_jsxs(_components.td,{children:[`[Optional] Whether or not to apply Lin et al. 2004 smoothing. Defaults to `,_jsx(_components.code,{children:`False`})]})]})]})]}),`\n`,_jsx(Heading,{level:`4`,id:`code-example`,children:`Code Example`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`6`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`bleu_scores`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` =`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` bleu`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` response_col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`df['summary'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` references_col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`df['reference_summary'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` max_order`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`5,`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # optional field`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` smooth`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`True`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # optional field`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`4`,id:`sacre_bleu`,children:_jsx(_components.code,{children:`sacre_bleu`})}),`\n`,_jsx(_components.p,{children:`A hassle-free computation of shareable, comparable, and reproducible BLEU scores. Inspired by Rico Sennrichās multi-bleu-detok.perl, it produces the official Workshop on Machine Translation (WMT) scores but works with plain text.`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Argument`}),_jsx(_components.th,{children:`Expected Type`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`response_col`})}),_jsx(_components.td,{children:`pd.Series`}),_jsx(_components.td,{children:`[Required] Pandas Series containing translations (as strings) to score`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`references_col`})}),_jsx(_components.td,{children:`pd.Series`}),_jsxs(_components.td,{children:[`[Required] Pandas Series containing a reference, or list of several references, per prediction `,_jsx(_components.strong,{children:`Note:`}),` There must be the same number of references for each prediction (i.e. all sub-lists must be of the same length)`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`smooth_method`})}),_jsx(_components.td,{children:`str`}),_jsxs(_components.td,{children:[`[Optional] The smoothing method to use, defaults to `,_jsx(_components.code,{children:`'exp'`}),`. Possible values are: - `,_jsx(_components.code,{children:`'none'`}),`: no smoothing - `,_jsx(_components.code,{children:`'floor'`}),`: increment zero counts - `,_jsx(_components.code,{children:`'add-k'`}),`: increment num/denom by k for n>1 - `,_jsx(_components.code,{children:`'exp'`}),`: exponential decay`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`smooth_value`})}),_jsx(_components.td,{children:`float`}),_jsxs(_components.td,{children:[`[Optional] The smoothing value. Defaults to `,_jsx(_components.code,{children:`None`}),` `,_jsx(_components.code,{children:`smooth_method='floor'`}),`, `,_jsx(_components.code,{children:`smooth_value`}),` defaults to `,_jsx(_components.code,{children:`0.1`}),` `,_jsx(_components.code,{children:`smooth_method='add-k'`}),`, `,_jsx(_components.code,{children:`smooth_value`}),` defaults to `,_jsx(_components.code,{children:`1`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`lowercase`})}),_jsx(_components.td,{children:`bool`}),_jsxs(_components.td,{children:[`[Optional] If True, lowercases the input, enabling case-insensitivity. Defaults to `,_jsx(_components.code,{children:`False`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`force`})}),_jsx(_components.td,{children:`bool`}),_jsxs(_components.td,{children:[`[Optional] If True, insists that your tokenized input is actually de-tokenized. Defaults to `,_jsx(_components.code,{children:`False`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`use_effective_order`})}),_jsx(_components.td,{children:`bool`}),_jsxs(_components.td,{children:[`[Optional] If True, stops including n-gram orders for which precision is 0. This should be True, if sentence-level BLEU will be computed. Defaults to `,_jsx(_components.code,{children:`False`})]})]})]})]}),`\n`,_jsx(Heading,{level:`4`,id:`code-example-2`,children:`Code Example`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`7`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`7`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`sacre_bleu_scores`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` =`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` sacre_bleu`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` response_col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`df['summary'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` references_col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`df['reference_summary'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` smooth_method`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`'floor',`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # optional field`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` smooth_value`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`0.1,`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # defaulted to 0.1 since smooth_method is 'floor'`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` lowercase`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`True`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # optional field`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`4`,id:`google_bleu`,children:_jsx(_components.code,{children:`google_bleu`})}),`\n`,_jsx(_components.p,{children:`BLEU score is typically used as a corpus measure, and it has some limitations when applied to single sentences. To overcome this issue in RL experiments, there exists a variation called the GLEU score. The GLEU score is the minimum of recall and precision.`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Argument`}),_jsx(_components.th,{children:`Expected Type`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`response_col`})}),_jsx(_components.td,{children:`pd.Series`}),_jsx(_components.td,{children:`Pandas Series containing translations (as strings) to score`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`references_col`})}),_jsx(_components.td,{children:`pd.Series`}),_jsx(_components.td,{children:`Pandas Series containing a reference, or list of several references, for each translation`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`min_len`})}),_jsx(_components.td,{children:`int`}),_jsxs(_components.td,{children:[`The minimum order of n-gram this function should extract. Defaults to `,_jsx(_components.code,{children:`1`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`max_len`})}),_jsx(_components.td,{children:`int`}),_jsxs(_components.td,{children:[`The maximum order of n-gram this function should extract. Defaults to `,_jsx(_components.code,{children:`4`})]})]})]})]}),`\n`,_jsx(Heading,{level:`4`,id:`code-example-3`,children:`Code Example`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`6`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`google_bleu_scores`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` =`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` google_bleu`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` response_col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`df['summary'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` references_col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`df['reference_summary'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` min_len`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`2,`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # optional field`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` max_len`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`5`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # optional field`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`4`,id:`rouge`,children:_jsx(_components.code,{children:`rouge`})}),`\n`,_jsx(_components.p,{children:`A software package and a set of metrics commonly used to evaluate machine translation and automatic summarization software in natural language processing. These metrics involve comparing a machine-produced summary or translation with a reference or set of references that have been human-produced.`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Argument`}),_jsx(_components.th,{children:`Expected Type`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`response_col`})}),_jsx(_components.td,{children:`pd.Series`}),_jsx(_components.td,{children:`[Required] Pandas Series containing predictions (as strings) to score`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`references_col`})}),_jsx(_components.td,{children:`pd.Series`}),_jsx(_components.td,{children:`[Required] Pandas Series containing a reference, or list of several references, per prediction`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`rogue_types`})}),_jsx(_components.td,{children:`List[str]`}),_jsxs(_components.td,{children:[`[Optional] A list of rouge types to calculate. Defaults to `,_jsx(_components.code,{children:`['rougeL']`}),`. Valid rogue types: -`,_jsx(_components.code,{children:`rogue1`}),`: unigram (1-gram) based scoring -`,_jsx(_components.code,{children:`rogue2`}),`: bigram (2-gram) based scoring-`,_jsx(_components.code,{children:`rogueL`}),`: longest common subsequence based scoring -`,_jsx(_components.code,{children:`rogueLSum`}),`: splits text using ā/nā`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`use_stemmer`})}),_jsx(_components.td,{children:`bool`}),_jsxs(_components.td,{children:[`[Optional] If True, uses Porter stemmer to strip word suffixes. Defaults to `,_jsx(_components.code,{children:`False`}),`.`]})]})]})]}),`\n`,_jsx(Heading,{level:`4`,id:`code-example-4`,children:`Code Example`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`5`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`rouge_scores`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` =`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` rouge`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` response_col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`df['summary'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` references_col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`df['reference_summary'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` rouge_types`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`['rouge1',`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` 'rouge2',`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` 'rougeL',`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` 'rougeLsum'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# optional field`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`4`,id:`meteor`,children:_jsx(_components.code,{children:`meteor`})}),`\n`,_jsx(_components.p,{children:`An automatic metric typically used to evaluate machine translation, which is based on a generalized concept of unigram matching between the machine-produced translation and the reference human-produced translations.`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Argument`}),_jsx(_components.th,{children:`Expected Type`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`response_col`})}),_jsx(_components.td,{children:`pd.Series`}),_jsx(_components.td,{children:`[Required] Pandas Series containing predictions (as strings) to score`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`references_col`})}),_jsx(_components.td,{children:`pd.Series`}),_jsx(_components.td,{children:`[Required] Pandas Series containing a reference, or list of several references, per prediction`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`alpha`})}),_jsx(_components.td,{children:`float`}),_jsxs(_components.td,{children:[`[Optional] Parameter for controlling relative weights of precision and recall. Default is `,_jsx(_components.code,{children:`0.9`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`beta`})}),_jsx(_components.td,{children:`float`}),_jsxs(_components.td,{children:[`[Optional] Parameter for controlling shape of penalty as a function of fragmentation. Default is `,_jsx(_components.code,{children:`3`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`gamma`})}),_jsx(_components.td,{children:`float`}),_jsxs(_components.td,{children:[`[Optional] The relative weight assigned to fragmentation penalty. Default is `,_jsx(_components.code,{children:`0.5`})]})]})]})]}),`\n`,_jsx(Heading,{level:`4`,id:`code-example-5`,children:`Code Example`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`7`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`7`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`meteor_scores`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` =`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` meteor`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` response_col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`df['summary'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` references_col`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`df['reference_summary'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` alpha`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`0.8,`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # optional field`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` beta`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`4,`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # optional field`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` gamma`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`0.4`}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # optional field`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`]})})})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"codeStyling":{"theme":{"light":"github-light-default","dark":"github-dark-default"}},"config":{},"pageMetadata":{"title":"NLP_Metrics","sidebarTitle":"NLPĀ Metrics","description":"Install extra dependencies to compute LLM evaluation metrics","href":"/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Metrics","slug":"metrics","depth":3,"children":[]},{"title":"bleu","slug":"bleu","depth":4,"children":[]},{"title":"Code Example","slug":"code-example","depth":4,"children":[]},{"title":"sacre_bleu","slug":"sacre_bleu","depth":4,"children":[]},{"title":"Code Example","slug":"code-example-2","depth":4,"children":[]},{"title":"google_bleu","slug":"google_bleu","depth":4,"children":[]},{"title":"Code Example","slug":"code-example-3","depth":4,"children":[]},{"title":"rouge","slug":"rouge","depth":4,"children":[]},{"title":"Code Example","slug":"code-example-4","depth":4,"children":[]},{"title":"meteor","slug":"meteor","depth":4,"children":[]},{"title":"Code Example","slug":"code-example-5","depth":4,"children":[]}],"codeExamples":{}},"jsLinks":["/docs/_mintlify/custom/arize-ax/6a30b7a67da02161/analytics.js","/docs/_mintlify/custom/arize-ax/67a4d27b62fababa/product-selector.js"],"dynamicCss":".mint-m-0 {\n margin: 0px\n}\n.mint-inline {\n display: inline\n}","subdomain":"arize-ax","siteName":"Arize AX Docs","title":"NLP_Metrics - Arize AX Docs","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-11-11T22:06:22.479Z","originalFileLocation":"api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"dark\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Install extra dependencies to compute LLM evaluation metrics\">\u003cmeta data-mint-head name=\"application-name\" content=\"Arize AX Docs\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"NLP_Metrics - Arize AX Docs\">\u003cmeta data-mint-head property=\"og:description\" content=\"Install extra dependencies to compute LLM evaluation metrics\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"Arize AX Docs\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://arize-ax.mintlify.site/docs/_mintlify/api/og/v2?division=Pandas+Batch+Logging\u0026amp;title=NLP_Metrics\u0026amp;description=Install+extra+dependencies+to+compute+LLM+evaluation+metrics\u0026amp;theme=972bd1d2c11db90cc08206e1\u0026amp;signature=fae76f649ce28cb87ef5a3cb4053b0aa140346918be7d4d11807eac51167b73d\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"NLP_Metrics - Arize AX Docs\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Install extra dependencies to compute LLM evaluation metrics\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://arize-ax.mintlify.site/docs/_mintlify/api/og/v2?division=Pandas+Batch+Logging\u0026amp;title=NLP_Metrics\u0026amp;description=Install+extra+dependencies+to+compute+LLM+evaluation+metrics\u0026amp;theme=972bd1d2c11db90cc08206e1\u0026amp;signature=fae76f649ce28cb87ef5a3cb4053b0aa140346918be7d4d11807eac51167b73d\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/docs/mintlify-assets/_mintlify/favicons/arize-ax/vxzD83QsbAzzm43O/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"Arize AX Docs\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#EA338A\">\u003clink data-mint-head rel=\"canonical\" href=\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/docs/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://arize.com/#organization\",\"name\":\"Arize AX Docs\",\"url\":\"https://arize.com\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/arize-ax/V-uvingrGLOCK4VD/logo/light.svg?fit=max\u0026auto=format\u0026n=V-uvingrGLOCK4VD\u0026q=85\u0026s=cc58ee9388d5b5253d39b65cc662908f\"}},{\"@type\":\"WebSite\",\"@id\":\"https://arize.com/docs#website\",\"name\":\"Arize AX Docs\",\"url\":\"https://arize.com/docs\",\"publisher\":{\"@id\":\"https://arize.com/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation#webpage\",\"url\":\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation\",\"name\":\"NLP_Metrics\",\"description\":\"Install extra dependencies to compute LLM evaluation metrics\",\"dateModified\":\"2026-11-11T22:06:22.479Z\",\"isPartOf\":{\"@id\":\"https://arize.com/docs#website\"},\"breadcrumb\":{\"@id\":\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"API clients\",\"item\":\"https://arize.com/docs/api-clients/python/overview\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Python SDK\",\"item\":\"https://arize.com/docs/api-clients/python/overview\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Version 7\",\"item\":\"https://arize.com/docs/api-clients/python/version-7/overview\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Pandas Batch Logging\",\"item\":\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas\"},{\"@type\":\"ListItem\",\"position\":5,\"name\":\"NLP_Metrics\",\"item\":\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation#article\",\"headline\":\"NLP_Metrics\",\"name\":\"NLP_Metrics\",\"description\":\"Install extra dependencies to compute LLM evaluation metrics\",\"url\":\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation\",\"mainEntityOfPage\":{\"@id\":\"https://arize.com/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation#webpage\"},\"dateModified\":\"2026-11-11T22:06:22.479Z\",\"publisher\":{\"@id\":\"https://arize.com/#organization\"},\"isPartOf\":{\"@id\":\"https://arize.com/docs#website\"}}]}\u003c/script>","agentPointerHref":"/docs/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation.md","siteName":"Arize AX Docs","nav":{"global":null,"tabs":[{"tab":"Get Started","pages":[{"title":"Get Started","href":"/ax"}]},{"tab":"Guides","pages":[{"title":"Guides","href":"/ax/cookbooks"}]},{"tab":"Integrations","pages":[{"title":"Integrations","href":"/ax/integrations"}]},{"tab":"APIs, SDKs, Skills \u0026 MCP","pages":[{"title":"APIs, SDKs \u0026 Skills","sidebarTitle":"Overview","href":"/api-reference/overview"},{"title":"Choose the right interface for your workload","sidebarTitle":"Choosing an interface","href":"/api-reference/choosing-an-interface"},{"group":"Skills","pages":[{"title":"Skills","sidebarTitle":"Overview","href":"/ax/skills/overview"},{"title":"Install \u0026 authenticate","sidebarTitle":"Install \u0026 authenticate","href":"/ax/skills/install"},{"title":"Skill catalog","sidebarTitle":"Skill catalog","href":"/ax/skills/catalog"}]},{"group":"MCP server","pages":[{"title":"Arize AX MCP Server","sidebarTitle":"Overview","href":"/api-clients/mcp/overview"},{"title":"Connect to the AX MCP Server","sidebarTitle":"Connect your client","href":"/api-clients/mcp/connect"},{"group":"Reference","pages":[{"title":"AX MCP Authentication","sidebarTitle":"Authentication","href":"/api-clients/mcp/authentication"},{"title":"AX MCP Conventions","sidebarTitle":"Conventions","href":"/api-clients/mcp/conventions"},{"title":"AX MCP Endpoints","sidebarTitle":"Endpoints","href":"/api-clients/mcp/endpoints"},{"title":"Arize AX MCP Server FAQ","sidebarTitle":"FAQ","href":"/api-clients/mcp/faq"},{"title":"AX MCP Tool Catalog","sidebarTitle":"Tool catalog","href":"/api-clients/mcp/tool-catalog"}]}]},{"group":"API reference","pages":[{"group":"REST","pages":[{"title":"Arize REST API","sidebarTitle":"Overview","href":"/ax/rest-reference/overview"},{"title":"Error Reference","sidebarTitle":"Errors","href":"/ax/rest-reference/errors"},{"title":"Download OpenAPI Spec","href":"/ax/rest-reference/download-openapi-spec"},{"group":"Endpoints","pages":[{"group":"AI Integrations","pages":[{"title":"List AI integrations","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/ai-integrations","href":"/api-reference/ai-integrations/list-ai-integrations"},{"title":"Create an AI integration","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/ai-integrations","href":"/api-reference/ai-integrations/create-an-ai-integration"},{"title":"Get an AI integration","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/ai-integrations/{integration_id}","href":"/api-reference/ai-integrations/get-an-ai-integration"},{"title":"Delete an AI integration","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/ai-integrations/{integration_id}","href":"/api-reference/ai-integrations/delete-an-ai-integration"},{"title":"Update an AI integration","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/ai-integrations/{integration_id}","href":"/api-reference/ai-integrations/update-an-ai-integration"}]},{"group":"Alignment","pages":[{"title":"List alignment tasks","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/alignment-tasks","href":"/api-reference/alignment/list-alignment-tasks"},{"title":"Create alignment task","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/alignment-tasks","href":"/api-reference/alignment/create-alignment-task"},{"title":"Get alignment task","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/alignment-tasks/{alignment_task_id}","href":"/api-reference/alignment/get-alignment-task"},{"title":"List alignment runs","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/alignment-tasks/{alignment_task_id}/runs","href":"/api-reference/alignment/list-alignment-runs"},{"title":"Create alignment run","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/alignment-tasks/{alignment_task_id}/runs","href":"/api-reference/alignment/create-alignment-run"}]},{"group":"Annotation Configs","pages":[{"title":"List annotation configs","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/annotation-configs","href":"/api-reference/annotation-configs/list-annotation-configs"},{"title":"Create an annotation config","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/annotation-configs","href":"/api-reference/annotation-configs/create-an-annotation-config"},{"title":"Get an annotation config","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/annotation-configs/{annotation_config_id}","href":"/api-reference/annotation-configs/get-an-annotation-config"},{"title":"Delete an annotation config","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/annotation-configs/{annotation_config_id}","href":"/api-reference/annotation-configs/delete-an-annotation-config"},{"title":"Update an annotation config","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/annotation-configs/{annotation_config_id}","href":"/api-reference/annotation-configs/update-an-annotation-config"},{"title":"List tags on an annotation config","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/annotation-configs/{annotation_config_id}/tags","href":"/api-reference/annotation-configs/list-tags-on-an-annotation-config"},{"title":"Attach tags to a annotation config","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/annotation-configs/{annotation_config_id}/tags","href":"/api-reference/annotation-configs/attach-tags-to-a-annotation-config"}]},{"group":"Annotation Queues","pages":[{"title":"List annotation queues","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/annotation-queues","href":"/api-reference/annotation-queues/list-annotation-queues"},{"title":"Create an annotation queue","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/annotation-queues","href":"/api-reference/annotation-queues/create-an-annotation-queue"},{"title":"Get an annotation queue","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/annotation-queues/{annotation_queue_id}","href":"/api-reference/annotation-queues/get-an-annotation-queue"},{"title":"Delete an annotation queue","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/annotation-queues/{annotation_queue_id}","href":"/api-reference/annotation-queues/delete-an-annotation-queue"},{"title":"Update an annotation queue","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/annotation-queues/{annotation_queue_id}","href":"/api-reference/annotation-queues/update-an-annotation-queue"},{"title":"List annotation queue records","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/annotation-queues/{annotation_queue_id}/records","href":"/api-reference/annotation-queues/list-annotation-queue-records"},{"title":"Create annotation queue records","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/annotation-queues/{annotation_queue_id}/records","href":"/api-reference/annotation-queues/create-annotation-queue-records"},{"title":"Delete annotation queue records","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/annotation-queues/{annotation_queue_id}/records","href":"/api-reference/annotation-queues/delete-annotation-queue-records"},{"title":"Annotate a record","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/annotation-queues/{annotation_queue_id}/records/{annotation_queue_record_id}/annotate","href":"/api-reference/annotation-queues/annotate-a-record"},{"title":"Assign users to a record","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/annotation-queues/{annotation_queue_id}/records/{annotation_queue_record_id}/assign","href":"/api-reference/annotation-queues/assign-users-to-a-record"},{"title":"List tags on an annotation queue","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/annotation-queues/{annotation_queue_id}/tags","href":"/api-reference/annotation-queues/list-tags-on-an-annotation-queue"},{"title":"Attach tags to a annotation queue","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/annotation-queues/{annotation_queue_id}/tags","href":"/api-reference/annotation-queues/attach-tags-to-a-annotation-queue"}]},{"group":"API Keys","pages":[{"title":"List API keys","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/api-keys","href":"/api-reference/api-keys/list-api-keys"},{"title":"Create an API key","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/api-keys","href":"/api-reference/api-keys/create-an-api-key"},{"title":"Refresh an API key","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/api-keys/{api_key_id}/refresh","href":"/api-reference/api-keys/refresh-an-api-key"},{"title":"Revoke an API key","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml put /v2/api-keys/{api_key_id}/revoke","href":"/api-reference/api-keys/revoke-an-api-key"}]},{"group":"Audit Logs","pages":[{"title":"List audit logs","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/audit-logs","href":"/api-reference/audit-logs/list-audit-logs"}]},{"group":"Dashboards","pages":[{"title":"List dashboards","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/dashboards","href":"/api-reference/dashboards/list-dashboards"},{"title":"Create a dashboard","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/dashboards","href":"/api-reference/dashboards/create-a-dashboard"},{"title":"Get a dashboard","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/dashboards/{dashboard_id}","href":"/api-reference/dashboards/get-a-dashboard"},{"title":"Delete a dashboard","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/dashboards/{dashboard_id}","href":"/api-reference/dashboards/delete-a-dashboard"},{"title":"Update a dashboard","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/dashboards/{dashboard_id}","href":"/api-reference/dashboards/update-a-dashboard"},{"title":"List dashboard widgets","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/dashboards/{dashboard_id}/widgets","href":"/api-reference/dashboards/list-dashboard-widgets"},{"title":"Create a dashboard widget","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/dashboards/{dashboard_id}/widgets","href":"/api-reference/dashboards/create-a-dashboard-widget"},{"title":"Get a dashboard widget","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/dashboards/{dashboard_id}/widgets/{widget_id}","href":"/api-reference/dashboards/get-a-dashboard-widget"},{"title":"Delete a dashboard widget","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/dashboards/{dashboard_id}/widgets/{widget_id}","href":"/api-reference/dashboards/delete-a-dashboard-widget"}]},{"group":"Datasets","pages":[{"title":"List datasets","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/datasets","href":"/api-reference/datasets/list-datasets"},{"title":"Create a dataset","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/datasets","href":"/api-reference/datasets/create-a-dataset"},{"title":"Get a dataset","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/datasets/{dataset_id}","href":"/api-reference/datasets/get-a-dataset"},{"title":"Delete a dataset","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/datasets/{dataset_id}","href":"/api-reference/datasets/delete-a-dataset"},{"title":"Update a dataset","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/datasets/{dataset_id}","href":"/api-reference/datasets/update-a-dataset"},{"title":"List dataset examples","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/datasets/{dataset_id}/examples","href":"/api-reference/datasets/list-dataset-examples"},{"title":"Add new examples to a dataset","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/datasets/{dataset_id}/examples","href":"/api-reference/datasets/add-new-examples-to-a-dataset"},{"title":"Delete dataset examples","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/datasets/{dataset_id}/examples","href":"/api-reference/datasets/delete-dataset-examples"},{"title":"Update existing examples in a dataset","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/datasets/{dataset_id}/examples","href":"/api-reference/datasets/update-existing-examples-in-a-dataset"},{"title":"Annotate a batch of dataset examples","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/datasets/{dataset_id}/examples/annotate","href":"/api-reference/datasets/annotate-a-batch-of-dataset-examples"},{"title":"Search dataset examples","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/datasets/{dataset_id}/examples/search","href":"/api-reference/datasets/search-dataset-examples"},{"title":"List tags on a dataset","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/datasets/{dataset_id}/tags","href":"/api-reference/datasets/list-tags-on-a-dataset"},{"title":"Attach tags to a dataset","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/datasets/{dataset_id}/tags","href":"/api-reference/datasets/attach-tags-to-a-dataset"},{"title":"Detach tags from a dataset","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/datasets/{dataset_id}/tags","href":"/api-reference/datasets/detach-tags-from-a-dataset"}]},{"group":"Evaluators","pages":[{"title":"List evaluator templates","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/evaluator-templates","href":"/api-reference/evaluators/list-evaluator-templates"},{"title":"Get evaluator version","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/evaluator-versions/{version_id}","href":"/api-reference/evaluators/get-evaluator-version"},{"title":"List evaluators","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/evaluators","href":"/api-reference/evaluators/list-evaluators"},{"title":"Create evaluator","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/evaluators","href":"/api-reference/evaluators/create-evaluator"},{"title":"Get evaluator","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/evaluators/{evaluator_id}","href":"/api-reference/evaluators/get-evaluator"},{"title":"Delete evaluator","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/evaluators/{evaluator_id}","href":"/api-reference/evaluators/delete-evaluator"},{"title":"Update evaluator","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/evaluators/{evaluator_id}","href":"/api-reference/evaluators/update-evaluator"},{"title":"List tags on an evaluator","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/evaluators/{evaluator_id}/tags","href":"/api-reference/evaluators/list-tags-on-an-evaluator"},{"title":"Attach tags to a evaluator","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/evaluators/{evaluator_id}/tags","href":"/api-reference/evaluators/attach-tags-to-a-evaluator"},{"title":"Detach tags from a evaluator","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/evaluators/{evaluator_id}/tags","href":"/api-reference/evaluators/detach-tags-from-a-evaluator"},{"title":"List evaluator versions","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/evaluators/{evaluator_id}/versions","href":"/api-reference/evaluators/list-evaluator-versions"},{"title":"Create evaluator version","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/evaluators/{evaluator_id}/versions","href":"/api-reference/evaluators/create-evaluator-version"},{"title":"Delete evaluator versions","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/evaluators/{evaluator_id}/versions","href":"/api-reference/evaluators/delete-evaluator-versions"}]},{"group":"Experiments","pages":[{"title":"List experiments","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/experiments","href":"/api-reference/experiments/list-experiments"},{"title":"Create an experiment","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/experiments","href":"/api-reference/experiments/create-an-experiment"},{"title":"Get an experiment","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/experiments/{experiment_id}","href":"/api-reference/experiments/get-an-experiment"},{"title":"Delete an experiment","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/experiments/{experiment_id}","href":"/api-reference/experiments/delete-an-experiment"},{"title":"List experiment runs","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/experiments/{experiment_id}/runs","href":"/api-reference/experiments/list-experiment-runs"},{"title":"Append runs to an experiment","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/experiments/{experiment_id}/runs","href":"/api-reference/experiments/append-runs-to-an-experiment"},{"title":"Annotate a batch of experiment runs","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/experiments/{experiment_id}/runs/annotate","href":"/api-reference/experiments/annotate-a-batch-of-experiment-runs"},{"title":"Search experiment runs","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/experiments/{experiment_id}/runs/search","href":"/api-reference/experiments/search-experiment-runs"},{"title":"List tags on an experiment","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/experiments/{experiment_id}/tags","href":"/api-reference/experiments/list-tags-on-an-experiment"},{"title":"Attach tags to a experiment","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/experiments/{experiment_id}/tags","href":"/api-reference/experiments/attach-tags-to-a-experiment"}]},{"group":"Integrations","pages":[{"title":"List integrations","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/integrations","href":"/api-reference/integrations/list-integrations"},{"title":"Create an integration","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/integrations","href":"/api-reference/integrations/create-an-integration"},{"title":"Get an integration","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/integrations/{integration_id}","href":"/api-reference/integrations/get-an-integration"},{"title":"Delete an integration","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/integrations/{integration_id}","href":"/api-reference/integrations/delete-an-integration"},{"title":"Update an integration","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/integrations/{integration_id}","href":"/api-reference/integrations/update-an-integration"}]},{"group":"Monitors","pages":[{"title":"List monitors","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/monitors","href":"/api-reference/monitors/list-monitors"},{"title":"Create a monitor","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/monitors","href":"/api-reference/monitors/create-a-monitor"},{"title":"Get a monitor","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/monitors/{monitor_id}","href":"/api-reference/monitors/get-a-monitor"},{"title":"Delete a monitor","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/monitors/{monitor_id}","href":"/api-reference/monitors/delete-a-monitor"},{"title":"List tags on a monitor","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/monitors/{monitor_id}/tags","href":"/api-reference/monitors/list-tags-on-a-monitor"},{"title":"Attach tags to a monitor","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/monitors/{monitor_id}/tags","href":"/api-reference/monitors/attach-tags-to-a-monitor"},{"title":"Detach tags from a monitor","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/monitors/{monitor_id}/tags","href":"/api-reference/monitors/detach-tags-from-a-monitor"}]},{"group":"Organizations","pages":[{"title":"List organizations","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/organizations","href":"/api-reference/organizations/list-organizations"},{"title":"Create an organization","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/organizations","href":"/api-reference/organizations/create-an-organization"},{"title":"Get an organization","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/organizations/{org_id}","href":"/api-reference/organizations/get-an-organization"},{"title":"Delete an organization","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/organizations/{org_id}","href":"/api-reference/organizations/delete-an-organization"},{"title":"Update an organization","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/organizations/{org_id}","href":"/api-reference/organizations/update-an-organization"},{"title":"Add a user to an organization","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/organizations/{org_id}/users","href":"/api-reference/organizations/add-a-user-to-an-organization"},{"title":"Remove a user from an organization","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/organizations/{org_id}/users/{user_id}","href":"/api-reference/organizations/remove-a-user-from-an-organization"}]},{"group":"Projects","pages":[{"title":"List projects","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/projects","href":"/api-reference/projects/list-projects"},{"title":"Create a project","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/projects","href":"/api-reference/projects/create-a-project"},{"title":"Get a project","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/projects/{project_id}","href":"/api-reference/projects/get-a-project"},{"title":"Delete a project","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/projects/{project_id}","href":"/api-reference/projects/delete-a-project"},{"title":"Update a project","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/projects/{project_id}","href":"/api-reference/projects/update-a-project"},{"title":"List tags on a project","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/projects/{project_id}/tags","href":"/api-reference/projects/list-tags-on-a-project"},{"title":"Attach tags to a project","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/projects/{project_id}/tags","href":"/api-reference/projects/attach-tags-to-a-project"},{"title":"Detach tags from a project","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/projects/{project_id}/tags","href":"/api-reference/projects/detach-tags-from-a-project"}]},{"group":"Prompts","pages":[{"title":"List prompts","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/prompts","href":"/api-reference/prompts/list-prompts"},{"title":"Create a prompt","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/prompts","href":"/api-reference/prompts/create-a-prompt"},{"title":"Get a prompt","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/prompts/{prompt_id}","href":"/api-reference/prompts/get-a-prompt"},{"title":"Delete a prompt","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/prompts/{prompt_id}","href":"/api-reference/prompts/delete-a-prompt"},{"title":"Update a prompt","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/prompts/{prompt_id}","href":"/api-reference/prompts/update-a-prompt"},{"title":"Resolve a label to a prompt version","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/prompts/{prompt_id}/labels/{label_name}","href":"/api-reference/prompts/resolve-a-label-to-a-prompt-version"},{"title":"List tags on a prompt","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/prompts/{prompt_id}/tags","href":"/api-reference/prompts/list-tags-on-a-prompt"},{"title":"Attach tags to a prompt","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/prompts/{prompt_id}/tags","href":"/api-reference/prompts/attach-tags-to-a-prompt"},{"title":"Detach tags from a prompt","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/prompts/{prompt_id}/tags","href":"/api-reference/prompts/detach-tags-from-a-prompt"},{"title":"List prompt versions","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/prompts/{prompt_id}/versions","href":"/api-reference/prompts/list-prompt-versions"},{"title":"Create a prompt version","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/prompts/{prompt_id}/versions","href":"/api-reference/prompts/create-a-prompt-version"},{"title":"Get a prompt version","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/prompt-versions/{version_id}","href":"/api-reference/prompts/get-a-prompt-version"},{"title":"Set labels on a prompt version","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml put /v2/prompt-versions/{version_id}/labels","href":"/api-reference/prompts/set-labels-on-a-prompt-version"},{"title":"Remove a label from a prompt version","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/prompt-versions/{version_id}/labels/{label_name}","href":"/api-reference/prompts/remove-a-label-from-a-prompt-version"}]},{"group":"Resource Restrictions","pages":[{"title":"List resource restrictions the caller is permitted to manage.","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/resource-restrictions","href":"/api-reference/resource-restrictions/list-resource-restrictions-the-caller-is-permitted-to-manage"},{"title":"Restrict a resource","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/resource-restrictions","href":"/api-reference/resource-restrictions/restrict-a-resource"},{"title":"Unrestrict a resource","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/resource-restrictions/{resource_id}","href":"/api-reference/resource-restrictions/unrestrict-a-resource"}]},{"group":"Role Bindings","pages":[{"title":"List role bindings","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/role-bindings","href":"/api-reference/role-bindings/list-role-bindings"},{"title":"Create a role binding","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/role-bindings","href":"/api-reference/role-bindings/create-a-role-binding"},{"title":"Get a role binding","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/role-bindings/{binding_id}","href":"/api-reference/role-bindings/get-a-role-binding"},{"title":"Delete a role binding","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/role-bindings/{binding_id}","href":"/api-reference/role-bindings/delete-a-role-binding"},{"title":"Update a role binding","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/role-bindings/{binding_id}","href":"/api-reference/role-bindings/update-a-role-binding"}]},{"group":"Roles","pages":[{"title":"List roles","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/roles","href":"/api-reference/roles/list-roles"},{"title":"Create a role","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/roles","href":"/api-reference/roles/create-a-role"},{"title":"Get a role","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/roles/{role_id}","href":"/api-reference/roles/get-a-role"},{"title":"Delete a role","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/roles/{role_id}","href":"/api-reference/roles/delete-a-role"},{"title":"Update a role","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/roles/{role_id}","href":"/api-reference/roles/update-a-role"}]},{"group":"SAML","pages":[{"title":"List SAML identity providers","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/saml-identity-providers","href":"/api-reference/saml/list-saml-identity-providers"},{"title":"Create a SAML identity provider","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/saml-identity-providers","href":"/api-reference/saml/create-a-saml-identity-provider"},{"title":"Get a SAML identity provider","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/saml-identity-providers/{idp_id}","href":"/api-reference/saml/get-a-saml-identity-provider"},{"title":"Remove a SAML identity provider","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/saml-identity-providers/{idp_id}","href":"/api-reference/saml/remove-a-saml-identity-provider"},{"title":"Update supplied SAML configuration fields","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/saml-identity-providers/{idp_id}","href":"/api-reference/saml/update-supplied-saml-configuration-fields"},{"title":"List SAML role mappings","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/saml-identity-providers/{idp_id}/role-mappings","href":"/api-reference/saml/list-saml-role-mappings"},{"title":"Add a SAML role mapping","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/saml-identity-providers/{idp_id}/role-mappings","href":"/api-reference/saml/add-a-saml-role-mapping"},{"title":"Remove a SAML role mapping","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/saml-identity-providers/{idp_id}/role-mappings/{mapping_id}","href":"/api-reference/saml/remove-a-saml-role-mapping"},{"title":"Update supplied mapping fields","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/saml-identity-providers/{idp_id}/role-mappings/{mapping_id}","href":"/api-reference/saml/update-supplied-mapping-fields"}]},{"group":"Spaces","pages":[{"title":"List spaces","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/spaces","href":"/api-reference/spaces/list-spaces"},{"title":"Create a space","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/spaces","href":"/api-reference/spaces/create-a-space"},{"title":"Get a space","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/spaces/{space_id}","href":"/api-reference/spaces/get-a-space"},{"title":"Delete a space","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/spaces/{space_id}","href":"/api-reference/spaces/delete-a-space"},{"title":"Update a space","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/spaces/{space_id}","href":"/api-reference/spaces/update-a-space"},{"title":"Add a user to a space","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/spaces/{space_id}/users","href":"/api-reference/spaces/add-a-user-to-a-space"},{"title":"Remove a user from a space","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/spaces/{space_id}/users/{user_id}","href":"/api-reference/spaces/remove-a-user-from-a-space"}]},{"group":"Spans","pages":[{"title":"List spans","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/spans","href":"/api-reference/spans/list-spans"},{"title":"Delete spans","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/spans","href":"/api-reference/spans/delete-spans"},{"title":"Annotate a batch of project spans","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/spans/annotate","href":"/api-reference/spans/annotate-a-batch-of-project-spans"}]},{"group":"Tags","pages":[{"title":"Create a tag","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/tags","href":"/api-reference/tags/create-a-tag"},{"title":"Delete a tag","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/tags/{tag_id}","href":"/api-reference/tags/delete-a-tag"},{"title":"Update a tag","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/tags/{tag_id}","href":"/api-reference/tags/update-a-tag"}]},{"group":"Tasks","pages":[{"title":"List tasks","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/tasks","href":"/api-reference/tasks/list-tasks"},{"title":"Create task","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/tasks","href":"/api-reference/tasks/create-task"},{"title":"Get task","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/tasks/{task_id}","href":"/api-reference/tasks/get-task"},{"title":"Delete task","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/tasks/{task_id}","href":"/api-reference/tasks/delete-task"},{"title":"Update task","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/tasks/{task_id}","href":"/api-reference/tasks/update-task"},{"title":"List task runs","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/tasks/{task_id}/runs","href":"/api-reference/tasks/list-task-runs"},{"title":"Trigger a task run","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/tasks/{task_id}/trigger","href":"/api-reference/tasks/trigger-a-task-run"},{"title":"Get task run","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/task-runs/{run_id}","href":"/api-reference/tasks/get-task-run"},{"title":"Cancel task run","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/task-runs/{run_id}/cancel","href":"/api-reference/tasks/cancel-task-run"}]},{"group":"Traces","pages":[{"title":"List traces","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/traces","href":"/api-reference/traces/list-traces"}]},{"group":"Users","pages":[{"title":"List users","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/users","href":"/api-reference/users/list-users"},{"title":"Create a user","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/users","href":"/api-reference/users/create-a-user"},{"title":"Get a user","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/users/{user_id}","href":"/api-reference/users/get-a-user"},{"title":"Delete a user","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/users/{user_id}","href":"/api-reference/users/delete-a-user"},{"title":"Update a user","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/users/{user_id}","href":"/api-reference/users/update-a-user"},{"title":"Resend a user invitation","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/users/{user_id}/resend-invitation","href":"/api-reference/users/resend-a-user-invitation"},{"title":"Trigger a password-reset email for a user","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/users/{user_id}/reset-password","href":"/api-reference/users/trigger-a-password-reset-email-for-a-user"}]},{"group":"Webhooks","pages":[{"title":"List webhook subscriptions","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/webhook-subscriptions","href":"/api-reference/webhooks/list-webhook-subscriptions"},{"title":"Create a webhook subscription","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/webhook-subscriptions","href":"/api-reference/webhooks/create-a-webhook-subscription"},{"title":"Get a webhook subscription","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/webhook-subscriptions/{subscription_id}","href":"/api-reference/webhooks/get-a-webhook-subscription"},{"title":"Delete a webhook subscription","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/webhook-subscriptions/{subscription_id}","href":"/api-reference/webhooks/delete-a-webhook-subscription"},{"title":"List webhooks","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/webhooks","href":"/api-reference/webhooks/list-webhooks"},{"title":"Create a webhook","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/webhooks","href":"/api-reference/webhooks/create-a-webhook"},{"title":"Get a webhook","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/webhooks/{webhook_id}","href":"/api-reference/webhooks/get-a-webhook"},{"title":"Delete a webhook","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml delete /v2/webhooks/{webhook_id}","href":"/api-reference/webhooks/delete-a-webhook"},{"title":"Update a webhook","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml patch /v2/webhooks/{webhook_id}","href":"/api-reference/webhooks/update-a-webhook"},{"title":"List a webhook's delivery attempts","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml get /v2/webhooks/{webhook_id}/delivery-attempts","href":"/api-reference/webhooks/list-a-webhooks-delivery-attempts"},{"title":"Send a test event to a webhook","deprecated":null,"version":null,"openapi":"https://api.arize.com/v2/spec.yaml post /v2/webhooks/{webhook_id}/test","href":"/api-reference/webhooks/send-a-test-event-to-a-webhook"}]}]}]},{"group":"GraphQL","pages":[{"title":"GraphQL API","sidebarTitle":"GraphQL API Overview","href":"/ax/graphql-reference"},{"title":"Getting Started with GraphQL","href":"/ax/graphql-reference/overview/getting-started-with-graphql"},{"title":"API Explorer","href":"/ax/graphql-reference/overview/api-explorer"},{"group":"How to use GraphQL","pages":[{"title":"How to Use GraphQL","sidebarTitle":"Overview","href":"/ax/graphql-reference/overview/how-to-use-graphql"},{"title":"Forming Calls","href":"/ax/graphql-reference/overview/how-to-use-graphql/forming-calls"},{"title":"Using global node IDs","href":"/ax/graphql-reference/overview/how-to-use-graphql/using-global-node-ids"},{"title":"Querying Nested Data","href":"/ax/graphql-reference/overview/how-to-use-graphql/querying-nested-data"},{"title":"Notebook Examples","href":"/ax/graphql-reference/overview/how-to-use-graphql/notebook-examples"},{"title":"Mutations","href":"/ax/graphql-reference/overview/how-to-use-graphql/mutations"}]},{"title":"Resource Limitations","href":"/ax/graphql-reference/overview/resource-limitations"},{"group":"Guides","pages":[{"title":"Managing spaces, keys and SSO with GraphQL","href":"/ax/graphql-reference/guides/admin"},{"title":"Managing monitors with GraphQL","href":"/ax/graphql-reference/guides/monitors"},{"title":"Managing dashboards with GraphQL","href":"/ax/graphql-reference/guides/dashboards"},{"title":"Managing projects and models with GraphQL","href":"/ax/graphql-reference/guides/projects-and-models"},{"title":"Importing data with GraphQL","href":"/ax/graphql-reference/guides/data-import"},{"title":"Managing evaluators and tasks with GraphQL","href":"/ax/graphql-reference/guides/evaluators-and-tasks"},{"title":"Managing datasets and annotations with GraphQL","href":"/ax/graphql-reference/guides/datasets-and-annotations"},{"title":"Managing prompts with GraphQL","href":"/ax/graphql-reference/guides/prompts"},{"title":"Managing LLM integrations with GraphQL","href":"/ax/graphql-reference/guides/llm-integrations"},{"title":"Managed agents and Signal via GraphQL","href":"/ax/graphql-reference/guides/managed-agents-and-signal"}]},{"group":"Mutation reference","pages":[{"title":"All GraphQL mutations","href":"/ax/graphql-reference/mutations/index"},{"title":"Admin mutations","href":"/ax/graphql-reference/mutations/admin"},{"title":"Monitor mutations","href":"/ax/graphql-reference/mutations/monitors"},{"title":"Dashboard mutations","href":"/ax/graphql-reference/mutations/dashboards"},{"title":"Project and model mutations","href":"/ax/graphql-reference/mutations/projects-and-models"},{"title":"Data import mutations","href":"/ax/graphql-reference/mutations/data-import"},{"title":"Evaluator and task mutations","href":"/ax/graphql-reference/mutations/evaluators-and-tasks"},{"title":"Dataset and annotation mutations","href":"/ax/graphql-reference/mutations/datasets-and-annotations"},{"title":"Prompt mutations","href":"/ax/graphql-reference/mutations/prompts"},{"title":"LLM integration mutations","href":"/ax/graphql-reference/mutations/llm-integrations"},{"title":"Managed agent and Signal mutations","href":"/ax/graphql-reference/mutations/managed-agents-and-signal"}]},{"group":"Query reference","pages":[{"title":"Root queries","href":"/ax/graphql-reference/queries/root-queries"},{"title":"Object graph","href":"/ax/graphql-reference/queries/object-graph"}]}]}]},{"group":"API clients","pages":[{"group":"Python SDK","pages":[{"title":"Python SDK","sidebarTitle":"Overview","href":"/api-clients/python/overview"},{"group":"Version 8","pages":[{"title":"Python SDK v8","sidebarTitle":"Overview","href":"/api-clients/python/version-8/overview"},{"group":"Tutorial","pages":[{"title":"Send Traces to Arize AX","href":"/api-clients/python/version-8/tutorial/get-started-tracing-with-arize-sdk"},{"title":"Run and Log Evals to Measure Performance","href":"/api-clients/python/version-8/tutorial/get-started-evaluations-with-arize-sdk"},{"title":"Iterate with Experiments","href":"/api-clients/python/version-8/tutorial/get-started-datasets-experiments-with-arize-sdk"}]},{"title":"OpenTelemetry Tracing","href":"/api-clients/python/version-8/tracing"},{"title":"AI Integrations","href":"/api-clients/python/version-8/client-resources/ai-integrations"},{"title":"Annotation Configs","href":"/api-clients/python/version-8/client-resources/annotation-configs"},{"title":"Annotation Queues","href":"/api-clients/python/version-8/client-resources/annotation-queues"},{"title":"API Keys","href":"/api-clients/python/version-8/client-resources/api-keys"},{"title":"Audit Logs","href":"/api-clients/python/version-8/client-resources/audit-logs"},{"title":"Datasets","href":"/api-clients/python/version-8/client-resources/datasets"},{"title":"Evaluators","href":"/api-clients/python/version-8/client-resources/evaluators"},{"title":"Experiments","href":"/api-clients/python/version-8/client-resources/experiments"},{"title":"Integrations","href":"/api-clients/python/version-8/client-resources/integrations"},{"title":"ML Models","href":"/api-clients/python/version-8/client-resources/ml-models"},{"title":"Organizations","href":"/api-clients/python/version-8/client-resources/organizations"},{"title":"Projects","href":"/api-clients/python/version-8/client-resources/projects"},{"title":"Prompts","href":"/api-clients/python/version-8/client-resources/prompts"},{"title":"Resource Restrictions","href":"/api-clients/python/version-8/client-resources/resource-restrictions"},{"title":"Role Bindings","href":"/api-clients/python/version-8/client-resources/role-bindings"},{"title":"Roles","href":"/api-clients/python/version-8/client-resources/roles"},{"title":"SAML Identity Providers","href":"/api-clients/python/version-8/client-resources/saml-identity-providers"},{"title":"Spaces","href":"/api-clients/python/version-8/client-resources/spaces"},{"title":"Spans","href":"/api-clients/python/version-8/client-resources/spans"},{"title":"Tasks","href":"/api-clients/python/version-8/client-resources/tasks"},{"title":"Traces","href":"/api-clients/python/version-8/client-resources/traces"},{"title":"Users","href":"/api-clients/python/version-8/client-resources/users"},{"title":"Webhooks","href":"/api-clients/python/version-8/client-resources/webhooks"},{"title":"Embeddings","href":"/api-clients/python/version-8/embeddings"},{"title":"Utilities","href":"/api-clients/python/version-8/utils"},{"group":"Migration from v7","pages":[{"title":"Migration Overview","sidebarTitle":"Overview","href":"/api-clients/python/version-8/migration/index"},{"title":"Migrate Pandas Client","sidebarTitle":"Pandas Client","href":"/api-clients/python/version-8/migration/pandas-client"},{"title":"Migrate Stream Client","sidebarTitle":"Stream Client","href":"/api-clients/python/version-8/migration/stream-client"},{"group":"Datasets Client","pages":[{"title":"Migrate Datasets Client","sidebarTitle":"Datasets","href":"/api-clients/python/version-8/migration/datasets-client"},{"title":"Migrate Experiments Client","sidebarTitle":"Experiments","href":"/api-clients/python/version-8/migration/experiments-client"}]},{"title":"Migrate Exporter Client","sidebarTitle":"Exporter","href":"/api-clients/python/version-8/migration/exporter-client"}]}]},{"group":"Version 7","pages":[{"title":"Python SDK v7","sidebarTitle":"Overview","href":"/api-clients/python/version-7/overview"},{"title":"Export Data API","href":"/api-clients/python/version-7/export-data-api"},{"title":"Prompt Hub API","href":"/api-clients/python/version-7/prompt-hub-api"},{"group":"Pandas Batch Logging","pages":[{"title":"Pandas batch logging","sidebarTitle":"Overview","href":"/api-clients/python/version-7/ml-logging/arize-pandas"},{"title":"Client","href":"/api-clients/python/version-7/ml-logging/arize-pandas/client"},{"title":"Log","href":"/api-clients/python/version-7/ml-logging/arize-pandas/log"},{"title":"Schema","href":"/api-clients/python/version-7/ml-logging/arize-pandas/schema"},{"title":"TypedColumns","href":"/api-clients/python/version-7/ml-logging/arize-pandas/typedcolumns"},{"title":"EmbeddingColumnNames","href":"/api-clients/python/version-7/ml-logging/arize-pandas/embeddingcolumnnames"},{"title":"ObjectDetectionColumnNames","href":"/api-clients/python/version-7/ml-logging/arize-pandas/objectdetectioncolumnnames"},{"title":"PromptTemplateColumnNames","href":"/api-clients/python/version-7/ml-logging/arize-pandas/prompttemplatecolumnnames"},{"title":"LLMConfigColumnNames","href":"/api-clients/python/version-7/ml-logging/arize-pandas/llmconfigcolumnnames"},{"title":"LLMRunMetadataColumnNames","description":"Ingest metadata about your LLM inferences","href":"/api-clients/python/version-7/ml-logging/arize-pandas/llmrunmetadatacolumnnames"},{"title":"NLP_Metrics","sidebarTitle":"NLPĀ Metrics","href":"/api-clients/python/version-7/ml-logging/arize-pandas/llm-evaluation"},{"title":"AutoEmbeddings","description":"Install extra dependencies to generate embedding vectors","href":"/api-clients/python/version-7/ml-logging/arize-pandas/autoembeddings"},{"title":"Utils.Types.ModelTypes","href":"/api-clients/python/version-7/ml-logging/arize-pandas/utils.types.modeltypes"},{"title":"Utils.Types.Metrics","href":"/api-clients/python/version-7/ml-logging/arize-pandas/utils.types.metrics"},{"title":"Utils.Types.Environments","href":"/api-clients/python/version-7/ml-logging/arize-pandas/utils.types.environments"}]},{"group":"Single Record Logging","pages":[{"title":"Single Record Logging","sidebarTitle":"Overview","href":"/api-clients/python/version-7/ml-logging/arize-log"},{"title":"Client","href":"/api-clients/python/version-7/ml-logging/arize-log/client"},{"title":"Log","href":"/api-clients/python/version-7/ml-logging/arize-log/logpage"},{"title":"TypedValue","href":"/api-clients/python/version-7/ml-logging/arize-log/typedvalue"},{"title":"Ranking","href":"/api-clients/python/version-7/ml-logging/arize-log/ranking"},{"title":"Multi class","href":"/api-clients/python/version-7/ml-logging/arize-log/multi-class"},{"title":"Object detection","href":"/api-clients/python/version-7/ml-logging/arize-log/objectdetection"},{"title":"Embedding","href":"/api-clients/python/version-7/ml-logging/arize-log/embedding"},{"title":"LLMRunMetadata","href":"/api-clients/python/version-7/ml-logging/arize-log/llmrunmetadata"},{"title":"Utils.Types.ModelTypes","href":"/api-clients/python/version-7/ml-logging/arize-log/utils.types.modeltypes"},{"title":"Utils.Types.Metrics","href":"/api-clients/python/version-7/ml-logging/arize-log/utils.types.metrics"},{"title":"Utils.Types.Environments","href":"/api-clients/python/version-7/ml-logging/arize-log/utils.types.environments"}]}]}]},{"group":"TypeScript SDK","pages":[{"group":"Version 1","pages":[{"title":"TypeScript SDK v1","sidebarTitle":"Overview","href":"/api-clients/typescript/version-1/overview"},{"title":"AI Integrations","href":"/api-clients/typescript/version-1/client-resources/ai-integrations"},{"title":"Annotation Configs","href":"/api-clients/typescript/version-1/client-resources/annotation-configs"},{"title":"Annotation Queues","href":"/api-clients/typescript/version-1/client-resources/annotation-queues"},{"title":"API Keys","href":"/api-clients/typescript/version-1/client-resources/api-keys"},{"title":"Audit Logs","href":"/api-clients/typescript/version-1/client-resources/audit-logs"},{"title":"Datasets","href":"/api-clients/typescript/version-1/client-resources/datasets"},{"title":"Evaluators","href":"/api-clients/typescript/version-1/client-resources/evaluators"},{"title":"Experiments","href":"/api-clients/typescript/version-1/client-resources/experiments"},{"title":"Integrations","href":"/api-clients/typescript/version-1/client-resources/integrations"},{"title":"Organizations","href":"/api-clients/typescript/version-1/client-resources/organizations"},{"title":"Projects","href":"/api-clients/typescript/version-1/client-resources/projects"},{"title":"Prompts","href":"/api-clients/typescript/version-1/client-resources/prompts"},{"title":"Resource Restrictions","href":"/api-clients/typescript/version-1/client-resources/resource-restrictions"},{"title":"Role Bindings","href":"/api-clients/typescript/version-1/client-resources/role-bindings"},{"title":"Roles","href":"/api-clients/typescript/version-1/client-resources/roles"},{"title":"Spaces","href":"/api-clients/typescript/version-1/client-resources/spaces"},{"title":"Spans","href":"/api-clients/typescript/version-1/client-resources/spans"},{"title":"Tasks","href":"/api-clients/typescript/version-1/client-resources/tasks"},{"title":"Traces","href":"/api-clients/typescript/version-1/client-resources/traces"},{"title":"Users","href":"/api-clients/typescript/version-1/client-resources/users"},{"title":"Webhooks","href":"/api-clients/typescript/version-1/client-resources/webhooks"}]}]},{"group":"Golang SDK","pages":[{"group":"Version 2","pages":[{"title":"Go SDK v2","sidebarTitle":"Overview","href":"/api-clients/go/version-2/overview"},{"title":"AI Integrations","href":"/api-clients/go/version-2/client-resources/ai-integrations"},{"title":"Annotation Configs","href":"/api-clients/go/version-2/client-resources/annotation-configs"},{"title":"Annotation Queues","href":"/api-clients/go/version-2/client-resources/annotation-queues"},{"title":"API Keys","href":"/api-clients/go/version-2/client-resources/api-keys"},{"title":"Audit Logs","href":"/api-clients/go/version-2/client-resources/audit-logs"},{"title":"Datasets","href":"/api-clients/go/version-2/client-resources/datasets"},{"title":"Evaluators","href":"/api-clients/go/version-2/client-resources/evaluators"},{"title":"Experiments","href":"/api-clients/go/version-2/client-resources/experiments"},{"title":"Integrations","href":"/api-clients/go/version-2/client-resources/integrations"},{"title":"Organizations","href":"/api-clients/go/version-2/client-resources/organizations"},{"title":"Projects","href":"/api-clients/go/version-2/client-resources/projects"},{"title":"Prompts","href":"/api-clients/go/version-2/client-resources/prompts"},{"title":"Resource Restrictions","href":"/api-clients/go/version-2/client-resources/resource-restrictions"},{"title":"Role Bindings","href":"/api-clients/go/version-2/client-resources/role-bindings"},{"title":"Roles","href":"/api-clients/go/version-2/client-resources/roles"},{"title":"Spaces","href":"/api-clients/go/version-2/client-resources/spaces"},{"title":"Spans","href":"/api-clients/go/version-2/client-resources/spans"},{"title":"Tasks","href":"/api-clients/go/version-2/client-resources/tasks"},{"title":"Traces","href":"/api-clients/go/version-2/client-resources/traces"},{"title":"Users","href":"/api-clients/go/version-2/client-resources/users"},{"title":"Webhooks","href":"/api-clients/go/version-2/client-resources/webhooks"}]}]},{"group":"CLI","pages":[{"title":"AX CLI","sidebarTitle":"Overview","href":"/api-clients/cli/overview"},{"title":"AI Integrations","href":"/api-clients/cli/ai-integrations"},{"title":"Annotation Configs","href":"/api-clients/cli/annotation-configs"},{"title":"Annotation Queues","href":"/api-clients/cli/annotation-queues"},{"title":"API Keys","href":"/api-clients/cli/api-keys"},{"title":"Auth","href":"/api-clients/cli/auth"},{"title":"Datasets","href":"/api-clients/cli/datasets"},{"title":"Evaluators","href":"/api-clients/cli/evaluators"},{"title":"Experiments","href":"/api-clients/cli/experiments"},{"title":"Instrument","href":"/api-clients/cli/instrument"},{"title":"Integrations","href":"/api-clients/cli/integrations"},{"title":"Organizations","href":"/api-clients/cli/organizations"},{"title":"Profiles","href":"/api-clients/cli/profiles"},{"title":"Projects","href":"/api-clients/cli/projects"},{"title":"Prompts","href":"/api-clients/cli/prompts"},{"title":"Resource Restrictions","href":"/api-clients/cli/resource-restrictions"},{"title":"Role Bindings","href":"/api-clients/cli/role-bindings"},{"title":"Roles","href":"/api-clients/cli/roles"},{"title":"Skills","href":"/api-clients/cli/skills"},{"title":"Spaces","href":"/api-clients/cli/spaces"},{"title":"Spans","href":"/api-clients/cli/spans"},{"title":"Tasks","href":"/api-clients/cli/tasks"},{"title":"Traces","href":"/api-clients/cli/traces"},{"title":"Upgrade","href":"/api-clients/cli/upgrade"},{"title":"Users","href":"/api-clients/cli/users"},{"title":"Webhooks","href":"/api-clients/cli/webhooks"}]},{"group":"Managed Agents CLI","pages":[{"title":"Managed Agents CLI (ag)","sidebarTitle":"Overview","href":"/api-clients/ag/overview"},{"title":"Sessions","href":"/api-clients/ag/sessions"},{"title":"Presets","href":"/api-clients/ag/presets"},{"title":"Skills","href":"/api-clients/ag/skills"},{"title":"Automations","href":"/api-clients/ag/automations"},{"title":"Signal","href":"/api-clients/ag/signal"}]}]}]},{"tab":"Courses","pages":[{"title":"Courses","href":"/ax/learn/courses/overview"}]},{"tab":"Concepts","pages":[{"title":"Concepts","href":"/ax/concepts/otel-openinference/overview"}]},{"tab":"Self-Hosting","pages":[{"title":"Self-Hosting","href":"/ax/selfhosting"}]},{"tab":"Changelog","pages":[{"title":"Changelog","href":"/ax/release-notes"}]}]},"basePath":"","title":"NLP_Metrics - Arize AX Docs","deploymentId":"dpl_B1qJC7o4Nbc1BrDLB8Y5TDHq3c2t","buildId":"dpl_B1qJC7o4Nbc1BrDLB8Y5TDHq3c2t","dynamics":["a74be98ba7"],"chunks":["zoom-image-lightbox-Dq2Z5ult.js","lightbox-handle-r17bVUz3.js"]}