# Okareo Docs > Refernce documentation for Okareo, the platform for voice/text agent simulation, evaluation, and monitoring. This file contains links to documentation sections following the llmstxt.org standard. ## Table of Contents - [API Token](https://okareo.com/docs/introduction/api-token.md): Your Okareo API token authenticates all requests to Okareo—whether you use the SDKs, CLI, REST API, or the proxy for monitoring. This page gets you... - [Getting Started](https://okareo.com/docs/introduction/getting-started.md): This guide will walk you through two fundamental capabilities using simple code examples: running a **Prompt Evaluation** to assess model performan... - [Overview](https://okareo.com/docs/introduction/overview.md): Voice and chat agents don’t crash — they fail quietly. A voice assistant can give the wrong policy, drift off task, or miss a step without triggeri... - [Terminology](https://okareo.com/docs/introduction/terminology.md): Here we provide a comprehensive list of key terms and concepts that are essential for understanding our platform and its capabilities in Agent simu... - [API Token](https://okareo.com/reference/api-token.md): Okareo API Token - [CLI](https://okareo.com/reference/cli.md): The Okareo CLI makes it easy to run model evaluation and scenario workflows in Python or TypeScript—without changing your core codebase. Designed f... - [Introduction](https://okareo.com/reference/introduction.md): Welcome to the Okareo Reference section. This area provides comprehensive technical details and guides for interacting with the Okareo platform pro... - [TypeScript SDK](https://okareo.com/reference/typescript-sdk.md): Okareo's TypeScript SDK helps you trap errors and evaluate large language models (LLMs), agents, or RAG in both structured test scenarios and real-... - [Add Datapoint](https://okareo.com/docs/api/add-datapoint-v-0-datapoints-post.md): Add a datapoint for a particular registered model - [Add Model To Group](https://okareo.com/docs/api/add-model-to-group-v-0-groups-group-id-models-post.md): Add a model to a group. - [Add Test Data Point](https://okareo.com/docs/api/add-test-data-point-v-0-test-data-point-post.md): Add a test run metric - [Admin](https://okareo.com/docs/api/admin.md): Admin - [Analytics](https://okareo.com/docs/api/analytics.md): Analytics - [Auth & API Keys](https://okareo.com/docs/api/auth-api-keys.md): Auth & API Keys - [Calibrate Check](https://okareo.com/docs/api/calibrate-check-v-0-test-runs-test-run-id-calibrate-check-post.md): Score a draft Check against a completed Test Run without saving anything. - [Chat](https://okareo.com/docs/api/chat.md): Chat - [Check Create Or Update](https://okareo.com/docs/api/check-create-or-update-v-0-check-create-or-update-post.md): Create or get check - [Check Delete](https://okareo.com/docs/api/check-delete-v-0-check-check-id-delete.md): Deletes a check - [Check Generate](https://okareo.com/docs/api/check-generate-v-0-check-generate-post.md): Generate code for an evaluator for testing. - [Check Integration Connection](https://okareo.com/docs/api/check-integration-connection-v-0-voice-integration-integration-id-test-post.md): Check a saved integration's credentials against the provider: with its - [Check Unsaved Integration Connection](https://okareo.com/docs/api/check-unsaved-integration-connection-v-0-voice-integration-test-post.md): Check settings before they are saved. Nothing is stored. - [Check Validate](https://okareo.com/docs/api/check-validate-v-0-check-validate-post.md): Validate a check without persisting. - [Checks](https://okareo.com/docs/api/checks.md): Checks - [Clone Project](https://okareo.com/docs/api/clone-project-v-0-projects-project-id-clone-post.md): Clones the Project: a new Project with the given name receives a copy of - [Compare Test Runs](https://okareo.com/docs/api/compare-test-runs-v-0-test-runs-compare-post.md): Compare one or two test runs. - [Create Api Key](https://okareo.com/docs/api/create-api-key-v-0-api-keys-post.md): Create Api Key - [Create Dashboard](https://okareo.com/docs/api/create-dashboard-v-0-dashboards-post.md): Create Dashboard - [Create Failure Scenario](https://okareo.com/docs/api/create-failure-scenario-v-0-test-run-create-failure-scenario-test-run-id-post.md): Create a ScenarioSet using the failed data points from a TestRun. - [Create Filter](https://okareo.com/docs/api/create-filter-v-0-filters-post.md): Create a new datapoint filter. - [Create Group](https://okareo.com/docs/api/create-group-v-0-groups-post.md): Create a new group for models under test or return existing group with the same name. - [Create Invite Link](https://okareo.com/docs/api/create-invite-link-v-0-admin-users-invite-link-post.md): Create Invite Link - [Create Note](https://okareo.com/docs/api/create-note-v-0-notes-post.md): Create Note - [Create Project](https://okareo.com/docs/api/create-project-v-0-projects-post.md): Create a project - [Create Provider Integration](https://okareo.com/docs/api/create-provider-integration-v-0-voice-integration-post.md): Create Provider Integration - [Create Scenario Set](https://okareo.com/docs/api/create-scenario-set-v-0-scenario-sets-post.md): Triggers a generation of a test scenario set, could be long running. - [Create Scenario Trends Dashboard](https://okareo.com/docs/api/create-scenario-trends-dashboard-v-0-dashboards-workflows-scenario-trends-post.md): Create Scenario Trends Dashboard - [Create Session](https://okareo.com/docs/api/create-session-v-0-chat-sessions-post.md): Create Session - [Create Trace Eval](https://okareo.com/docs/api/create-trace-eval-v-0-groups-group-id-trace-eval-post.md): Create a trace evaluation for a group - [Dashboards](https://okareo.com/docs/api/dashboards.md): Dashboards - [Datapoints](https://okareo.com/docs/api/datapoints.md): Datapoints - [Delete Api Key](https://okareo.com/docs/api/delete-api-key-v-0-api-keys-key-id-delete.md): Delete Api Key - [Delete Dashboard](https://okareo.com/docs/api/delete-dashboard-v-0-dashboards-dashboard-id-delete.md): Delete Dashboard - [Delete Datapoint](https://okareo.com/docs/api/delete-datapoint-v-0-datapoints-datapoint-id-delete.md): Delete a single monitoring datapoint and its associated data. - [Delete Driver Model](https://okareo.com/docs/api/delete-driver-model-v-0-driver-driver-id-delete.md): Delete Driver Model - [Delete Filter](https://okareo.com/docs/api/delete-filter-v-0-filters-delete.md): Delete a datapoint filter. - [Delete Model Under Test](https://okareo.com/docs/api/delete-model-under-test-v-0-models-under-test-mut-id-delete.md): Delete all versions of a model under test and cascade delete TestRun and TestDataPoint objects associated with all versions. - [Delete Note](https://okareo.com/docs/api/delete-note-v-0-notes-note-id-delete.md): Delete Note - [Delete Notification History Config](https://okareo.com/docs/api/delete-notification-history-config-v-0-notifications-delete.md): Update notification settings for the current project. - [Delete Provider Integration](https://okareo.com/docs/api/delete-provider-integration-v-0-voice-integration-integration-id-delete.md): Delete Provider Integration - [Delete Scenario Set](https://okareo.com/docs/api/delete-scenario-set-v-0-scenario-sets-scenario-id-delete.md): Deletes the scenario set and cascades deletes to all related entities. - [Delete Test Run](https://okareo.com/docs/api/delete-test-run-v-0-test-runs-delete.md): Deletes one or more test runs and cascades deletes to all related entities, including datapoints. - [Delete User](https://okareo.com/docs/api/delete-user-v-0-admin-users-user-id-delete.md): Delete User - [Disable User](https://okareo.com/docs/api/disable-user-v-0-admin-users-user-id-disable-post.md): Disable User - [Download File By Call Sid](https://okareo.com/docs/api/download-file-by-call-sid-v-0-voice-call-sid-call-sid-get.md): Download a voice recording by call_sid. - [Download File](https://okareo.com/docs/api/download-file-v-0-voice-file-project-id-file-id-get.md): Download File - [Drivers](https://okareo.com/docs/api/drivers.md): Drivers - [Elevenlabs Webhook](https://okareo.com/docs/api/elevenlabs-webhook-v-0-voice-elevenlabs-webhook-post.md): Webhook receiver for ElevenLabs post-call events (prio 2). - [Enable User](https://okareo.com/docs/api/enable-user-v-0-admin-users-user-id-enable-post.md): Enable User - [Evaluate](https://okareo.com/docs/api/evaluate-v-0-evaluate-post.md): Evaluate - [Find Test Data Points](https://okareo.com/docs/api/find-test-data-points-v-0-find-test-data-points-post.md): Find Test Data Points - [Find Test Run](https://okareo.com/docs/api/find-test-run-v-0-find-test-runs-post.md): Find Test Run - [Find Traces](https://okareo.com/docs/api/find-traces-v-0-find-traces-post.md): Find Traces - [Generate Driver Prompt](https://okareo.com/docs/api/generate-driver-prompt-v-0-generate-driver-prompt-post.md): Generate Driver Prompt - [Generate Scenario Set](https://okareo.com/docs/api/generate-scenario-set-v-0-scenario-sets-generate-post.md): Triggers a generation of a scenario set based on either a source scenario ID or a list of source scenario rows. - [Generate Slack Auth Url](https://okareo.com/docs/api/generate-slack-auth-url-v-0-generate-slack-auth-url-post.md): Generates a Slack authentication URL with a temporary authorization token. - [Get Account](https://okareo.com/docs/api/get-account-v-0-admin-account-get.md): Get Account - [Get Active Call](https://okareo.com/docs/api/get-active-call-v-0-voice-active-calls-call-sid-get.md): Lookup a single live call by CallSid. - [Get All Checks](https://okareo.com/docs/api/get-all-checks-v-0-checks-get.md): Get a list of checks for this organization - [Get All Drivers](https://okareo.com/docs/api/get-all-drivers-v-0-drivers-get.md): Get All Drivers - [Get All Models Under Test](https://okareo.com/docs/api/get-all-models-under-test-v-0-models-under-test-get.md): Get a list of models under test for this organization - [Get All Notification ](https://okareo.com/docs/api/get-all-notification-v-0-notifications-get.md): Retrieve a specific notification configuration. - [Get All Projects](https://okareo.com/docs/api/get-all-projects-v-0-projects-get.md): Get a list of projects for this organization - [Get Analytics Dimension Values](https://okareo.com/docs/api/get-analytics-dimension-values-v-0-analytics-dimension-values-get.md): Get Analytics Dimension Values - [Get Analytics Meta](https://okareo.com/docs/api/get-analytics-meta-v-0-analytics-meta-get.md): Get Analytics Meta - [Get Analytics Trace Attribute Keys](https://okareo.com/docs/api/get-analytics-trace-attribute-keys-v-0-analytics-trace-attribute-keys-get.md): Get Analytics Trace Attribute Keys - [Get Available Target Models](https://okareo.com/docs/api/get-available-target-models-v-0-available-target-models-get.md): Get a list of available target models from the model configuration registry. - [Get Check](https://okareo.com/docs/api/get-check-v-0-check-check-id-get.md): Get a check - [Get Dashboard](https://okareo.com/docs/api/get-dashboard-v-0-dashboards-dashboard-id-get.md): Get Dashboard - [Get Datapoint Counts](https://okareo.com/docs/api/get-datapoint-counts-v-0-filter-counts-post.md): Gets only the counts for datapoints matching given filter criteria without retrieving the actual data. - [Get Datapoints Filter](https://okareo.com/docs/api/get-datapoints-filter-v-0-find-datapoints-filter-post.md): Gets all the datapoints for given search criteria. - [Get Datapoints In Group Trace](https://okareo.com/docs/api/get-datapoints-in-group-trace-v-0-traces-group-id-datapoints-get.md): Get all datapoints in a specific group. - [Get Datapoints Summary](https://okareo.com/docs/api/get-datapoints-summary-v-0-summary-datapoints-post.md): Expects a date range and returns a summary of datapoint counts by group and feedback range. - [Get Datapoints](https://okareo.com/docs/api/get-datapoints-v-0-find-datapoints-post.md): Gets all the datapoints for given search criteria. - [Get Driver Profiles](https://okareo.com/docs/api/get-driver-profiles-v-0-driver-profiles-get.md): Get the list of available voice profiles for driver models. - [Get Driver](https://okareo.com/docs/api/get-driver-v-0-driver-identifier-get.md): Get Driver - [Get Driver Voices](https://okareo.com/docs/api/get-driver-voices-v-0-driver-voices-get.md): Get the list of available voices for driver models with metadata. - [Get Filter Group Notifications](https://okareo.com/docs/api/get-filter-group-notifications-v-0-filter-group-notifications-get.md): Get all notification settings for filter groups in the current project. - [Get Filter](https://okareo.com/docs/api/get-filter-v-0-filters-filter-group-id-get.md): Get a specific datapoint filter. - [Get Filters Metadata](https://okareo.com/docs/api/get-filters-metadata-v-0-filters-metadata-get.md): Get all viable user-defined metadata column names for filters. - [Get Filters](https://okareo.com/docs/api/get-filters-v-0-filters-get.md): Get all datapoint filters for a project. - [Get Groups](https://okareo.com/docs/api/get-groups-v-0-groups-get.md): Get all groups for the current organization and project. - [Get Model Under Test By Name And Version](https://okareo.com/docs/api/get-model-under-test-by-name-and-version-v-0-models-under-test-name-version-get.md): Get a model under test by name and version. - [Get Model Under Test](https://okareo.com/docs/api/get-model-under-test-v-0-models-under-test-mut-id-get.md): Get a model under test - [Get Models In Group](https://okareo.com/docs/api/get-models-in-group-v-0-groups-group-id-models-get.md): Get all models in a specific group. - [Get Notification History](https://okareo.com/docs/api/get-notification-history-v-0-notification-history-get.md): Get notification history for the current project and organization. - [Get Organization Settings](https://okareo.com/docs/api/get-organization-settings-v-0-organization-settings-get.md): Get the organization's settings - [Get Profile](https://okareo.com/docs/api/get-profile-v-0-admin-profile-get.md): Get Profile - [Get Project](https://okareo.com/docs/api/get-project-v-0-projects-project-id-get.md): Get a project - [Get Provider Integration](https://okareo.com/docs/api/get-provider-integration-v-0-voice-integration-integration-id-get.md): Get Provider Integration - [Get Scenario Set Data Points](https://okareo.com/docs/api/get-scenario-set-data-points-v-0-scenario-data-points-scenario-id-get.md): Get all scenarios datapoints - [Get Scenario Sets Download](https://okareo.com/docs/api/get-scenario-sets-download-v-0-scenario-sets-download-scenario-id-get.md): Get the scenario set for a particular scenario ID - [Get Scenario Sets](https://okareo.com/docs/api/get-scenario-sets-v-0-scenario-sets-get.md): Find all scenario sets based on either the project id or the scenario id - [Get Session](https://okareo.com/docs/api/get-session-v-0-chat-sessions-session-id-get.md): Get Session - [Get Target Model By Name](https://okareo.com/docs/api/get-target-model-by-name-v-0-target-target-model-name-get.md): Get Target Model By Name - [Get Template Variables](https://okareo.com/docs/api/get-template-variables-v-0-template-variables-get.md): Get Template Variables - [Get Test Data Point](https://okareo.com/docs/api/get-test-data-point-v-0-test-data-points-test-data-point-id-get.md): Get a Test Run - [Get Test Run Trace Metrics](https://okareo.com/docs/api/get-test-run-trace-metrics-v-0-test-runs-test-run-id-trace-metrics-get.md): Get aggregate trace check metrics for a test run's simulation. - [Get Test Run](https://okareo.com/docs/api/get-test-run-v-0-test-runs-test-run-id-get.md): Get a Test Run - [Get usage metrics](https://okareo.com/docs/api/get-usage-metrics-v-0-usage-metrics-get.md): Get usage metrics for the authenticated organization. - [Ingest Conversations](https://okareo.com/docs/api/ingest-conversations-v-0-conversations-ingest-post.md): Ingest voice conversations for monitoring. - [Invite User](https://okareo.com/docs/api/invite-user-v-0-admin-users-invite-post.md): Invite User - [List Active Calls](https://okareo.com/docs/api/list-active-calls-v-0-voice-active-calls-get.md): Return live Twilio + Vonage calls visible to the caller's org. - [List Api Keys](https://okareo.com/docs/api/list-api-keys-v-0-api-keys-get.md): List Api Keys - [List Chat Tool Registry](https://okareo.com/docs/api/list-chat-tool-registry-v-0-chat-tool-registry-get.md): List Chat Tool Registry - [List Dashboards](https://okareo.com/docs/api/list-dashboards-v-0-dashboards-get.md): List Dashboards - [List Messages](https://okareo.com/docs/api/list-messages-v-0-chat-sessions-session-id-messages-get.md): List Messages - [List Notes](https://okareo.com/docs/api/list-notes-v-0-notes-get.md): List Notes - [List Provider Integrations](https://okareo.com/docs/api/list-provider-integrations-v-0-voice-integrations-get.md): List Provider Integrations - [List Scenario Trends Checks](https://okareo.com/docs/api/list-scenario-trends-checks-v-0-dashboards-workflows-scenario-trends-checks-get.md): List Scenario Trends Checks - [List Target Performance Checks](https://okareo.com/docs/api/list-target-performance-checks-v-0-dashboards-workflows-target-performance-checks-get.md): List Target Performance Checks - [List Users](https://okareo.com/docs/api/list-users-v-0-admin-users-get.md): List Users - [Livekit Webhook](https://okareo.com/docs/api/livekit-webhook-v-0-voice-livekit-monitor-public-id-post.md): Webhook receiver for LiveKit egress events. - [Models](https://okareo.com/docs/api/models.md): Models - [Monitoring](https://okareo.com/docs/api/monitoring.md): Monitoring - [Move Scenario Set](https://okareo.com/docs/api/move-scenario-set-v-0-scenario-sets-scenario-id-move-post.md): Moves the Scenario Set and everything under it — its simulations (Runs), - [Notifications](https://okareo.com/docs/api/notifications.md): Notifications - [Observer Register](https://okareo.com/docs/api/observer-register-v-0-voice-observer-register-post.md): Attach a Twilio Stream subresource to a live call. - [Observer Status](https://okareo.com/docs/api/observer-status-v-0-voice-observer-status-get.md): Observer-specific state. Process liveness is covered by Okareo health. - [Observer Twilio Stream Status](https://okareo.com/docs/api/observer-twilio-stream-status-v-0-voice-observer-twilio-stream-status-post.md): Twilio Stream subresource statusCallback receiver. - [Observer Unregister](https://okareo.com/docs/api/observer-unregister-v-0-voice-observer-unregister-call-sid-post.md): Stop the Twilio Stream subresource and drop local state. - [Okareo API](https://okareo.com/docs/api/okareo-api.md): Once you start to use deliver applications that include AI models for key functionality, regression becomes a significant concern.

- [Python Examples](https://okareo.com/example/python.md): Runnable Python notebooks and toolkits, grouped by use case. Click any card to open in Colab or GitHub. - [TypeScript Examples](https://okareo.com/example/typescript.md): Runnable TypeScript examples and toolkits, grouped by use case. Click any card to open in GitHub. - [Introduction](https://okareo.com/datasets/introduction.md): In Okareo, a **scenario** is a structured representation of a dataset used to test, evaluate, and improve large language models (LLMs). Each scenar... - [Evaluation Scenarios](https://okareo.com/datasets/scenarios.md): Scenarios let you evaluate your LLMs and agents. This guide will help you decide the proper format for your scenario based on the type of evaluatio... - [Synthetic Scenarios](https://okareo.com/datasets/synthetic.md): In addition to driving evaluations, Okareo scenarios can also be used to generate synthetic data. Generated scenarios can be a powerful tool to imp... - [Classification](https://okareo.com/evaluation/classification.md): Classification is a useful first step in an agentic network. Any agent or agent network that needs to route queries to specialist agents can be eva... - [Embedding](https://okareo.com/evaluation/embedding.md): The previous guide provided a basic introduction to running retrieval evaluations in Okareo. This guide dives deeper into comparing specific embedd... - [Functions](https://okareo.com/evaluation/functions.md): Cookbook examples for this guide are available: - [Generation](https://okareo.com/evaluation/generation.md): With the availability of LLMs the generation of content has become easier than ever. However there are many dimensions - [Introduction](https://okareo.com/evaluation/introduction.md): Okareo Evaluations helps you trap errors and evaluate large language models (LLMs), agents, or RAG in both structured test scenarios and real-world... - [Multi-Agent](https://okareo.com/evaluation/multi-agent.md): Quickly connect your agent-building experience with Okareo to easily debug, evaluate, and optimize your agents. - [Retrieval](https://okareo.com/evaluation/retrieval.md): Retrieval is a vital component used to ground LLM/agent responses with relevant information. At a basic level, any retrieval system involves two co... - [Configuring the MCP](https://okareo.com/mcp/configuration.md): The Okareo MCP server is hosted at **`https://tools.okareo.com/mcp`**. You connect your coding agent to it directly — no install, no Python, no con... - [Introduction to Okareo MCP](https://okareo.com/mcp/introduction.md): The Okareo MCP server turns your coding agent into an evaluation and testing workbench. Instead of writing SDK scripts or switching to the dashboar... - [Classification Metrics](https://okareo.com/metrics/classification.md): okareo = Okareo(OKAREO_API_KEY) - [Code Checks](https://okareo.com/metrics/code-checks.md): A **Code Check** uses Python code to evaluate the generated response. Your code runs on each evaluation row and returns a result — a pass/fail bool... - [Custom Checks](https://okareo.com/metrics/custom-checks.md): If the predefined checks do not serve your needs, you can create your own. Okareo supports two types of custom checks: - [Introduction](https://okareo.com/metrics/introduction.md): Okareo evaluations span a range of input and output data types, and consequently, we use different metrics to measure the performance across these ... - [Model Checks](https://okareo.com/metrics/judge-checks.md): A **Model Check** uses a prompt template to evaluate the generated response. At runtime, a judge LLM reads your prompt -- with template variables r... - [Retrieval Metrics](https://okareo.com/metrics/retrieval.md): Performing a query on a VectorDB results in one or more embeddings being returned. The total number of returned embeddings is referred to as "k". ... - [Agentforce Session Traces](https://okareo.com/monitoring/agentforce.md): Connect a Salesforce org so Okareo collects its Agentforce session traces, links them to your simulations, and scores each turn, routing decision, ... - [Behavioral Drift vs Data Drift](https://okareo.com/monitoring/behavioral-drift.md): How behavioral drift differs from classical data drift, why LLM and agent applications need a different monitoring approach, and how Okareo measure... - [Proxy Configuration](https://okareo.com/monitoring/configuration.md): Okareo provides two ways to collect data from your application. The **Proxy** sits between your application and the LLM provider, so it captures LL... - [Getting Started](https://okareo.com/monitoring/introduction.md): :::info Map of monitoring - [Notifications](https://okareo.com/monitoring/notifications.md): Once your app is streaming data into Okareo and you've set up real-time monitoring, the next critical step is ensuring your team knows when somethi... - [Observability Overview](https://okareo.com/monitoring/observability-overview.md): This page is your **map** of monitoring in Okareo: what observability means here, the main pieces (monitors, checks, notifications, reports), and w... - [OpenTelemetry Tracing](https://okareo.com/monitoring/opentelemetry-tracing.md): Checks and monitors tell you _what_ happened: a conversation scored low, or a check failed. Traces tell you _why_. By sending OpenTelemetry traces ... - [Reports and Dashboards](https://okareo.com/monitoring/reports-dashboards.md): After you run evaluations, simulations, or stream traffic through monitoring, Okareo surfaces results in the app so you can see runs, metrics, tren... - [Voice Monitoring](https://okareo.com/monitoring/voice-monitoring.md): :::tip Test before you ship - [Introduction](https://okareo.com/projects/introduction.md): A Project is a workspace. Everything you do in Okareo happens inside one — the traffic you - [Configuration](https://okareo.com/proxy/configuration.md): The Okareo Proxy is available as a hosted cloud service or can be self managed locallly or in your managed cloud. The proxy itself is lightweight a... - [Introduction](https://okareo.com/proxy/introduction.md): The Okareo Proxy is a powerful tool designed to facilitate seamless integration and monitoring of Large Language Model (LLM) requests. It provides ... - [Adversarial Drivers](https://okareo.com/red-teaming/adversarial-drivers.md): How to author adversarial driver personas for multi-turn red-teaming scenarios. Anatomy, tactics, and concrete templates from the open-source compl... - [Validating Guardrails](https://okareo.com/red-teaming/guardrails-validation.md): The independent-test pattern for validating guardrail layers. How to measure block rate, false-positive rate, and bypass rate on the safety layer t... - [Red Teaming Overview](https://okareo.com/red-teaming/introduction.md): Programmatic red teaming as a repeatable discipline. How to scope, run, and report on adversarial testing of LLM and agent applications using Okareo. - [OWASP LLM & Agentic Top 10](https://okareo.com/red-teaming/owasp.md): How to test your LLM and agent applications against the OWASP LLM Top 10 (2025) and OWASP Agentic AI Top 10 (2026) using Okareo and the open-source... - [Creating Drivers](https://okareo.com/simulation/drivers.md): [Multi-Turn Simulation](/simulation/introduction) **Drivers** are language-model “actors” that simulate real users pushing your [Target](/simulatio... - [Custom Endpoint](https://okareo.com/simulation/custom-endpoint.md): Okareo can **drive a full conversation against _your_ running service** (RAG pipeline, tool-calling agent, or any HTTP API) by mapping requests & J... - [First Speaker](https://okareo.com/simulation/first-speaker.md): The `first_turn` parameter controls which party opens the conversation in a simulation. This is a simulation configuration that determines who send... - [Introduction](https://okareo.com/simulation/introduction.md): :::tip Building a voice agent? - [Prompt Target](https://okareo.com/simulation/prompt-target.md): Okareo lets you **simulate full back-and-forth conversations using only prompts**—no need to implement any custom API endpoints or HTTP handlers. - [Running Simulations](https://okareo.com/simulation/running-simulations.md): This page explains how many simulations run when you start a run, how to run from the UI vs the SDK, and how to cover multiple scenarios, drivers, ... - [Scheduling Simulations](https://okareo.com/simulation/scheduling.md): Run simulations on a **schedule**—daily, on push, or at a fixed time—so you get ongoing validation without manual runs. Okareo provides a CLI and S... - [AI Agent & Integration Testing](https://okareo.com/voice-simulation/ai-agent-testing.md): Voice agents sit at the front of your business. When one misfires (wrong policy, missed backend lookup, hallucinated refund status) the customer's ... - [Voice Augmentation](https://okareo.com/voice-simulation/augmentation.md): An agent that passes every test under clean audio conditions may fail the same tests under realistic ones. Voice benchmarking research consistently... - [CI Gating](https://okareo.com/voice-simulation/ci-gating.md): Automate quality gates in your CI pipeline. Run a voice simulation suite on every push, pull request, or nightly schedule, and fail the build when ... - [Experimentation and A/B Testing](https://okareo.com/voice-simulation/comparison.md): Every change to an AI agent carries risk. A new system prompt, a different LLM, an updated ASR model, a revised tool schema. Any of these can impro... - [Your First Voice Simulation](https://okareo.com/voice-simulation/first-call.md): Two minutes. A scored phone call against your voice agent, with a full transcript, recording, and quality checks. No code required. - [AI-to-Human Handoff Testing](https://okareo.com/voice-simulation/handoff-testing.md): The handoff decision is the highest-stakes judgment a voice agent makes. Getting it wrong in either direction costs real money: - [Introduction](https://okareo.com/voice-simulation/introduction.md): Voice simulations are synthetic, controllable reproductions of real customer calls. They have become a critical capability for any organization dep... - [Load Testing](https://okareo.com/voice-simulation/load-testing.md): Load testing isn't just about the AI model. It stress-tests the entire stack: network, routing engine, AI agent, CRM integrations, and agent deskto... - [Personas and Scenarios](https://okareo.com/voice-simulation/personas-scenarios.md): A test is only as good as the scenario driving it. A generic caller who asks politely about anything will make almost any agent look competent. Rea... - [Re-Scoring Past Runs](https://okareo.com/voice-simulation/rescore.md): Phone calls cost money. Re-scoring lets you try different checks against an existing run without placing a single new call. The transcripts and rec... - [Voice Targets and Channels](https://okareo.com/voice-simulation/targets.md): A voice target tells Okareo who to call and how. Everything else in a simulation (drivers, scenarios, checks, augmentation) is identical across cha... - [Verification Codes (OTP) and Notifications](https://okareo.com/voice-simulation/verification-codes.md): A voice agent sends the caller a lot that is not speech. A one-time passcode (OTP) by text before it touches the account. A confirmation number aft... - [Voice Checks](https://okareo.com/voice-simulation/checks.md): Checks score what happened in the conversation. For voice runs, three categories work together: **code checks** (deterministic Python), **model che... - [Deploy Standalone with Docker](https://okareo.com/standalone/deploy-docker.md): This guide deploys Okareo Standalone on a single host using explicit `docker run` commands. Each container is started individually with the correct... - [Deploy Standalone on Kubernetes](https://okareo.com/standalone/deploy-kubernetes.md): This guide deploys Okareo standalone with Helm on any conformant Kubernetes cluster. - [Okareo Standalone](https://okareo.com/docs/internal/standalone/standalone.md): Okareo Standalone is a self-hosted deployment mode where Okareo runs entirely inside your environment using container images you control. It is des... - [Model Configuration](https://okareo.com/standalone/model-catalog.md): Okareo uses `model.json` to define which model providers are available, how to reach them, and which models serve as targets, evaluators, and drive... - [Operations and Maintenance](https://okareo.com/standalone/operations.md): This page covers post-deployment concerns for Okareo Standalone: webhook notifications, secret handling, TLS termination, backups, upgrades, and su... - [Okareo Standalone SSO](https://okareo.com/standalone/sso.md): Okareo Standalone authenticates browser users through your Identity Provider using OpenID Connect. Your IdP decides who gets in. Your IdP policies ... - [Configure HTTPS / TLS Termination](https://okareo.com/standalone/tls.md): HTTPS is required for production Okareo Standalone deployments. OIDC providers require a secure redirect URI, and browsers will not send SSO cookie... - [Load Testing API](https://okareo.com/voice-simulation/run-load-test.md): `run_load_test` runs a **closed-loop** voice load test: it holds a target number of - [okareo.augmentations](https://okareo.com/docs/reference/python-sdk/augmentations.md): Voice simulation augmentation payload wrappers. - [okareo.checks](https://okareo.com/docs/reference/python-sdk/checks.md): ```python - [okareo.model_under_test](https://okareo.com/docs/reference/python-sdk/model_under_test.md): ```python - [Use the logger config to log completions from CrewAI or Autogen](https://okareo.com/docs/reference/python-sdk/okareo.md): ```python