# AI SDK documentation

> The TypeScript toolkit for building AI applications and agents.

## When to use AI SDK

- Category: Developer tools
- Audience: JavaScript and TypeScript developers
Common use cases:

- Generate text and structured data
- Build agents and chat interfaces
- Integrate language model providers

## Documentation

---
title: Backpressure
description: How to handle backpressure and cancellation when working with the AI SDK
url: "https://ai-sdk.dev/docs/advanced/backpressure"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

This page focuses on understanding back-pressure and cancellation when working with streams. You do not need to know this information to use the AI SDK, but for those interested, it offers a deeper dive on why and how the SDK optimally streams responses.

In the following sections, we'll explore back-pressure and cancellation in the context of a simple example program. We'll discuss the issues that can arise from an eager approach and demonstrate how a lazy approach can resolve them.

## Back-pressure and Cancellation with Streams

Let's begin by setting up a simple example program:

```jsx
// A generator that will yield positive integers
async function* integers() {
  let i = 1;
  while (true) {
    console.log(`yielding ${i}`);
    yield i++;

    await sleep(100);
  }
}
function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

// Wraps a generator into a ReadableStream
function createStream(iterator) {
  return new ReadableStream({
    async start(controller) {
      for await (const v of iterator) {
        controller.enqueue(v);
      }
      controller.close();
    },
  });
}

// Collect data from stream
async function run() {
  // Set up a stream of integers
  const stream = createStream(integers());

  // Read values from our stream
  const reader = stream.getReader();
  for (let i = 0; i < 10_000; i++) {
    // we know our stream is infinite, so there's no need to check `done`.
    const { value } = await reader.read();
    console.log(`read ${value}`);

    await sleep(1_000);
  }
}
run();
```

In this example, we create an async-generator that yields positive integers, a `ReadableStream` that wraps our integer generator, and a reader which will read values out of our stream. Notice, too, that our integer generator logs out `"yielding ${i}"`, and our reader logs out `"read ${value}"`. Both take an arbitrary amount of time to process data, represented with a 100ms sleep in our generator, and a 1sec sleep in our reader.

## Back-pressure

If you were to run this program, you'd notice something funny. We'll see roughly 10 "yield" logs for every "read" log. This might seem obvious, the generator can push values 10x faster than the reader can pull them out. But it represents a problem, our `stream` has to maintain an ever expanding queue of items that have been pushed in but not pulled out.

The problem stems from the way we wrap our generator into a stream. Notice the use of `for await (…)` inside our `start` handler. This is an **eager** for-loop, and it is constantly running to get the next value from our generator to be enqueued in our stream. This means our stream does not respect back-pressure, the signal from the consumer to the producer that more values aren't needed *yet*. We've essentially spawned a thread that will perpetually push more data into the stream, one that runs as fast as possible to push new data immediately. Worse, there's no way to signal to this thread to stop running when we don't need additional data.

To fix this, `ReadableStream` allows a `pull` handler. `pull` is called every time the consumer attempts to read more data from our stream (if there's no data already queued internally). But it's not enough to just move the `for await(…)` into `pull`, we also need to convert from an eager enqueuing to a **lazy** one. By making these 2 changes, we'll be able to react to the consumer. If they need more data, we can easily produce it, and if they don't, then we don't need to spend any time doing unnecessary work.

```jsx
function createStream(iterator) {
  return new ReadableStream({
    async pull(controller) {
      const { value, done } = await iterator.next();

      if (done) {
        controller.close();
      } else {
        controller.enqueue(value);
      }
    },
  });
}
```

Our `createStream` is a little more verbose now, but the new code is important. First, we need to manually call our `iterator.next()` method. This returns a `Promise` for an object with the type signature `{ done: boolean, value: T }`. If `done` is `true`, then we know that our iterator won't yield any more values and we must `close` the stream (this allows the consumer to know that the stream is also finished producing values). Else, we need to `enqueue` our newly produced value.

When we run this program, we see that our "yield" and "read" logs are now paired. We're no longer yielding 10x integers for every read! And, our stream now only needs to maintain 1 item in its internal buffer. We've essentially given control to the consumer, so that it's responsible for producing new values as it needs it. Neato!

## Cancellation

Let's go back to our initial eager example, with 1 small edit. Now instead of reading 10,000 integers, we're only going to read 3:

```jsx
// A generator that will yield positive integers
async function* integers() {
  let i = 1;
  while (true) {
    console.log(`yielding ${i}`);
    yield i++;

    await sleep(100);
  }
}
function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

// Wraps a generator into a ReadableStream
function createStream(iterator) {
  return new ReadableStream({
    async start(controller) {
      for await (const v of iterator) {
        controller.enqueue(v);
      }
      controller.close();
    },
  });
}
// Collect data from stream
async function run() {
  // Set up a stream of integers
  const stream = createStream(integers());

  // Read values from our stream
  const reader = stream.getReader();
  // We're only reading 3 items this time:
  for (let i = 0; i < 3; i++) {
    // we know our stream is infinite, so there's no need to check `done`.
    const { value } = await reader.read();
    console.log(`read ${value}`);

    await sleep(1000);
  }
}
run();
```

We're back to yielding 10x the number of values read. But notice now, after we've read 3 values, we're continuing to yield new values. We know that our reader will never read another value, but our stream doesn't! The eager `for await (…)` will continue forever, loudly enqueuing new values into our stream's buffer and increasing our memory usage until it consumes all available program memory.

The fix to this is exactly the same: use `pull` and manual iteration. By producing values ***lazily***, we tie the lifetime of our integer generator to the lifetime of the reader. Once the reads stop, the yields will stop too:

```jsx
// Wraps a generator into a ReadableStream
function createStream(iterator) {
  return new ReadableStream({
    async pull(controller) {
      const { value, done } = await iterator.next();

      if (done) {
        controller.close();
      } else {
        controller.enqueue(value);
      }
    },
  });
}
```

Since the solution is the same as implementing back-pressure, it shows that they're just 2 facets of the same problem: Pushing values into a stream should be done **lazily**, and doing it eagerly results in expected problems.

## Tying Stream Laziness to AI Responses

Now let's imagine you're integrating AIBot service into your product. Users will be able to prompt "count from 1 to infinity", the browser will fetch your AI API endpoint, and your servers connect to AIBot to get a response. But "infinity" is, well, infinite. The response will never end!

After a few seconds, the user gets bored and navigates away. Or maybe you're doing local development and a hot-module reload refreshes your page. The browser will have ended its connection to the API endpoint, but will your server end its connection with AIBot?

If you used the eager `for await (...)` approach, then the connection is still running and your server is asking for more and more data from AIBot. Our server spawned a "thread" and there's no signal when we can end the eager pulls. Eventually, the server is going to run out of memory (remember, there's no active fetch connection to read the buffering responses and free them).

With the lazy approach, this is taken care of for you. Because the stream will only request new data from AIBot when the consumer requests it, navigating away from the page naturally frees all resources. The fetch connection aborts and the server can clean up the response. The `ReadableStream` tied to that response can now be garbage collected. When that happens, the connection it holds to AIBot can then be freed.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Caching
description: How to handle caching when working with the AI SDK
url: "https://ai-sdk.dev/docs/advanced/caching"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Depending on the type of application you're building, you may want to cache the responses you receive from your AI provider, at least temporarily.

## Using Language Model Middleware (Recommended)

The recommended approach to caching responses is using [language model middleware](/docs/ai-sdk-core/middleware)
and the [`simulateReadableStream`](/docs/reference/ai-sdk-core/simulate-readable-stream) function.

Language model middleware is a way to enhance the behavior of language models by intercepting and modifying the calls to the language model.
Let's see how you can use language model middleware to cache responses.

```ts title="ai/middleware.ts"
import { Redis } from '@upstash/redis';
import {
  type LanguageModelV4,
  type LanguageModelV4Middleware,
  type LanguageModelV4StreamPart,
  simulateReadableStream,
} from 'ai';

const redis = new Redis({
  url: process.env.KV_URL,
  token: process.env.KV_TOKEN,
});

export const cacheMiddleware: LanguageModelV4Middleware = {
  specificationVersion: 'v4',
  wrapGenerate: async ({ doGenerate, params }) => {
    const cacheKey = JSON.stringify(params);

    const cached = (await redis.get(cacheKey)) as Awaited<
      ReturnType<LanguageModelV4['doGenerate']>
    > | null;

    if (cached !== null) {
      return {
        ...cached,
        response: {
          ...cached.response,
          timestamp: cached?.response?.timestamp
            ? new Date(cached?.response?.timestamp)
            : undefined,
        },
      };
    }

    const result = await doGenerate();

    redis.set(cacheKey, result);

    return result;
  },
  wrapStream: async ({ doStream, params }) => {
    const cacheKey = JSON.stringify(params);

    // Check if the result is in the cache
    const cached = await redis.get(cacheKey);

    // If cached, return a simulated ReadableStream that yields the cached result
    if (cached !== null) {
      // Format the timestamps in the cached response
      const formattedChunks = (cached as LanguageModelV4StreamPart[]).map(p => {
        if (p.type === 'response-metadata' && p.timestamp) {
          return { ...p, timestamp: new Date(p.timestamp) };
        } else return p;
      });
      return {
        stream: simulateReadableStream({
          initialDelayInMs: 0,
          chunkDelayInMs: 10,
          chunks: formattedChunks,
        }),
      };
    }

    // If not cached, proceed with streaming
    const { stream, ...rest } = await doStream();

    const fullResponse: LanguageModelV4StreamPart[] = [];

    const transformStream = new TransformStream<
      LanguageModelV4StreamPart,
      LanguageModelV4StreamPart
    >({
      transform(chunk, controller) {
        fullResponse.push(chunk);
        controller.enqueue(chunk);
      },
      flush() {
        // Store the full response in the cache after streaming is complete
        redis.set(cacheKey, fullResponse);
      },
    });

    return {
      stream: stream.pipeThrough(transformStream),
      ...rest,
    };
  },
};
```

This example uses `@upstash/redis` to store and retrieve the assistant's
responses but you can use any KV storage provider you would like.

This middleware caches the raw model response before AI SDK validates
structured output. When using structured output, cache only a response that
has passed your schema validation; otherwise, an invalid response can be
replayed from the cache on later requests.

`LanguageModelV4Middleware` has the `wrapGenerate` and `wrapStream` wrapping methods, plus the optional `transformParams` method. `wrapGenerate` is called when using [`generateText`](/docs/reference/ai-sdk-core/generate-text), while `wrapStream` is called when using [`streamText`](/docs/reference/ai-sdk-core/stream-text).

For `wrapGenerate`, you can cache the response directly. Instead, for `wrapStream`, you cache an array of the stream parts, which can then be used with [`simulateReadableStream`](/docs/ai-sdk-core/testing#simulate-ui-message-stream-responses) function to create a simulated `ReadableStream` that returns the cached response. In this way, the cached response is returned chunk-by-chunk as if it were being generated by the model. You can control the initial delay and delay between chunks by adjusting the `initialDelayInMs` and `chunkDelayInMs` parameters of `simulateReadableStream`.

You can see a full example of caching with Redis in a Next.js application in our [Caching Middleware Recipe](/cookbook/next/caching-middleware).

## Using Lifecycle Callbacks

Alternatively, each AI SDK Core function has special lifecycle callbacks you can use. The one of interest is likely `onEnd`, which is called when the generation is complete. This is where you can cache the full response.

Here's an example of how you can use [Upstash Redis](https://upstash.com/redis) and Next.js to cache the OpenAI response for 1 hour:

```tsx title="app/api/chat/route.ts"
import {
  convertToModelMessages,
  createTextStreamResponse,
  streamText,
  toTextStream,
  UIMessage,
} from 'ai';
import { Redis } from '@upstash/redis';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

const redis = new Redis({
  url: process.env.KV_URL,
  token: process.env.KV_TOKEN,
});

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  // come up with a key based on the request:
  const key = JSON.stringify(messages);

  // Check if we have a cached response
  const cached = (await redis.get(key)) as string | null;
  if (cached != null) {
    return new Response(cached, {
      status: 200,
      headers: { 'Content-Type': 'text/plain' },
    });
  }

  // Call the language model:
  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
    async onEnd({ text }) {
      // Cache the response text:
      await redis.set(key, text);
      await redis.expire(key, 60 * 60);
    },
  });

  // Respond with a text stream, matching the cached plain text response
  return createTextStreamResponse({
    stream: toTextStream({ stream: result.stream }),
  });
}
```

Both the cached and the fresh responses use the
[text stream protocol](/docs/ai-sdk-ui/stream-protocol#text-stream-protocol),
so use `TextStreamChatTransport` when calling this route with `useChat`.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Advanced
description: Learn how to use advanced functionality within the AI SDK and RSC API.
url: "https://ai-sdk.dev/docs/advanced"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

This section covers advanced topics and concepts for the AI SDK and RSC API. Working with LLMs often requires a different mental model compared to traditional software development.

After these concepts, you should have a better understanding of the paradigms behind the AI SDK and RSC API, and how to use them to build more AI applications.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Language Models as Routers
description: Generative User Interfaces and Language Models as Routers
url: "https://ai-sdk.dev/docs/advanced/model-as-router"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Since language models can render user interfaces as part of their generations, the resulting model generations are referred to as generative user interfaces.

In this section we will learn more about generative user interfaces and their impact on the way AI applications are built.

## Deterministic Routes and Probabilistic Routing

Generative user interfaces are not deterministic in nature because they depend on the model's generation output. Since these generations are probabilistic in nature, it is possible for every user query to result in a different user interface.

Users expect their experience using your application to be predictable, so non-deterministic user interfaces can sound like a bad idea at first. However, language models can be set up to limit their generations to a particular set of outputs using their ability to call functions.

When language models are provided with a set of function definitions and instructed to execute any of them based on user query, they do either one of the following things:

- Execute a function that is most relevant to the user query.
- Not execute any function if the user query is out of bounds of the set of functions available to them.

```tsx title='app/actions.ts'
const sendMessage = (prompt: string) =>
  generateText({
    model: "anthropic/claude-sonnet-5.5",
    instructions: 'you are a friendly weather assistant!',
    prompt,
    tools: {
      getWeather: {
        description: 'Get the weather in a location',
        inputSchema: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: async ({ location }: { location: string }) => ({
          location,
          temperature: 72 + Math.floor(Math.random() * 21) - 10,
        }),
      },
    },
  });

sendMessage('What is the weather in San Francisco?'); // getWeather is called
sendMessage('What is the weather in New York?'); // getWeather is called
sendMessage('What events are happening in London?'); // No function is called
```

This way, it is possible to ensure that the generations result in deterministic outputs, while the choice a model makes still remains to be probabilistic.

This emergent ability exhibited by a language model to choose whether a function needs to be executed or not based on a user query is believed to be models emulating "reasoning".

As a result, the combination of language models being able to reason which function to execute as well as render user interfaces at the same time gives you the ability to build applications where language models can be used as a router.

## Language Models as Routers

Historically, developers had to write routing logic that connected different parts of an application to be navigable by a user and complete a specific task.

In web applications today, most of the routing logic takes place in the form of routes:

- `/login` would navigate you to a page with a login form.
- `/user/john` would navigate you to a page with profile details about John.
- `/api/events?limit=5` would display the five most recent events from an events database.

While routes help you build web applications that connect different parts of an application into a seamless user experience, it can also be a burden to manage them as the complexity of applications grow.

Next.js has helped reduce complexity in developing with routes by introducing:

- File-based routing system
- Dynamic routing
- API routes
- Middleware
- App router, and so on...

With language models becoming better at reasoning, we believe that there is a future where developers only write core application specific components while models take care of routing them based on the user's state in an application.

With generative user interfaces, the language model decides which user interface to render based on the user's state in the application, giving users the flexibility to interact with your application in a conversational manner instead of navigating through a series of predefined routes.

### Routing by parameters

For routes like:

- `/profile/[username]`
- `/search?q=[query]`
- `/media/[id]`

that have segments dependent on dynamic data, the language model can generate the correct parameters and render the user interface.

For example, when you're in a search application, you can ask the language model to search for artworks from different artists. The language model will call the search function with the artist's name as a parameter and render the search results.

<div className="not-prose">
  #### Media Search

  Let your users see more than words can say by rendering components directly within your search experience.
</div>

### Routing by sequence

For actions that require a sequence of steps to be completed by navigating through different routes, the language model can generate the correct sequence of routes to complete in order to fulfill the user's request.

For example, when you're in a calendar application, you can ask the language model to schedule a happy hour evening with your friends. The language model will then understand your request and will perform the right sequence of [tool calls](/docs/ai-sdk-core/tools-and-tool-calling) to:

1. Lookup your calendar
2. Lookup your friends' calendars
3. Determine the best time for everyone
4. Search for nearby happy hour spots
5. Create an event and send out invites to your friends

<div className="not-prose">
  #### Planning an Event

  The model calls functions and generates interfaces based on user intent, acting like a router.
</div>

Just by defining functions to lookup contacts, pull events from a calendar, and search for nearby locations, the model is able to sequentially navigate the routes for you.

To learn more, check out these [examples](/examples/next-app/interface) using the `streamUI` function to stream generative user interfaces to the client based on the response from the language model.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Multiple Streamables
description: Learn to handle multiple streamables in your application.
url: "https://ai-sdk.dev/docs/advanced/multiple-streamables"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

## Multiple Streamable UIs

The AI SDK RSC APIs allow you to compose and return any number of streamable UIs, along with other data, in a single request. This can be useful when you want to decouple the UI into smaller components and stream them separately.

```tsx title='app/actions.tsx'
'use server';

import { createStreamableUI } from '@ai-sdk/rsc';

export async function getWeather() {
  const weatherUI = createStreamableUI();
  const forecastUI = createStreamableUI();

  weatherUI.update(<div>Loading weather...</div>);
  forecastUI.update(<div>Loading forecast...</div>);

  getWeatherData().then(weatherData => {
    weatherUI.done(<div>{weatherData}</div>);
  });

  getForecastData().then(forecastData => {
    forecastUI.done(<div>{forecastData}</div>);
  });

  // Return both streamable UIs and other data fields.
  return {
    requestedAt: Date.now(),
    weather: weatherUI.value,
    forecast: forecastUI.value,
  };
}
```

The client side code is similar to the previous example, but the [tool call](/docs/ai-sdk-core/tools-and-tool-calling) will return the new data structure with the weather and forecast UIs. Depending on the speed of getting weather and forecast data, these two components might be updated independently.

## Nested Streamable UIs

You can stream UI components within other UI components. This allows you to create complex UIs that are built up from smaller, reusable components. In the example below, we pass a `historyChart` streamable as a prop to a `StockCard` component. The StockCard can render the `historyChart` streamable, and it will automatically update as the server responds with new data.

```tsx title='app/actions.tsx'
async function getStockHistoryChart({ symbol }: { symbol: string }) {
  'use server';

  const ui = createStreamableUI(<Spinner />);

  // We need to wrap this in an async IIFE to avoid blocking.
  (async () => {
    const price = await getStockPrice({ symbol });

    // Show a spinner as the history chart for now.
    const historyChart = createStreamableUI(<Spinner />);
    ui.done(<StockCard historyChart={historyChart.value} price={price} />);

    // Getting the history data and then update that part of the UI.
    const historyData = await fetch('https://my-stock-data-api.com');
    historyChart.done(<HistoryChart data={historyData} />);
  })();

  return ui;
}
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Multistep Interfaces
description: Concepts behind building multistep interfaces
url: "https://ai-sdk.dev/docs/advanced/multistep-interfaces"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Multistep interfaces refer to user interfaces that require multiple independent steps to be executed in order to complete a specific task.

In order to understand multistep interfaces, it is important to understand two concepts:

- Tool composition
- Application context

**Tool composition** is the process of combining multiple [tools](/docs/ai-sdk-core/tools-and-tool-calling) to create a new tool. This is a powerful concept that allows you to break down complex tasks into smaller, more manageable steps.

**Application context** refers to the state of the application at any given point in time. This includes the user's input, the output of the language model, and any other relevant information.

When designing multistep interfaces, you need to consider how the tools in your application can be composed together to form a coherent user experience as well as how the application context changes as the user progresses through the interface.

## Application Context

The application context can be thought of as the conversation history between the user and the language model. The richer the context, the more information the model has to generate relevant responses.

In the context of multistep interfaces, the application context becomes even more important. This is because **the user's input in one step may affect the output of the model in the next step**.

For example, consider a meal logging application that helps users track their daily food intake. The language model is provided with the following tools:

- `log_meal` takes in parameters like the name of the food, the quantity, and the time of consumption to log a meal.
- `delete_meal` takes in the name of the meal to be deleted.

When the user logs a meal, the model generates a response confirming the meal has been logged.

```txt {2}
User: Log a chicken shawarma for lunch.
Tool: log_meal("chicken shawarma", "250g", "12:00 PM")
Model: Chicken shawarma has been logged for lunch.
```

Now when the user decides to delete the meal, the model should be able to reference the previous step to identify the meal to be deleted.

```txt {7}
User: Log a chicken shawarma for lunch.
Tool: log_meal("chicken shawarma", "250g", "12:00 PM")
Model: Chicken shawarma has been logged for lunch.
...
...
User: I skipped lunch today, can you update my log?
Tool: delete_meal("chicken shawarma")
Model: Chicken shawarma has been deleted from your log.
```

In this example, managing the application context is important for the model to generate the correct response. The model needs to have information about the previous actions in order for it to use generate the parameters for the `delete_meal` tool.

## Tool Composition

Tool composition is the process of combining multiple tools to create a new tool. This involves defining the inputs and outputs of each tool, as well as how they interact with each other.

The design of how these tools can be composed together to form a multistep interface is crucial to both the user experience of your application and the model's ability to generate the correct output.

For example, consider a flight booking assistant that can help users book flights. The assistant can be designed to have the following tools:

- `searchFlights`: Searches for flights based on the user's query.
- `lookupFlight`: Looks up details of a specific flight based on the flight number.
- `bookFlight`: Books a flight based on the user's selection.

The `searchFlights` tool is called when the user wants to lookup flights for a specific route. This would typically mean the tool should be able to take in parameters like the origin and destination of the flight.

The `lookupFlight` tool is called when the user wants to get more details about a specific flight. This would typically mean the tool should be able to take in parameters like the flight number and return information about seat availability.

The `bookFlight` tool is called when the user decides to book a flight. In order to identify the flight to book, the tool should be able to take in parameters like the flight number, trip date, and passenger details.

So the conversation between the user and the model could look like this:

```txt {8}
User: I want to book a flight from New York to London.
Tool: searchFlights("New York", "London")
Model: Here are the available flights from New York to London.
User: I want to book flight number BA123 on 12th December for myself and my wife.
Tool: lookupFlight("BA123") -> "4 seats available"
Model: Sure, there are seats available! Can you provide the names of the passengers?
User: John Doe and Jane Doe.
Tool: bookFlight("BA123", "12th December", ["John Doe", "Jane Doe"])
Model: Your flight has been booked!
```

After `lookupFlight` returns, the passenger details needed by the `bookFlight` tool are not yet available in the application context. The model asks the user for those details; once the user supplies the names, the model can call `bookFlight`.

Looking up passenger information could've been another tool that the model could've called before calling the `bookFlight` tool assuming that the user is logged into the application. This way, the model does not have to ask the user for the passenger details and can proceed with the booking.

```txt {5,6}
User: I want to book a flight from New York to London.
Tool: searchFlights("New York", "London")
Model: Here are the available flights from New York to London.
User: I want to book flight number BA123 on 12th December for myself and my wife.
Tool: lookupContacts() -> ["John Doe", "Jane Doe"]
Tool: bookFlight("BA123", "12th December", ["John Doe", "Jane Doe"])
Model: Your flight has been booked!
```

The `lookupContacts` tool is called before the `bookFlight` tool to ensure that the passenger details are available in the application context when booking the flight. This way, the model can reduce the number of steps required from the user and use its ability to call tools that populate its context and use that information to complete the booking process.

Now, let's introduce another tool called `lookupBooking` that can be used to show booking details by taking in the name of the passenger as parameter. This tool can be composed with the existing tools to provide a more complete user experience.

```txt {2-4}
User: What's the status of my wife's upcoming flight?
Tool: lookupContacts() -> ["John Doe", "Jane Doe"]
Tool: lookupBooking("Jane Doe") -> "BA123 confirmed"
Tool: lookupFlight("BA123") -> "Flight BA123 is scheduled to depart on 12th December."
Model: Your wife's flight BA123 is confirmed and scheduled to depart on 12th December.
```

In this example, the `lookupBooking` tool is used to provide the user with the status of their wife's upcoming flight. By composing this tool with the existing tools, the model is able to generate a response that includes the booking status and the departure date of the flight without requiring the user to provide additional information.

As a result, the more tools you design that can be composed together, the more complex and powerful your application can become.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Prompt Engineering
description: Learn how to engineer prompts for LLMs with the AI SDK
url: "https://ai-sdk.dev/docs/advanced/prompt-engineering"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

## What is a Large Language Model (LLM)?

A Large Language Model is essentially a prediction engine that takes a sequence of words as input and aims to predict the most likely sequence to follow. It does this by assigning probabilities to potential next sequences and then selecting one. The model continues to generate sequences until it meets a specified stopping criterion.

These models learn by training on massive text corpuses, which means they will be better suited to some use cases than others. For example, a model trained on GitHub data would understand the probabilities of sequences in source code particularly well. However, it's crucial to understand that the generated sequences, while often seeming plausible, can sometimes be random and not grounded in reality. As these models become more accurate, many surprising abilities and applications emerge.

## What is a prompt?

Prompts are the starting points for LLMs. They are the inputs that trigger the model to generate text. The scope of prompt engineering involves not just crafting these prompts but also understanding related concepts such as hidden prompts, tokens, token limits, and the potential for prompt hacking, which includes phenomena like jailbreaks and leaks.

## Why is prompt engineering needed?

Prompt engineering currently plays a pivotal role in shaping the responses of LLMs. It allows us to tweak the model to respond more effectively to a broader range of queries. This includes the use of techniques like semantic search, command grammars, and the ReActive model architecture. The performance, context window, and cost of LLMs varies between models and model providers which adds further constraints to the mix. For example, the GPT-4 model is more expensive than GPT-3.5-turbo and significantly slower, but it can also be more effective at certain tasks. And so, like many things in software engineering, there is a trade-off between cost and performance.

To assist with comparing and tweaking LLMs, we've built an AI playground that allows you to compare the performance of different models side-by-side online. When you're ready, you can even generate code with the AI SDK to quickly use your prompt and your selected model into your own applications.

## Example: Build a Slogan Generator

### Start with an instruction

Imagine you want to build a slogan generator for marketing campaigns. Creating catchy slogans isn't always straightforward!

First, you'll need a prompt that makes it clear what you want. Let's start with an instruction. Submit this prompt to generate your first completion.

Not bad! Now, try making your instruction more specific.

Introducing a single descriptive term to our prompt influences the completion. Essentially, crafting your prompt is the means by which you "instruct" or "program" the model.

### Include examples

Clear instructions are key for quality outcomes, but that might not always be enough. Let's try to enhance your instruction further.

These slogans are fine, but could be even better. It appears the model overlooked the 'live' part in our prompt. Let's change it slightly to generate more appropriate suggestions.

Often, it's beneficial to both demonstrate and tell the model your requirements. Incorporating examples in your prompt can aid in conveying patterns or subtleties. Test this prompt that carries a few examples.

Great! Incorporating examples of expected output for a certain input prompted the model to generate the kind of names we aimed for.

### Tweak your settings

Apart from designing prompts, you can influence completions by tweaking model settings. A crucial setting is the **temperature**.

You might have seen that the same prompt, when repeated, yielded the same or nearly the same completions. This happens when your temperature is at 0.

Attempt to re-submit the identical prompt a few times with temperature set to 1.

Notice the difference? With a temperature above 0, the same prompt delivers varied completions each time.

Keep in mind that the model forecasts the text most likely to follow the preceding text. Temperature, a value from 0 to 1, essentially governs the model's confidence level in making these predictions. A lower temperature implies lesser risks, leading to more precise and deterministic completions. A higher temperature yields a broader range of completions.

For your slogan generator, you might want a large pool of name suggestions. A moderate temperature of 0.6 should serve well.

## Recommended Resources

Prompt Engineering is evolving rapidly, with new methods and research papers surfacing every week. Here are some resources that we've found useful for learning about and experimenting with prompt engineering:

- [The Vercel AI Playground](/playground)
- [Brex Prompt Engineering](https://github.com/brexhq/prompt-engineering)
- [Prompt Engineering Guide by Dair AI](https://www.promptingguide.ai/)

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Rate Limiting
description: Learn how to rate limit your application.
url: "https://ai-sdk.dev/docs/advanced/rate-limiting"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Rate limiting helps you protect your APIs from abuse. It involves setting a
maximum threshold on the number of requests a client can make within a
specified timeframe. This simple technique acts as a gatekeeper,
preventing excessive usage that can degrade service performance and incur
unnecessary costs.

## Rate Limiting with Upstash Redis and Upstash Ratelimit

In this example, you will protect an API endpoint using [Upstash Redis](https://upstash.com/redis) and [Upstash Ratelimit](https://github.com/upstash/ratelimit).

```tsx title='app/api/generate/route.ts'
import {
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
} from 'ai';
import { Ratelimit } from '@upstash/ratelimit';
import { Redis } from '@upstash/redis';
import { NextRequest } from 'next/server';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

// Create Rate limit
const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.fixedWindow(5, '30s'),
});

export async function POST(req: NextRequest) {
  // call ratelimit with request ip
  const ip = req.ip ?? 'ip';
  const { success, remaining } = await ratelimit.limit(ip);

  // block the request if unsuccessful
  if (!success) {
    return new Response('Ratelimited!', { status: 429 });
  }

  const { messages } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages,
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

## Simplify API Protection

With Upstash Redis and Upstash Ratelimit, it is possible to protect your APIs
from such attacks with ease. To learn more about how Ratelimit works and
how it can be configured to your needs, see [Ratelimit Documentation](https://upstash.com/docs/oss/sdks/ts/ratelimit/overview).

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Rendering UI with Language Models
description: Rendering UI with Language Models
url: "https://ai-sdk.dev/docs/advanced/rendering-ui-with-language-models"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Language models generate text, so at first it may seem like you would only need to render text in your application.

```tsx {16} title="app/actions.tsx"
const text = generateText({
  model: "anthropic/claude-sonnet-5.5",
  instructions: 'You are a friendly assistant',
  prompt: 'What is the weather in SF?',
  tools: {
    getWeather: {
      description: 'Get the weather for a location',
      inputSchema: z.object({
        city: z.string().describe('The city to get the weather for'),
        unit: z
          .enum(['C', 'F'])
          .describe('The unit to display the temperature in'),
      }),
      execute: async ({ city, unit }) => {
        const weather = getWeather({ city, unit });
        return `It is currently ${weather.value}°${unit} and ${weather.description} in ${city}!`;
      },
    },
  },
});
```

Above, the language model is passed a [tool](/docs/ai-sdk-core/tools-and-tool-calling) called `getWeather` that returns the weather information as text. However, instead of returning text, if you return a JSON object that represents the weather information, you can use it to render a React component instead.

```tsx {18-23} title="app/action.ts"
const text = generateText({
  model: "anthropic/claude-sonnet-5.5",
  instructions: 'You are a friendly assistant',
  prompt: 'What is the weather in SF?',
  tools: {
    getWeather: {
      description: 'Get the weather for a location',
      inputSchema: z.object({
        city: z.string().describe('The city to get the weather for'),
        unit: z
          .enum(['C', 'F'])
          .describe('The unit to display the temperature in'),
      }),
      execute: async ({ city, unit }) => {
        const weather = getWeather({ city, unit });
        const { temperature, description, forecast } = weather;

        return {
          temperature,
          unit,
          description,
          forecast,
        };
      },
    },
  },
});
```

Now you can use the object returned by the `getWeather` function to conditionally render a React component `<WeatherCard/>` that displays the weather information by passing the object as props.

```tsx title="app/page.tsx"
return (
  <div>
    {messages.map(message => {
      // Check assistant message parts for tool results
      if (message.role === 'assistant') {
        return message.parts.map(part => {
          if (
            part.type === 'tool-weather' &&
            part.state === 'output-available'
          ) {
            const { temperature, unit, description, forecast } = part.output;

            return (
              <WeatherCard
                weather={{
                  temperature,
                  unit,
                  description,
                  forecast,
                }}
              />
            );
          }
        });
      }
    })}
  </div>
);
```

Here's a little preview of what that might look like.

<div className="not-prose flex flex-col">
  #### Weather

  An example of an assistant that renders the weather information in a streamed component.
</div>

Rendering interfaces as part of language model generations elevates the user experience of your application, allowing people to interact with language models beyond text.

They also make it easier for you to interpret [sequential tool calls](/docs/ai-sdk-rsc/multistep-interfaces) that take place in multiple steps and help identify and debug where the model reasoned incorrectly.

## Rendering Multiple User Interfaces

To recap, an application has to go through the following steps to render user interfaces as part of model generations:

1. The user prompts the language model.
2. The language model generates a response that includes a tool call.
3. The tool call returns a JSON object that represents the user interface.
4. The response is sent to the client.
5. The client receives the response and checks if the latest message was a tool call.
6. If it was a tool call, the client renders the user interface based on the JSON object returned by the tool call.

Most applications have multiple tools that are called by the language model, and each tool can return a different user interface.

For example, a tool that searches for courses can return a list of courses, while a tool that searches for people can return a list of people. As this list grows, the complexity of your application will grow as well and it can become increasingly difficult to manage these user interfaces.

```tsx title='app/page.tsx'
{
  message.parts.map(part => {
    if (part.state !== 'output-available') return null;

    switch (part.type) {
      case 'tool-api-search-course':
        return <Courses courses={part.output} />;
      case 'tool-api-search-profile':
        return <People people={part.output} />;
      case 'tool-api-meetings':
        return <Meetings meetings={part.output} />;
      case 'tool-api-search-building':
        return <Buildings buildings={part.output} />;
      case 'tool-api-events':
        return <Events events={part.output} />;
      case 'tool-api-meals':
        return <Meals meals={part.output} />;
      case 'text':
        return <div>{part.text}</div>;
      default:
        return null;
    }
  });
}
```

## Rendering User Interfaces on the Server

The **AI SDK RSC (`@ai-sdk/rsc`)** takes advantage of RSCs to solve the problem of managing all your React components on the client side, allowing you to render React components on the server and stream them to the client.

Rather than conditionally rendering user interfaces on the client based on the data returned by the language model, you can directly stream them from the server during a model generation.

```tsx {3,22-31,38} title="app/action.ts"
import { createStreamableUI } from '@ai-sdk/rsc';

const uiStream = createStreamableUI();

const text = generateText({
  model: "anthropic/claude-sonnet-5.5",
  instructions: 'you are a friendly assistant',
  prompt: 'what is the weather in SF?',
  tools: {
    getWeather: {
      description: 'Get the weather for a location',
      inputSchema: z.object({
        city: z.string().describe('The city to get the weather for'),
        unit: z
          .enum(['C', 'F'])
          .describe('The unit to display the temperature in'),
      }),
      execute: async ({ city, unit }) => {
        const weather = getWeather({ city, unit });
        const { temperature, description, forecast } = weather;

        uiStream.done(
          <WeatherCard
            weather={{
              temperature: 47,
              unit: 'F',
              description: 'sunny',
              forecast,
            }}
          />,
        );
      },
    },
  },
});

return {
  display: uiStream.value,
};
```

The [`createStreamableUI`](/docs/reference/ai-sdk-rsc/create-streamable-ui) function belongs to the `@ai-sdk/rsc` module and creates a stream that can send React components to the client.

On the server, you render the `<WeatherCard/>` component with the props passed to it, and then stream it to the client. On the client side, you only need to render the UI that is streamed from the server.

```tsx title="app/page.tsx" {4}
return (
  <div>
    {messages.map(message => (
      <div>{message.display}</div>
    ))}
  </div>
);
```

Now the steps involved are simplified:

1. The user prompts the language model.
2. The language model generates a response that includes a tool call.
3. The tool call renders a React component along with relevant props that represent the user interface.
4. The response is streamed to the client and rendered directly.

> **Note:** You can also render text on the server and stream it to the client using React Server Components. This way, all operations from language model generation to UI rendering can be done on the server, while the client only needs to render the UI that is streamed from the server.

Check out this [example](/examples/next-app/interface/stream-component-updates) for a full illustration of how to stream component updates with React Server Components in Next.js App Router.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Secure URL Fetching
description: How the AI SDK protects server-side fetches of URLs returned by model providers, and how to harden your deployment further.
url: "https://ai-sdk.dev/docs/advanced/secure-url-fetching"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Many providers return a **URL in their response body** — a generated image,
audio, or video to download, or a polling URL to check job status. The AI SDK
fetches these server-side and returns the result to your code. Because that URL
comes from an external service, a malicious or compromised provider (or anyone
able to tamper with the response) could point it at an internal address such as
a cloud-metadata endpoint (`http://169.254.169.254/…`), a private host
(`http://10.0.0.5/…`), or `localhost`.

To prevent that, the SDK validates every response-supplied URL before fetching
it. This happens automatically inside the provider packages — you don't need to
configure anything.

For authenticated task-status polling, providers can construct the first URL
from the configured API endpoint. The SDK trusts that configured origin for the
initial request, but manually follows and validates every redirect away from it.
MiniMax, Kling AI, and ByteDance video polling use this protected path.

## What the SDK protects against

When the SDK fetches a URL taken from a provider response, it:

- **Rejects private, loopback, and link-local targets** — IPv4 (`10/8`,
  `172.16/12`, `192.168/16`, `127/8`, `169.254/16`, CGNAT, multicast, …) and the
  equivalent IPv6 ranges, plus `localhost`, `.local`, `.localhost`, and
  `.internal` hostnames. Non-`http(s)` schemes are rejected too.
- **Re-validates every redirect hop** — a URL that passes but then redirects to
  an internal address is blocked; the redirect is never followed blindly.
- **Validates DNS at connection time on Node.js** — every resolved address is
  checked, and the socket is pinned to the validated DNS result so DNS
  rebinding cannot introduce a different address between validation and
  connection.
- **Strips risky request headers** — proxy-forwarding, cloud-metadata, and
  cookie headers are removed before the request.
- **Drops credentials across origins** — caller headers (`Authorization`,
  `Cookie`, and provider-specific API-key headers alike) are not sent to a host
  on a different origin than the provider's; a redirect that crosses origin
  drops all of them except the user-agent.

A blocked URL surfaces as a `DownloadError`.

## Fetching untrusted URLs directly

For custom integrations using `@ai-sdk/provider-utils`, use `fetchUntrustedUrl`
when the URL comes from a response or other untrusted input. It validates URLs
and redirects and withholds credentials and unknown custom headers on the first
request unless the URL matches a configured origin:

```ts
import { fetchUntrustedUrl } from '@ai-sdk/provider-utils';

const response = await fetchUntrustedUrl({
  url: result.downloadUrl,
  headers: {
    authorization: `Bearer ${apiKey}`,
    accept: 'image/*',
  },
  credentialedOrigin: providerBaseURL,
});
```

Here, `Authorization` is sent only when the initial URL has the same origin as
`providerBaseURL`. Otherwise, only allowlisted metadata such as `Accept`,
`Range`, and `User-Agent` is sent. A cross-origin redirect drops all caller
headers except `User-Agent`, and credentials are never reattached on later hops.

`credentialedOrigin` must come from your configuration, never from the response
URL. It does not disable URL validation. For a configured private endpoint, pass
`trustedOrigin` as well; if `credentialedOrigin` is omitted, `fetchUntrustedUrl`
uses `trustedOrigin` as the origin allowed to receive credentials. Extra protocol
metadata can be allowed with `untrustedFirstHopHeaders: ['x-protocol-version']`.
Only list headers that are safe to disclose to any destination; use a configured
origin for credentials.

The existing `fetchWithValidatedRedirects` helper preserves its first-request
behavior: it sanitizes risky headers but still forwards authentication and
custom headers. Switch to `fetchUntrustedUrl` to opt into first-request
credential isolation. Existing calls to `getFromApi` also retain their behavior;
pass `credentialedOrigin` when fetching response-supplied URLs with credentials.

## Self-hosted and local endpoints

URLs that are same-origin with the provider endpoint **you configured** (e.g. a
custom `baseURL` pointing at a self-hosted or `localhost` deployment) are
exempt from these checks — they target exactly the host you told the SDK to
talk to. This also applies to task-status polling. Any redirect off that origin
is still validated before the redirected request is sent.

## DNS validation across runtimes

On Node.js, the default validated download fetch uses `node:dns` and an
`undici` connector hook to validate every resolved address at connection time.
The connector uses those exact results, closing both hostname-to-private-IP and
DNS-rebinding bypasses.

Wrapping or replacing global `fetch` does not disable this protection: the
default Node.js download transport is independent of global `fetch`.

Bun, Deno, Cloudflare Workers, and framework edge runtimes use their platform
fetch, even when they expose a Node-compatible `process` object.

If you explicitly inject a custom `fetch`, it is responsible for equivalent DNS
validation and connection pinning. Other runtimes do not expose Node's
DNS/socket hooks, so server deployments on those runtimes should restrict
network egress to private, loopback, link-local, and cloud-metadata ranges.

## Hardening your deployment

If your server fetches provider-supplied URLs and you want to close the DNS
gaps, use one (ideally both) of these:

### 1. Restrict outbound egress at the network layer

Deny your server's network egress to `169.254.0.0/16`, RFC-1918 ranges, and
loopback. This is the most robust control and is independent of application
code.

### 2. Harden an injected `fetch`

The Node.js default is already pinned. If you explicitly inject a custom
`fetch`, back it with an `undici`
`Agent` whose `connect.lookup` validates the resolved IP and lets the socket
connect only to a safe address — closing both the hostname-to-private and the
DNS-rebinding windows:

```ts
import { Agent, fetch as undiciFetch } from 'undici';
import { lookup } from 'node:dns';

// Your own check that returns true for private/loopback/link-local addresses.
declare function isUnsafeAddress(ip: string): boolean;

const safeLookup: typeof lookup = (hostname, options, callback) => {
  lookup(hostname, options as any, (err, address, family) => {
    if (!err && typeof address === 'string' && isUnsafeAddress(address)) {
      callback(new Error(`Refusing to connect to ${address}`), '', 0);
      return;
    }
    (callback as any)(err, address, family);
  });
};

const safeDispatcher = new Agent({ connect: { lookup: safeLookup } });

const safeFetch: typeof fetch = (input, init) =>
  undiciFetch(input, { ...init, dispatcher: safeDispatcher }) as any;
```

```ts
import { createFal } from '@ai-sdk/fal';

const fal = createFal({ fetch: safeFetch });
```

The SDK's URL validation and your custom fetch's connect-time pinning are
complementary — keep both.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Sequential Generations
description: "Learn how to implement sequential generations (\"chains\") with the AI SDK"
url: "https://ai-sdk.dev/docs/advanced/sequential-generations"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

When working with the AI SDK, you may want to create sequences of generations (often referred to as "chains" or "pipes"), where the output of one becomes the input for the next. This can be useful for creating more complex AI-powered workflows or for breaking down larger tasks into smaller, more manageable steps.

## Example

In a sequential chain, the output of one generation is directly used as input for the next generation. This allows you to create a series of dependent generations, where each step builds upon the previous one.

Here's an example of how you can implement sequential actions:

```typescript
import { generateText } from 'ai';

async function sequentialActions() {
  // Generate blog post ideas
  const ideasGeneration = await generateText({
    model: "anthropic/claude-sonnet-5.5",
    prompt: 'Generate 10 ideas for a blog post about making spaghetti.',
  });

  console.log('Generated Ideas:\n', ideasGeneration);

  // Pick the best idea
  const bestIdeaGeneration = await generateText({
    model: "anthropic/claude-sonnet-5.5",
    prompt: `Here are some blog post ideas about making spaghetti:
${ideasGeneration}

Pick the best idea from the list above and explain why it's the best.`,
  });

  console.log('\nBest Idea:\n', bestIdeaGeneration);

  // Generate an outline
  const outlineGeneration = await generateText({
    model: "anthropic/claude-sonnet-5.5",
    prompt: `We've chosen the following blog post idea about making spaghetti:
${bestIdeaGeneration}

Create a detailed outline for a blog post based on this idea.`,
  });

  console.log('\nBlog Post Outline:\n', outlineGeneration);
}

sequentialActions().catch(console.error);
```

In this example, we first generate ideas for a blog post, then pick the best idea, and finally create an outline based on that idea. Each step uses the output from the previous step as input for the next generation.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Stopping Streams
description: Learn how to cancel streams with the AI SDK
url: "https://ai-sdk.dev/docs/advanced/stopping-streams"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Canceling ongoing streams is often needed.
For example, users might want to stop a stream when they realize that the response is not what they want.

The different parts of the AI SDK support canceling streams in different ways.

## AI SDK Core

The AI SDK functions have an `abortSignal` argument that you can use to cancel a stream.
You would use this if you want to cancel a stream from the server side to the LLM API, e.g. by
forwarding the `abortSignal` from the request.

```tsx {10,11,12-16}
import { createTextStreamResponse, streamText, toTextStream } from 'ai';

export async function POST(req: Request) {
  const { prompt } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    prompt,
    // forward the abort signal:
    abortSignal: req.signal,
    onAbort: ({ steps }) => {
      // Handle cleanup when stream is aborted
      console.log('Stream aborted after', steps.length, 'steps');
      // Persist partial results to database
    },
  });

  return createTextStreamResponse({
    stream: toTextStream({ stream: result.stream }),
  });
}
```

## AI SDK UI

The hooks, e.g. `useChat` or `useCompletion`, provide a `stop` helper function that can be used to cancel a stream.
This aborts the HTTP request from the client. To also stop the model request on the server, your server runtime must propagate the client disconnect to the request's `AbortSignal`, and your route must forward that signal to the AI SDK Core call as shown above.

Stream abort functionality is not compatible with stream resumption. If you're
using `resume: true` in `useChat`, the abort functionality will break the
resumption mechanism. Choose either abort or resume functionality, but not
both.

```tsx title="app/page.tsx" {6,11-14}
'use client';

import { useCompletion } from '@ai-sdk/react';

export default function Chat() {
  const { input, completion, stop, status, handleSubmit, handleInputChange } =
    useCompletion();

  return (
    <div>
      {(status === 'submitted' || status === 'streaming') && (
        <button type="button" onClick={() => stop()}>
          Stop
        </button>
      )}
      {completion}
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
      </form>
    </div>
  );
}
```

### Vercel

On Vercel, [request cancellation](https://vercel.com/docs/functions/functions-api-reference#cancel-requests) is only supported in the Node.js runtime and must be enabled for each function that needs it. Add `supportsCancellation` to the function's configuration in `vercel.json`:

```json title="vercel.json"
{
  "functions": {
    "app/api/chat/route.ts": {
      "supportsCancellation": true
    }
  }
}
```

With cancellation enabled, calling `stop()` aborts the client request, Vercel aborts `req.signal`, and forwarding `req.signal` as `abortSignal` cancels the model request.

Without `supportsCancellation`, `stop()` still stops the client-side stream
but the server-side generation may continue.

## Handling stream abort cleanup

When streams are aborted, you may need to perform cleanup operations such as persisting partial results or cleaning up resources. The `onAbort` callback provides a way to handle these scenarios on the server side.

Unlike `onEnd`, which is called when a stream completes normally, `onAbort` is specifically called when a stream is aborted via `AbortSignal`. This distinction allows you to handle normal completion and aborted streams differently.

For UI message streams (`toUIMessageStreamResponse`), the `onEnd` callback
also receives an `isAborted` parameter that indicates whether the stream was
aborted. This allows you to handle both completion and abort scenarios in a
single callback.

```tsx {8-12}
import { streamText } from 'ai';

const result = streamText({
  model: "anthropic/claude-sonnet-5.5",
  prompt: 'Write a long story...',
  abortSignal: controller.signal,
  onAbort: async ({ steps }) => {
    // Called when stream is aborted - persist partial results
    await savePartialResults(steps);
    await logAbortEvent(steps.length);
  },
  onEnd: async ({ steps, totalUsage }) => {
    // Called when stream completes normally
    await saveFinalResults(steps, totalUsage);
  },
});
```

The `onAbort` callback receives:

- `steps`: Array of all completed steps before the abort occurred

This is particularly useful for:

- Persisting partial conversation history to database
- Saving partial progress for later continuation
- Cleaning up server-side resources or connections
- Logging abort events for analytics

You can also handle abort events directly in the stream using the `abort` stream part:

```tsx {6-9}
for await (const part of result.stream) {
  switch (part.type) {
    case 'text-delta':
      // Handle text delta content
      break;
    case 'abort':
      // Handle abort event directly in stream
      console.log('Stream was aborted');
      break;
    // ... other cases
  }
}
```

## UI Message Streams

When using `toUIMessageStream`, you need to handle stream abortion slightly differently. The `onEnd` callback receives an `isAborted` parameter, and you should pass `consumeStream` to `createUIMessageStreamResponse` to ensure proper abort handling:

```tsx {3,21,24-30,34}
import { openai } from '@ai-sdk/openai';
import {
  consumeStream,
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
    abortSignal: req.signal,
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      onEnd: async ({ isAborted }) => {
        if (isAborted) {
          console.log('Stream was aborted');
          // Handle abort-specific cleanup
        } else {
          console.log('Stream completed normally');
          // Handle normal completion
        }
      },
    }),
    consumeSseStream: consumeStream,
  });
}
```

The `consumeStream` function is necessary for proper abort handling in UI message streams. It ensures that the stream is properly consumed even when aborted, preventing potential memory leaks or hanging connections.

The `onEnd` callback distinguishes consumer cancellation from an observed
abort. When the consumer cancels the UI message stream before an outcome is
declared, such as during a client disconnect, `isCancelled` is `true`,
`outcome.status` remains `'unknown'`, and `isAborted` remains `false`. When the
stream observes an `abort` part first, `outcome.status` is `'aborted'`,
`isAborted` is `true`, and `isCancelled` is absent. Check both flags when the
same cleanup should run for either case.

## AI SDK RSC

The AI SDK RSC does not currently support stopping streams.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Vercel Deployment Guide
description: Learn how to deploy an AI application to production on Vercel
url: "https://ai-sdk.dev/docs/advanced/vercel-deployment-guide"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

In this guide, you will deploy an AI application to [Vercel](https://vercel.com) using [Next.js](https://nextjs.org) (App Router).

Vercel is a platform for developers that provides the tools, workflows, and infrastructure you need to build and deploy your web apps faster, without the need for additional configuration.

Vercel allows for automatic deployments on every branch push and merges onto the production branch of your GitHub, GitLab, and Bitbucket projects. It is a great option for deploying your AI application.

## Before You Begin

To follow along with this guide, you will need:

- a Vercel account
- an account with a Git provider (this tutorial will use [GitHub](https://github.com))
- an OpenAI API key

This guide will teach you how to deploy the application you built in the Next.js (App Router) quickstart tutorial to Vercel. If you haven’t completed the quickstart guide, you can start with [this repo](https://github.com/vercel-labs/ai-sdk-deployment-guide).

## Commit Changes

Vercel offers a powerful git-centered workflow that automatically deploys your application to production every time you push to your repository’s main branch.

Before committing your local changes, make sure that you have a `.gitignore`. Within your `.gitignore`, ensure that you are excluding your environment variables (`.env`) and your node modules (`node_modules`).

If you have any local changes, you can commit them by running the following commands:

```bash
git add .
git commit -m "init"
```

## Create Git Repo

You can create a GitHub repository from within your terminal, or on [github.com](https://github.com/). For this tutorial, you will use the GitHub CLI ([more info here](https://cli.github.com/)).

To create your GitHub repository:

1. Navigate to [github.com](https://github.com/)
2. In the top right corner, click the "plus" icon and select "New repository"
3. Pick a name for your repository (this can be anything)
4. Click "Create repository"

Once you have created your repository, GitHub will redirect you to your new repository.

1. Scroll down the page and copy the commands under the title "...or push an existing repository from the command line"
2. Go back to the terminal, paste and then run the commands

Note: if you run into the error "error: remote origin already exists.", this is because your local repository is still linked to the repository you cloned. To "unlink", you can run the following command:

```bash
rm -rf .git
git init
git add .
git commit -m "init"
```

Rerun the code snippet from the previous step.

## Import Project in Vercel

On the [New Project](https://vercel.com/new) page, under the **Import Git Repository** section, select the Git provider that you would like to import your project from. Follow the prompts to sign in to your GitHub account.

Once you have signed in, you should see your newly created repository from the previous step in the "Import Git Repository" section. Click the "Import" button next to that project.

### Add Environment Variables

Your application uses environment secrets to store your OpenAI API key using a `.env.local` file locally in development. To add this API key to your production deployment, expand the "Environment Variables" section and paste in your `.env.local` file. Vercel will automatically parse your variables and enter them in the appropriate `key:value` format.

### Deploy

Press the **Deploy** button. Vercel will create the Project and deploy it based on the chosen configurations.

### Enjoy the confetti!

To view your deployment, select the Project in the dashboard and then select the **Domain**. This page is now visible to anyone who has the URL.

## Considerations

When deploying an AI application, there are infrastructure-related considerations to be aware of.

### Function Duration

In most cases, you will call the large language model (LLM) on the server. By default, Vercel serverless functions have a maximum duration of 10 seconds on the Hobby Tier. Depending on your prompt, it can take an LLM more than this limit to complete a response. If the response is not resolved within this limit, the server will throw an error.

You can specify the maximum duration of your Vercel function using [route segment config](https://nextjs.org/docs/app/api-reference/file-conventions/route-segment-config). To update your maximum duration, add the following route segment config to the top of your route handler or the page which is calling your server action.

```ts
export const maxDuration = 30;
```

You can increase the max duration to 60 seconds on the Hobby Tier. For other tiers, [see the documentation](https://vercel.com/docs/functions/runtimes#max-duration) for limits.

### Request Cancellation

AI SDK UI helpers such as `useChat` and `useCompletion` abort the client request when you call `stop()`. To propagate that cancellation to a Vercel Function and its model request, you must:

1. Use the Node.js runtime, which is the default for Next.js route handlers.
2. Enable [`supportsCancellation`](https://vercel.com/docs/functions/functions-api-reference#cancel-requests) for the route in `vercel.json`.
3. Forward the route's `req.signal` to `streamText` or `generateText` as `abortSignal`.

For example, enable cancellation for your chat route:

```json title="vercel.json"
{
  "functions": {
    "app/api/chat/route.ts": {
      "supportsCancellation": true
    }
  }
}
```

Then forward the request signal to the AI SDK:

```ts title="app/api/chat/route.ts" {10}
import { convertToModelMessages, streamText, type UIMessage } from 'ai';

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: convertToModelMessages(messages),
    abortSignal: req.signal,
  });

  return result.toUIMessageStreamResponse();
}
```

Without `supportsCancellation`, calling `stop()` closes the client-side stream but does not cancel the Vercel Function or model request. See [Stopping Streams](/docs/advanced/stopping-streams) for the complete client and server setup.

## Security Considerations

Given the high cost of calling an LLM, it's important to have measures in place that can protect your application from abuse.

### Rate Limit

Rate limiting is a method used to regulate network traffic by defining a maximum number of requests that a client can send to a server within a given time frame.

Follow [this guide](https://vercel.com/guides/securing-ai-app-rate-limiting) to add rate limiting to your application.

### Firewall

A firewall helps protect your applications and websites from DDoS attacks and unauthorized access.

[Vercel Firewall](https://vercel.com/docs/security/vercel-firewall) is a set of tools and infrastructure, created specifically with security in mind. It automatically mitigates DDoS attacks and Enterprise teams can get further customization for their site, including dedicated support and custom rules for IP blocking.

## Troubleshooting

- Streaming not working when [proxied](/docs/troubleshooting/streaming-not-working-when-proxied)
- Experiencing [Timeouts](/docs/troubleshooting/timeout-on-vercel)

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Building Agents
description: Complete guide to creating agents with the ToolLoopAgent.
url: "https://ai-sdk.dev/docs/agents/building-agents"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

The ToolLoopAgent provides a structured way to encapsulate LLM configuration, tools, and behavior into reusable components. It handles the agent loop for you, allowing the LLM to call tools multiple times in sequence to accomplish complex tasks. Define agents once and use them across your application.

## Why Use the ToolLoopAgent Class?

When building AI applications, you often need to:

- **Reuse configurations** - Same model settings, tools, and prompts across different parts of your application
- **Maintain consistency** - Ensure the same behavior and capabilities throughout your codebase
- **Simplify API routes** - Reduce boilerplate in your endpoints
- **Type safety** - Get full TypeScript support for your agent's tools and outputs

The ToolLoopAgent class provides a single place to define your agent's behavior.

## Creating an Agent

Define an agent by instantiating the ToolLoopAgent class with your desired configuration:

```ts
import { ToolLoopAgent } from 'ai';

const myAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: 'You are a helpful assistant.',
  tools: {
    // Your tools here
  },
});
```

## Configuration Options

The ToolLoopAgent accepts all the same settings as `generateText` and `streamText`. Configure:

### Model and System Instructions

```ts
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: 'You are an expert software engineer.',
});
```

### Tools

Provide tools that the agent can use to accomplish tasks:

```ts
import { ToolLoopAgent, tool } from 'ai';
import { z } from 'zod';

const codeAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    runCode: tool({
      description: 'Execute Python code',
      inputSchema: z.object({
        code: z.string(),
      }),
      execute: async ({ code }) => {
        // Execute code and return result
        return { output: 'Code executed successfully' };
      },
    }),
  },
});
```

### Context and Agent State

Use `runtimeContext` as the agent's shared runtime state. It flows through the
agent loop and is available in `prepareStep`, lifecycle callbacks, and final
results. If a tool needs server-side values such as credentials, scoped
permissions, or default settings, pass them through `toolsContext` and declare
them with the tool's `contextSchema`.

Set `runtimeContext` and `toolsContext` in the `ToolLoopAgent` constructor.

```ts
import { ToolLoopAgent, tool } from 'ai';
import { z } from 'zod';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    searchTickets: tool({
      description: 'Search support tickets',
      inputSchema: z.object({
        query: z.string(),
      }),
      contextSchema: z.object({
        apiKey: z.string(),
        accountId: z.string(),
      }),
      execute: async ({ query }, { context }) =>
        searchTickets(query, context.accountId, context.apiKey),
    }),
  },
  runtimeContext: {
    requestId: 'req_abc',
    escalated: false,
  },
  toolsContext: {
    searchTickets: {
      apiKey: process.env.SUPPORT_API_KEY!,
      accountId: 'acct_123',
    },
  },
  prepareStep: async ({ runtimeContext }) => {
    if (runtimeContext.escalated) {
      return { temperature: 0.1 };
    }

    return {};
  },
});

const result = await agent.generate({
  prompt: 'Find open billing tickets for this account.',
});
```

For context that changes with each invocation, use `callOptionsSchema` and
`prepareCall`. See [Configuring Call Options](/docs/agents/configuring-call-options).

Model call settings returned from `prepareStep`, such as `temperature`, apply
only to the current step. Later steps use the agent's top-level setting unless
they return another override.

For the full model, including sensitive context filtering and where each context
value is available, see [Runtime and Tool
Context](/docs/ai-sdk-core/runtime-and-tool-context).

### Tools That Use Experimental Sandboxes

Pass `experimental_sandbox` when an agent tool needs a command or code execution
environment. The experimental sandbox is a per-call value, so provide it to
`generate()`, `stream()`, or the agent UI stream helper that invokes the agent.

```ts {13,19-23,31}
const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: 'You are a coding assistant. Use the shell tool when needed.',
  tools: {
    shell: tool({
      description: 'Execute shell commands in the experimental sandbox.',
      inputSchema: z.object({
        command: z.string(),
        workingDirectory: z.string().optional(),
      }),
      execute: async (
        { command, workingDirectory },
        { abortSignal, experimental_sandbox },
      ) => {
        if (!experimental_sandbox) {
          throw new Error('Experimental sandbox is not available');
        }

        return experimental_sandbox.run({
          command,
          workingDirectory,
          abortSignal,
        });
      },
    }),
  },
});

const result = await agent.generate({
  prompt: `Run the tests.\n\nSandbox:\n${experimental_sandbox.description}`,
  experimental_sandbox,
});
```

The experimental sandbox description is not added to the model prompt automatically. Include
it in the prompt or instructions when the model needs to know environment
details. Passing an experimental sandbox does not sandbox the tool itself; the tool must
explicitly delegate operations to the experimental sandbox. See the
[Experimental Sandbox section in Tool Calling](/docs/ai-sdk-core/tools-and-tool-calling#experimental-sandbox)
for more details.

You can also require approval before a tool executes. Configure approval on the
`ToolLoopAgent` with `toolApproval`:

```ts
const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    runCode: tool({
      description: 'Execute Python code',
      inputSchema: z.object({
        code: z.string(),
      }),
      execute: async ({ code }) => ({ output: code }),
    }),
  },
  toolApproval: {
    runCode: 'user-approval',
  },
});
```

For manual approvals, automatic approvals and denials, dynamic policy functions,
and `useChat` integration, see [Tool
Approvals](/docs/agents/tool-approvals).

### Loop Control

By default, agents run for 20 steps (`stopWhen: isStepCount(20)`). In each step, the model either generates text or calls a tool. If it generates text, the agent completes. If it calls a tool, the AI SDK executes that tool.

You can configure `stopWhen` differently to allow more steps. After each tool execution, the agent triggers a new generation where the model can call another tool or generate text:

```ts
import { ToolLoopAgent, isStepCount } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  stopWhen: isStepCount(50), // Increase default from 20 to 50.
});
```

Each step represents one generation (which results in either text or a tool call). The loop continues until:

- A finish reasoning other than tool-calls is returned, or
- A tool that is invoked does not have an execute function, or
- A tool call needs approval, or
- A stop condition is met

You can combine multiple conditions:

```ts
import { ToolLoopAgent, isStepCount } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  stopWhen: [
    isStepCount(20), // Maximum 20 steps
    yourCustomCondition(), // Custom logic for when to stop
  ],
});
```

Learn more about [loop control and stop conditions](/docs/agents/loop-control).

### Tool Choice

Control how the agent uses tools:

```ts
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    // your tools here
  },
  toolChoice: 'required', // Force tool use
  // or toolChoice: 'none' to disable tools
  // or toolChoice: 'auto' (default) to let the model decide
});
```

You can also force the use of a specific tool:

```ts
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    weather: weatherTool,
    cityAttractions: attractionsTool,
  },
  toolChoice: {
    type: 'tool',
    toolName: 'weather', // Force the weather tool to be used
  },
});
```

### Structured Output

Define structured output schemas:

```ts
import { ToolLoopAgent, Output } from 'ai';
import { z } from 'zod';

const analysisAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  output: Output.object({
    schema: z.object({
      sentiment: z.enum(['positive', 'neutral', 'negative']),
      summary: z.string(),
      keyPoints: z.array(z.string()),
    }),
  }),
});

const { output } = await analysisAgent.generate({
  prompt: 'Analyze customer feedback from the last quarter',
});
```

## Define Agent Behavior with System Instructions

System instructions define your agent's behavior, personality, and constraints. They set the context for all interactions and guide how the agent responds to user queries and uses tools.

### Basic System Instructions

Set the agent's role and expertise:

```ts
const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions:
    'You are an expert data analyst. You provide clear insights from complex data.',
});
```

### Detailed Behavioral Instructions

Provide specific guidelines for agent behavior:

```ts
const codeReviewAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: `You are a senior software engineer conducting code reviews.

  Your approach:
  - Focus on security vulnerabilities first
  - Identify performance bottlenecks
  - Suggest improvements for readability and maintainability
  - Be constructive and educational in your feedback
  - Always explain why something is an issue and how to fix it`,
});
```

### Constrain Agent Behavior

Set boundaries and ensure consistent behavior:

```ts
const customerSupportAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: `You are a customer support specialist for an e-commerce platform.

  Rules:
  - Never make promises about refunds without checking the policy
  - Always be empathetic and professional
  - If you don't know something, say so and offer to escalate
  - Keep responses concise and actionable
  - Never share internal company information`,
  tools: {
    checkOrderStatus,
    lookupPolicy,
    createTicket,
  },
});
```

### Tool Usage Instructions

Guide how the agent should use available tools:

```ts
const researchAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: `You are a research assistant with access to search and document tools.

  When researching:
  1. Always start with a broad search to understand the topic
  2. Use document analysis for detailed information
  3. Cross-reference multiple sources before drawing conclusions
  4. Cite your sources when presenting information
  5. If information conflicts, present both viewpoints`,
  tools: {
    webSearch,
    analyzeDocument,
    extractQuotes,
  },
});
```

### Format and Style Instructions

Control the output format and communication style:

```ts
const technicalWriterAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: `You are a technical documentation writer.

  Writing style:
  - Use clear, simple language
  - Avoid jargon unless necessary
  - Structure information with headers and bullet points
  - Include code examples where relevant
  - Write in second person ("you" instead of "the user")

  Always format responses in Markdown.`,
});
```

## Using an Agent

Once defined, you can use your agent in three ways:

### Generate Text

Use `generate()` for one-time text generation:

```ts
const result = await myAgent.generate({
  prompt: 'What is the weather like?',
});

console.log(result.text);
```

### Stream Text

Use `stream()` for streaming responses:

```ts
const result = await myAgent.stream({
  prompt: 'Tell me a story',
});

for await (const chunk of result.textStream) {
  console.log(chunk);
}
```

### Respond to UI Messages

Use `createAgentUIStreamResponse()` to create API responses for client applications:

```ts
// In your API route (e.g., app/api/chat/route.ts)
import { createAgentUIStreamResponse } from 'ai';

export async function POST(request: Request) {
  const { messages } = await request.json();

  return createAgentUIStreamResponse({
    agent: myAgent,
    uiMessages: messages,
  });
}
```

### Lifecycle Callbacks

Agents provide lifecycle callbacks that let you hook into different phases of the agent execution.
These are useful for logging, observability, debugging, and custom telemetry.

```ts
const result = await myAgent.generate({
  prompt: 'Research and summarize the latest AI trends',

  onStart({ modelId }) {
    console.log('Agent started', { modelId });
  },

  onStepStart({ stepNumber, modelId }) {
    console.log(`Step ${stepNumber} starting`, { modelId });
  },

  onToolExecutionStart({ toolCall }) {
    console.log(`Tool call starting: ${toolCall.toolName}`);
  },

  onToolExecutionEnd({ toolCall, toolExecutionMs, toolOutput }) {
    console.log(
      `Tool call finished: ${toolCall.toolName} (${toolExecutionMs}ms)`,
      {
        success: toolOutput.type === 'tool-result',
      },
    );
  },

  onStepEnd({ stepNumber, usage, performance, finishReason, toolCalls }) {
    console.log(`Step ${stepNumber} completed:`, {
      inputTokens: usage.inputTokens,
      outputTokens: usage.outputTokens,
      outputTokensPerSecond: performance.effectiveOutputTokensPerSecond,
      stepTimeMs: performance.stepTimeMs,
      finishReason,
      toolsUsed: toolCalls?.map(tc => tc.toolName),
    });
  },

  onEnd({ usage, steps }) {
    console.log('Agent finished:', {
      totalSteps: steps.length,
      totalTokens: usage.totalTokens,
    });
  },
});
```

The available lifecycle callbacks are:

- **`onStart`**: Called once when the agent operation begins, before any LLM calls. Receives model info, messages, settings, and `runtimeContext`.
- **`onStepStart`**: Called before each step (LLM call). Receives the step number, model, messages being sent, tools, and prior steps.
- **`onToolExecutionStart`**: Called right before a tool's `execute` function runs. Receives the tool call object, messages, and `toolContext`.
- **`onToolExecutionEnd`**: Called right after a tool's `execute` function completes or errors. Receives the tool call, `toolExecutionMs`, and a `toolOutput` discriminated union (`type: 'tool-result'` with `output`, or `type: 'tool-error'` with `error`).
- **`onStepEnd`**: Called after each step finishes. Receives step results including usage, performance, finish reason, and tool calls.
- **`onEnd`**: Called when all steps are finished and the response is complete. Receives all step results, total usage, and `runtimeContext`.

For the full event data reference, see [Lifecycle Callbacks](/docs/ai-sdk-core/lifecycle-callbacks). `ToolLoopAgent` uses the same generation lifecycle event types for `onStart`, `onStepStart`, `onToolExecutionStart`, `onToolExecutionEnd`, `onStepEnd`, and `onEnd`.

#### Constructor vs. Method Callbacks

All lifecycle callbacks can be defined in the constructor for agent-wide tracking, in the `generate()`/`stream()` call for per-call tracking, or both. When both are provided, both are called (constructor first, then the method callback):

```ts
const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  onStepEnd: async ({ stepNumber, usage }) => {
    // Agent-wide logging
    console.log(`Agent step ${stepNumber}:`, usage.totalTokens);
  },
});

// Method-level callback runs after constructor callback
const result = await agent.generate({
  prompt: 'Hello',
  onStepEnd: async ({ stepNumber, usage }) => {
    // Per-call tracking (e.g., for billing)
    await trackUsage(stepNumber, usage);
  },
});
```

## End-to-end Type Safety

You can infer types for your agent's `UIMessage`s:

```ts
import { ToolLoopAgent, InferAgentUIMessage } from 'ai';

const myAgent = new ToolLoopAgent({
  // ... configuration
});

// Infer the UIMessage type for UI components or persistence
export type MyAgentUIMessage = InferAgentUIMessage<typeof myAgent>;
```

Use this type in your client components with `useChat`:

```tsx title="components/chat.tsx"
'use client';

import { useChat } from '@ai-sdk/react';
import type { MyAgentUIMessage } from '@/agent/my-agent';

export function Chat() {
  const { messages } = useChat<MyAgentUIMessage>();
  // Full type safety for your messages and tools
}
```

## Next Steps

Now that you understand building agents, you can:

- Explore [workflow patterns](/docs/agents/workflows) for structured patterns using core functions
- Learn about [loop control](/docs/agents/loop-control) for advanced execution control
- See [manual loop examples](/cookbook/node/manual-agent-loop) for custom workflow implementations

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Configuring Call Options
description: Pass type-safe runtime inputs to dynamically configure agent behavior.
url: "https://ai-sdk.dev/docs/agents/configuring-call-options"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Call options allow you to pass type-safe structured inputs to your agent. Use them to dynamically modify any agent setting based on the specific request.

## Why Use Call Options?

When you need agent behavior to change based on runtime inputs:

- **Add dynamic context** - Inject retrieved documents, user preferences, or session data into prompts
- **Select models dynamically** - Choose faster or more capable models based on request complexity
- **Configure tools per request** - Pass user location to search tools or adjust tool behavior
- **Customize provider options** - Set reasoning effort, temperature, or other provider-specific settings

Without call options, you'd need to create multiple agents or handle configuration logic outside the agent.

## How It Works

Define call options in three steps:

1. **Define the schema** - Specify what inputs you accept using `callOptionsSchema`
2. **Configure with `prepareCall`** - Use those inputs to modify agent settings
3. **Pass options at runtime** - Provide the options when calling `generate()` or `stream()`

## Basic Example

Add user context to your agent's prompt at runtime:

```ts
import { ToolLoopAgent } from 'ai';
import { z } from 'zod';

const supportAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  callOptionsSchema: z.object({
    userId: z.string(),
    accountType: z.enum(['free', 'pro', 'enterprise']),
  }),
  instructions: 'You are a helpful customer support agent.',
  prepareCall: ({ options, ...settings }) => ({
    ...settings,
    instructions:
      settings.instructions +
      `\nUser context:
- Account type: ${options.accountType}
- User ID: ${options.userId}

Adjust your response based on the user's account level.`,
  }),
});

// Call the agent with specific user context
const result = await supportAgent.generate({
  prompt: 'How do I upgrade my account?',
  options: {
    userId: 'user_123',
    accountType: 'free',
  },
});
```

The `options` parameter is now required and type-checked. If you don't provide it or pass incorrect types, TypeScript will error.

## Modifying Agent Settings

Use `prepareCall` to modify any agent setting. The returned object replaces the call settings, so spread the incoming settings and override only the ones you want to change.

### Dynamic Model Selection

Choose models based on request characteristics:

```ts
import { ToolLoopAgent } from 'ai';
import { z } from 'zod';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5", // Default model
  callOptionsSchema: z.object({
    complexity: z.enum(['simple', 'complex']),
  }),
  prepareCall: ({ options, ...settings }) => ({
    ...settings,
    model:
      options.complexity === 'simple'
        ? 'openai/gpt-6-luna'
        : 'openai/gpt-6-astra',
  }),
});

// Use faster model for simple queries
await agent.generate({
  prompt: 'What is 2+2?',
  options: { complexity: 'simple' },
});

// Use more capable model for complex reasoning
await agent.generate({
  prompt: 'Explain quantum entanglement',
  options: { complexity: 'complex' },
});
```

### Dynamic Tool Configuration

Configure tools based on runtime inputs:

```ts
import { openai } from '@ai-sdk/openai';
import { ToolLoopAgent } from 'ai';
import { z } from 'zod';

const newsAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  callOptionsSchema: z.object({
    userCity: z.string().optional(),
    userRegion: z.string().optional(),
  }),
  tools: {
    web_search: openai.tools.webSearch(),
  },
  prepareCall: ({ options, ...settings }) => ({
    ...settings,
    tools: {
      web_search: openai.tools.webSearch({
        searchContextSize: 'low',
        userLocation: {
          type: 'approximate',
          city: options.userCity,
          region: options.userRegion,
          country: 'US',
        },
      }),
    },
  }),
});

await newsAgent.generate({
  prompt: 'What are the top local news stories?',
  options: {
    userCity: 'San Francisco',
    userRegion: 'California',
  },
});
```

### Provider-Specific Options

Configure provider settings dynamically:

```ts
import { OpenAILanguageModelResponsesOptions } from '@ai-sdk/openai';
import { ToolLoopAgent } from 'ai';
import { z } from 'zod';

const agent = new ToolLoopAgent({
  model: 'openai/gpt-6-astra',
  callOptionsSchema: z.object({
    taskDifficulty: z.enum(['low', 'medium', 'high']),
  }),
  prepareCall: ({ options, ...settings }) => ({
    ...settings,
    providerOptions: {
      openai: {
        reasoningEffort: options.taskDifficulty,
      } satisfies OpenAILanguageModelResponsesOptions,
    },
  }),
});

await agent.generate({
  prompt: 'Analyze this complex scenario...',
  options: { taskDifficulty: 'high' },
});
```

## Advanced Patterns

### Retrieval Augmented Generation (RAG)

Fetch relevant context and inject it into your prompt:

```ts
import { ToolLoopAgent } from 'ai';
import { z } from 'zod';

const ragAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  callOptionsSchema: z.object({
    query: z.string(),
  }),
  prepareCall: async ({ options, ...settings }) => {
    // Fetch relevant documents (this can be async)
    const documents = await vectorSearch(options.query);

    return {
      ...settings,
      instructions: `Answer questions using the following context:

${documents.map(doc => doc.content).join('\n\n')}`,
    };
  },
});

await ragAgent.generate({
  prompt: 'What is our refund policy?',
  options: { query: 'refund policy' },
});
```

The `prepareCall` function can be async, enabling you to fetch data before configuring the agent.

### Combining Multiple Modifications

Modify multiple settings together:

```ts
import { ToolLoopAgent } from 'ai';
import { z } from 'zod';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  callOptionsSchema: z.object({
    userRole: z.enum(['admin', 'user']),
    urgency: z.enum(['low', 'high']),
  }),
  tools: {
    readDatabase: readDatabaseTool,
    writeDatabase: writeDatabaseTool,
  },
  prepareCall: ({ options, ...settings }) => ({
    ...settings,
    // Upgrade model for urgent requests
    model: options.urgency === 'high' ? "anthropic/claude-sonnet-5.5" : settings.model,
    // Limit tools based on user role
    activeTools:
      options.userRole === 'admin'
        ? ['readDatabase', 'writeDatabase']
        : ['readDatabase'],
    // Adjust instructions
    instructions: `You are a ${options.userRole} assistant.
${options.userRole === 'admin' ? 'You have full database access.' : 'You have read-only access.'}`,
  }),
});

await agent.generate({
  prompt: 'Update the user record',
  options: {
    userRole: 'admin',
    urgency: 'high',
  },
});
```

## Using with createAgentUIStreamResponse

Pass call options through API routes to your agent:

```ts title="app/api/chat/route.ts"
import { createAgentUIStreamResponse } from 'ai';
import { myAgent } from '@/ai/agents/my-agent';

export async function POST(request: Request) {
  const { messages, userId, accountType } = await request.json();

  return createAgentUIStreamResponse({
    agent: myAgent,
    messages,
    options: {
      userId,
      accountType,
    },
  });
}
```

## Next Steps

- Learn about [loop control](/docs/agents/loop-control) for execution management
- Explore [workflow patterns](/docs/agents/workflows) for complex multi-step processes

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Loop Control
description: Control agent execution with built-in loop management using stopWhen and prepareStep
url: "https://ai-sdk.dev/docs/agents/loop-control"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

You can control both the execution flow and the settings at each step of the agent loop. The loop continues until:

- A finish reasoning other than tool-calls is returned, or
- A tool that is invoked does not have an execute function, or
- A tool call needs approval, or
- A stop condition is met

The AI SDK provides built-in loop control through two parameters: `stopWhen` for defining stopping conditions and `prepareStep` for modifying settings (model, tools, messages, and more) between steps.

## Stop Conditions

The `stopWhen` parameter controls when to stop execution when there are tool results in the last step. By default, `ToolLoopAgent` stops after 20 steps using `isStepCount(20)`. This default is a safety measure to prevent runaway loops that could result in excessive API calls and costs.

If this default limit stops a tool loop that could otherwise continue, the SDK logs a warning with guidance on configuring `stopWhen`. Set `stopWhen` explicitly to choose a limit, even if you want to keep `isStepCount(20)`. Explicit stop conditions do not produce this warning.

`WorkflowAgent` does not apply a default step limit. It continues until the model stops calling tools or another natural termination condition is met. Configure an explicit condition such as `isStepCount(20)` when you need to bound its model calls. See [WorkflowAgent Loop Control](/docs/agents/workflow-agent#loop-control) for details.

When you provide `stopWhen`, the agent continues executing after tool calls until a stopping condition is met. When the condition is an array, execution stops when any of the conditions are met.

### Use Built-in Conditions

The AI SDK provides several built-in stopping conditions:

- `isStepCount(count)` — stops after a specified number of steps
- `hasToolCall(...toolNames)` — stops when any of the specified tools is called
- `isLoopFinished()` — never triggers, letting the loop run until the agent is naturally finished

### Run Up to a Maximum Number of Steps

```ts
import { ToolLoopAgent, isStepCount } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    // your tools
  },
  stopWhen: isStepCount(50), // Increase ToolLoopAgent's default from 20 to 50.
});

const result = await agent.generate({
  prompt: 'Analyze this dataset and create a summary report',
});
```

### Run Until Finished

If you want a `ToolLoopAgent` to run until the model naturally stops making tool calls, use `isLoopFinished()`. This removes its default step limit:

```ts
import { ToolLoopAgent, isLoopFinished } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    // your tools
  },
  stopWhen: isLoopFinished(), // No maximum step limit.
});

const result = await agent.generate({
  prompt: 'Analyze this dataset and create a summary report',
});
```

Use `isLoopFinished()` with caution. Without a step limit, the agent could
potentially run indefinitely or incur significant costs if the model keeps
making tool calls.

### Combine Multiple Conditions

Combine multiple stopping conditions. The loop stops when it meets any condition:

```ts
import { ToolLoopAgent, isStepCount, hasToolCall } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    // your tools
  },
  stopWhen: [
    isStepCount(20), // Maximum 20 steps
    hasToolCall('someTool', 'done'), // Stop after calling either tool
  ],
});

const result = await agent.generate({
  prompt: 'Research and analyze the topic',
});
```

### Create Custom Conditions

Build custom stopping conditions for specific requirements:

```ts
import { ToolLoopAgent, StopCondition, ToolSet } from 'ai';

const tools = {
  // your tools
} satisfies ToolSet;

const hasAnswer: StopCondition<typeof tools> = ({ steps }) => {
  // Stop when the model generates text containing "ANSWER:"
  return steps.some(step => step.text?.includes('ANSWER:')) ?? false;
};

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools,
  stopWhen: hasAnswer,
});

const result = await agent.generate({
  prompt: 'Find the answer and respond with "ANSWER: [your answer]"',
});
```

Custom conditions receive step information across all steps:

```ts
const budgetExceeded: StopCondition<typeof tools> = ({ steps }) => {
  const totalUsage = steps.reduce(
    (acc, step) => ({
      inputTokens: acc.inputTokens + (step.usage?.inputTokens ?? 0),
      outputTokens: acc.outputTokens + (step.usage?.outputTokens ?? 0),
    }),
    { inputTokens: 0, outputTokens: 0 },
  );

  const costEstimate =
    (totalUsage.inputTokens * 0.01 + totalUsage.outputTokens * 0.03) / 1000;
  return costEstimate > 0.5; // Stop if cost exceeds $0.50
};
```

## Prepare Step

The `prepareStep` callback runs before each step in the loop and defaults to the initial settings if you don't return any changes. Use it to modify settings, manage context, or implement dynamic behavior based on execution history.

It receives `messages` for the current step, plus `initialMessages` and `responseMessages` when you need to distinguish the original input from assistant/tool messages accumulated in earlier steps. Treat `messages` as the loop's current message state: if you return a `messages` override, that override persists as the base for later steps, together with the response messages from each completed step.

### Dynamic Model Selection

Switch models based on step requirements:

```ts
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: 'openai/gpt-6-luna', // Default model
  tools: {
    // your tools
  },
  prepareStep: async ({ stepNumber, messages }) => {
    // Use a stronger model for complex reasoning after initial steps
    if (stepNumber > 2 && messages.length > 10) {
      return {
        model: "anthropic/claude-sonnet-5.5",
      };
    }
    // Continue with default settings
    return {};
  },
});

const result = await agent.generate({
  prompt: '...',
});
```

### Model Call Settings

Override provider-agnostic model call settings for an individual step. This can
be useful when tool-calling steps need more deterministic sampling than the
final response:

```ts
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  temperature: 0.7,
  tools: {
    // your tools
  },
  prepareStep: async ({ stepNumber }) => {
    if (stepNumber === 0) {
      return {
        temperature: 0,
        maxOutputTokens: 300,
      };
    }

    return {};
  },
});

const result = await agent.generate({
  prompt: '...',
});
```

`prepareStep` can override `maxOutputTokens`, `temperature`, `topP`, `topK`,
`presencePenalty`, `frequencyPenalty`, `stopSequences`, `seed`, and
`reasoning`. These overrides apply only to the current step. When a setting is
omitted or `undefined`, the top-level value is used for that step. Defined
falsy values such as `temperature: 0`, `seed: 0`, and an empty
`stopSequences` array are preserved.

### Context Management

Long-running agents can accumulate large tool results, reasoning parts, and assistant messages. Use `prepareStep` to mutate the message state that will be used by later steps. This is useful for compaction, and you decide when compaction should happen.

The `messages` value contains the messages that will be sent for the current step. When you return a `messages` override from `prepareStep`, that changed list becomes the base for later steps. New assistant and tool response messages are appended to it as the loop continues. If you need to rebuild a step from the discrete pieces instead of the persisted message state, use `initialMessages` for the original input and `responseMessages` for the model/tool response messages accumulated so far.

The `pruneMessages` helper provides a built-in way to remove selected messages. You can use it inside `prepareStep` when you want a simple compaction strategy.

```ts
import { ToolLoopAgent, pruneMessages, type ModelMessage } from 'ai';

const COMPACTION_THRESHOLD = 100_000;

const estimateTokens = (messages: ModelMessage[]) => {
  return JSON.stringify(messages).length / 4;
};

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    // your tools
  },
  prepareStep: async ({ messages }) => {
    if (estimateTokens(messages) > COMPACTION_THRESHOLD) {
      return {
        messages: pruneMessages({
          messages,
          reasoning: 'all',
          toolCalls: 'before-last-3-messages',
          emptyMessages: 'remove',
        }),
      };
    }
  },
});

const result = await agent.generate({
  prompt: '...',
});
```

The token estimator above is intentionally simple and only demonstrates one way to decide when to compact. The main point is that `prepareStep` can return a new `messages` array, and that array becomes the message state for following steps. The same pattern also works with `generateText` and `streamText`.

### Tool Selection

Control which tools are available at each step:

```ts
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    search: searchTool,
    analyze: analyzeTool,
    summarize: summarizeTool,
  },
  prepareStep: async ({ stepNumber, steps }) => {
    // Search phase (steps 0-2)
    if (stepNumber <= 2) {
      return {
        activeTools: ['search'],
        toolChoice: 'required',
      };
    }

    // Analysis phase (steps 3-5)
    if (stepNumber <= 5) {
      return {
        activeTools: ['analyze'],
      };
    }

    // Summary phase (step 6+)
    return {
      activeTools: ['summarize'],
      toolChoice: 'required',
    };
  },
});

const result = await agent.generate({
  prompt: '...',
});
```

You can also force a specific tool to be used:

```ts
prepareStep: async ({ stepNumber }) => {
  if (stepNumber === 0) {
    // Force the search tool to be used first
    return {
      toolChoice: { type: 'tool', toolName: 'search' },
    };
  }

  if (stepNumber === 5) {
    // Force the summarize tool after analysis
    return {
      toolChoice: { type: 'tool', toolName: 'summarize' },
    };
  }

  return {};
};
```

### Message Modification

Transform messages before sending them to the model. Returned messages carry forward to later steps, so later `messages` values include your transformed messages plus the assistant/tool response messages from completed steps:

```ts
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    // your tools
  },
  prepareStep: async ({ messages, stepNumber }) => {
    // Summarize tool results to reduce token usage
    const processedMessages = messages.map(msg => {
      if (msg.role === 'tool' && msg.content.length > 1000) {
        return {
          ...msg,
          content: summarizeToolResult(msg.content),
        };
      }
      return msg;
    });

    return { messages: processedMessages };
  },
});

const result = await agent.generate({
  prompt: '...',
});
```

### Experimental Sandbox Selection

Switch the experimental sandbox used for tool execution in a single step:

```ts
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    runCommand,
  },
  experimental_sandbox: defaultSandbox,
  prepareStep: async ({ stepNumber }) => {
    if (stepNumber === 0) {
      return {
        experimental_sandbox: setupSandbox,
      };
    }

    return {};
  },
});

const result = await agent.generate({
  prompt: '...',
});
```

Unlike `runtimeContext` and `toolsContext`, an experimental sandbox returned from `prepareStep`
only applies to tool execution in that step. Later steps use the top-level
`experimental_sandbox` unless they return their own experimental sandbox override.

## Access Step Information

Both `stopWhen` and `prepareStep` receive detailed information about the current execution:

```ts
prepareStep: async ({
  model, // Current model configuration
  stepNumber, // Current step number (0-indexed)
  steps, // All previous steps with their results
  messages, // Messages to be sent to the model
}) => {
  // Access previous tool calls and results
  const previousToolCalls = steps.flatMap(step => step.toolCalls);
  const previousResults = steps.flatMap(step => step.toolResults);

  // Make decisions based on execution history
  if (previousToolCalls.some(call => call.toolName === 'dataAnalysis')) {
    return {
      toolChoice: { type: 'tool', toolName: 'reportGenerator' },
    };
  }

  return {};
},
```

## Forced Tool Calling

You can force the agent to always use tools by combining `toolChoice: 'required'` with a `done` tool that has no `execute` function. This pattern ensures the agent uses tools for every step and stops only when it explicitly signals completion.

```ts
import { ToolLoopAgent, tool } from 'ai';
import { z } from 'zod';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    search: searchTool,
    analyze: analyzeTool,
    done: tool({
      description: 'Signal that you have finished your work',
      inputSchema: z.object({
        answer: z.string().describe('The final answer'),
      }),
      // No execute function - stops the agent when called
    }),
  },
  toolChoice: 'required', // Force tool calls at every step
});

const result = await agent.generate({
  prompt: 'Research and analyze this topic, then provide your answer.',
});

// extract answer from done tool call
const toolCall = result.staticToolCalls[0]; // tool call from final step
if (toolCall?.toolName === 'done') {
  console.log(toolCall.input.answer);
}
```

Key aspects of this pattern:

- **`toolChoice: 'required'`**: Forces the model to call a tool at every step instead of generating text directly. This ensures the agent follows a structured workflow.
- **`done` tool without `execute`**: A tool that has no `execute` function acts as a termination signal. When the agent calls this tool, the loop stops because there's no function to execute.
- **Accessing results**: The final answer is available in `result.staticToolCalls`, which contains tool calls that weren't executed.

This pattern is useful when you want the agent to always use specific tools for operations (like code execution or data retrieval) rather than attempting to answer directly.

## Manual Loop Control

For scenarios requiring complete control over the agent loop, you can use AI SDK Core functions (`generateText` and `streamText`) to implement your own loop management instead of using `stopWhen` and `prepareStep`. This approach provides maximum flexibility for complex workflows.

### Implementing a Manual Loop

Build your own agent loop when you need full control over execution:

```ts
import { generateText, ModelMessage } from 'ai';

const messages: ModelMessage[] = [{ role: 'user', content: '...' }];

let step = 0;
const maxSteps = 10;

while (step < maxSteps) {
  const result = await generateText({
    model: "anthropic/claude-sonnet-5.5",
    messages,
    tools: {
      // your tools here
    },
  });

  messages.push(...result.responseMessages);

  if (result.text) {
    break; // Stop when model generates text
  }

  step++;
}
```

This manual approach gives you complete control over:

- Message history management
- Step-by-step decision making
- Custom stopping conditions
- Dynamic tool and model selection
- Error handling and recovery

[Learn more about manual agent loops in the cookbook](/cookbook/node/manual-agent-loop).

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Memory
description: Add persistent memory to your agent using provider-defined tools, memory providers, or a custom tool.
url: "https://ai-sdk.dev/docs/agents/memory"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Memory lets your agent save information and recall it later. Without memory, every conversation starts fresh. With memory, your agent builds context over time, recalls previous interactions, and adapts to the user.

## Three Approaches

You can add memory to your agent with the AI SDK in three ways, each with different tradeoffs:

| Approach                                          | Effort | Flexibility | Provider Lock-in           |
| ------------------------------------------------- | ------ | ----------- | -------------------------- |
| [Provider-Defined Tools](#provider-defined-tools) | Low    | Medium      | Yes                        |
| [Memory Providers](#memory-providers)             | Low    | Low         | Depends on memory provider |
| [Custom Tool](#custom-tool)                       | High   | High        | No                         |

## Provider-Defined Tools

[Provider-defined tools](/docs/foundations/tools#types-of-tools) are tools where the provider specifies the tool's `inputSchema` and `description`, but you provide the `execute` function. The model has been trained to use these tools, which can result in better performance compared to custom tools.

### Anthropic Memory Tool

The [Anthropic Memory Tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool) gives Claude a structured interface for managing a `/memories` directory. Claude reads its memory before starting tasks, creates and updates files as it works, and references them in future conversations.

```ts
import { anthropic } from '@ai-sdk/anthropic';
import { ToolLoopAgent } from 'ai';

const memory = anthropic.tools.memory_20250818({
  execute: async action => {
    // `action` contains `command`, `path`, and other fields
    // depending on the command (view, create, str_replace,
    // insert, delete, rename).
    // Implement your storage backend here.
    // Return the result as a string.
  },
});

const agent = new ToolLoopAgent({
  model: 'anthropic/claude-haiku-5.5',
  tools: { memory },
});

const result = await agent.generate({
  prompt: 'Remember that my favorite editor is Neovim',
});
```

The tool receives structured commands (`view`, `create`, `str_replace`, `insert`, `delete`, `rename`), each with a `path` scoped to `/memories`. Your `execute` function maps these to your storage backend (the filesystem, a database, or any other persistence layer).

**When to use this**: you want memory with minimal implementation effort and are already using Anthropic models. The tradeoff is provider lock-in, since this tool only works with Claude.

## Memory Providers

Another approach is to use a provider that has memory built in. These providers wrap an external memory service and expose it through the AI SDK's standard interface. Memory storage, retrieval, and injection happen transparently, and you do not define any tools yourself.

### Letta

[Letta](https://letta.com) provides agents with persistent long-term memory. You create an agent on Letta's platform (cloud or self-hosted), configure its memory there, and use the AI SDK provider to interact with it. Letta's agent runtime handles memory management (core memory, archival memory, recall).

```bash
pnpm add @letta-ai/vercel-ai-sdk-provider
```

```ts
import { lettaCloud } from '@letta-ai/vercel-ai-sdk-provider';
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: lettaCloud(),
  providerOptions: {
    letta: {
      agent: { id: 'your-agent-id' },
    },
  },
});

const result = await agent.generate({
  prompt: 'Remember that my favorite editor is Neovim',
});
```

You can also use Letta's built-in memory tools alongside custom tools:

```ts
import { lettaCloud } from '@letta-ai/vercel-ai-sdk-provider';
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: lettaCloud(),
  tools: {
    core_memory_append: lettaCloud.tool('core_memory_append'),
    memory_insert: lettaCloud.tool('memory_insert'),
    memory_replace: lettaCloud.tool('memory_replace'),
  },
  providerOptions: {
    letta: {
      agent: { id: 'your-agent-id' },
    },
  },
});

const stream = agent.stream({
  prompt: 'What do you remember about me?',
});
```

See the [Letta provider documentation](/providers/community-providers/letta) for full setup and configuration.

### Mem0

[Mem0](https://mem0.ai) adds a memory layer on top of any supported LLM provider. It automatically extracts memories from conversations, stores them, and retrieves relevant ones for future prompts.

```bash
pnpm add @mem0/vercel-ai-provider
```

```ts
import { createMem0 } from '@mem0/vercel-ai-provider';
import { ToolLoopAgent } from 'ai';

const mem0 = createMem0({
  provider: 'openai',
  mem0ApiKey: process.env.MEM0_API_KEY,
  apiKey: process.env.OPENAI_API_KEY,
});

const agent = new ToolLoopAgent({
  model: mem0('gpt-6-astra', { user_id: 'user-123' }),
});

const { text } = await agent.generate({
  prompt: 'Remember that my favorite editor is Neovim',
});
```

Mem0 works across multiple LLM providers (OpenAI, Anthropic, Google, Groq, Cohere). You can also manage memories explicitly:

```ts
import { addMemories, retrieveMemories } from '@mem0/vercel-ai-provider';

await addMemories(messages, { user_id: 'user-123' });
const context = await retrieveMemories(prompt, { user_id: 'user-123' });
```

See the [Mem0 provider documentation](/providers/community-providers/mem0) for full setup and configuration.

### Supermemory

[Supermemory](https://supermemory.ai) is a long-term memory platform that adds persistent, self-growing memory to your AI applications. It provides tools that handle saving and retrieving memories automatically through semantic search.

```bash
pnpm add @supermemory/tools
```

```ts
import { supermemoryTools } from '@supermemory/tools/ai-sdk';
import { ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: supermemoryTools(process.env.SUPERMEMORY_API_KEY!),
});

const result = await agent.generate({
  prompt: 'Remember that my favorite editor is Neovim',
});
```

Supermemory works with any AI SDK provider. The tools give the model `addMemory` and `searchMemories` operations that handle storage and retrieval.

See the [Supermemory provider documentation](/providers/community-providers/supermemory) for full setup and configuration.

### Hindsight

[Hindsight](/providers/community-providers/hindsight) provides agents with persistent memory through five tools: `retain`, `recall`, `reflect`, `getMentalModel`, and `getDocument`. It can be self-hosted with Docker or used as a cloud service.

```bash
pnpm add @vectorize-io/hindsight-ai-sdk @vectorize-io/hindsight-client
```

```ts
import { HindsightClient } from '@vectorize-io/hindsight-client';
import { createHindsightTools } from '@vectorize-io/hindsight-ai-sdk';
import { ToolLoopAgent } from 'ai';
import { openai } from '@ai-sdk/openai';

const client = new HindsightClient({ baseUrl: process.env.HINDSIGHT_API_URL });

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: createHindsightTools({ client, bankId: 'user-123' }),
  instructions: 'You are a helpful assistant with long-term memory.',
});

const result = await agent.generate({
  prompt: 'Remember that my favorite editor is Neovim',
});
```

The `bankId` identifies the memory store and is typically a user ID. In multi-user apps, call `createHindsightTools` inside your request handler so each request gets the right bank. Hindsight works with any AI SDK provider.

See the [Hindsight provider documentation](/providers/community-providers/hindsight) for full setup and configuration.

### MongoDB

[`@mongodb-developer/vercel-ai-memory`](https://www.npmjs.com/package/@mongodb-developer/vercel-ai-memory) provides MongoDB Atlas-backed persistent memory with five structured tiers: **Session**, **Semantic**, **Procedural**, **Episodic**, and **Scratchpad**. Retrieval is powered by Atlas Vector Search using any AI SDK embedding model, with automatic index creation and per-type retention policies.

```bash
pnpm add @mongodb-developer/vercel-ai-memory
```

This integration targets AI SDK v6 because it relies on v6 APIs such as `ModelMessage`, `ToolLoopAgent`, and `isLoopFinished()`:

```json
{
  "peerDependencies": {
    "ai": "^6.0.0",
    "mongodb": "^6.0.0",
    "zod": "^3.0.0"
  }
}
```

```ts
import { createMongoDBMemory } from '@mongodb-developer/vercel-ai-memory';
import { openai } from '@ai-sdk/openai';
import { ToolLoopAgent, isLoopFinished } from 'ai';

// Create the memory instance once at module/server level
const mongodbMemory = createMongoDBMemory({
  uri: process.env.MONGODB_URI!,
  embedder: openai.embedding('text-embedding-3-small'),
});

// Scope to a user and session per request
const agent = new ToolLoopAgent({
  model: openai('gpt-6-astra'),
  tools: mongodbMemory({ userId: 'alice', sessionId: 'sess-001' }),
  stopWhen: isLoopFinished(),
});

const result = await agent.generate({
  prompt: 'My name is Alice and I love hiking. Remember that.',
});
```

`isLoopFinished()` lets the agent keep running until the tool loop naturally finishes, which is useful when memory tools need to read and write before the final response.

Session memory supports two modes: **tool-driven** (the LLM decides when to read/write — good for prototypes) and **hook-driven** (the runtime persists every turn via `prepareCall` + `onEnd` hooks — recommended for production). The other memory tiers (semantic, procedural, episodic, scratchpad) are always LLM-controlled and selective by design.

MongoDB memory works with any AI SDK model and embedding provider, with no vendor lock-in beyond MongoDB Atlas.

**When to use memory providers**: these providers are a good fit when you want memory without building any storage infrastructure. The tradeoff is that the provider controls memory behavior, so you have less visibility into what gets stored and how it is retrieved. You also take on a dependency on an external service.

## Custom Tool

Building your own memory tool from scratch is the most flexible approach. You control the storage format, the interface, and the retrieval logic. This requires the most upfront work but gives you full ownership of how memory works, with no provider lock-in and no external dependencies.

There are two common patterns:

- **Structured actions**: you define explicit operations (`view`, `create`, `update`, `search`) and handle structured input yourself. Safe by design since you control every operation.
- **Bash-backed**: you give the model a sandboxed bash environment to compose shell commands (`cat`, `grep`, `sed`, `echo`) for flexible memory access. More powerful but requires command validation for safety.

For a full walkthrough of implementing a custom memory tool with a bash-backed interface, AST-based command validation, and filesystem persistence, see the **[Build a Custom Memory Tool](/cookbook/guides/custom-memory-tool)** recipe.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Overview
description: Learn how to build agents with the AI SDK.
url: "https://ai-sdk.dev/docs/agents/overview"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Agents are **large language models (LLMs)** that use **tools** in a **loop** to accomplish tasks.

These components work together:

- **LLMs** process input and decide the next action
- **Tools** extend capabilities beyond text generation (reading files, calling APIs, writing to databases)
- **Loop** orchestrates execution through:
  - **Context management** - Maintaining conversation history and deciding what the model sees (input) at each step
  - **Stopping conditions** - Determining when the loop (task) is complete

## Agent State and Context

Agents often need server-side state that should not be placed directly in the
prompt, such as tenant settings, request IDs, feature flags, credentials, or
progress through a task.

Use `runtimeContext` as the agent's shared runtime state. It flows through the
agent loop, is available in `prepareStep` and lifecycle callbacks, and can be
updated between steps. Use `toolsContext` for per-tool values such as API keys
or scoped permissions; each tool receives only its own typed `context` based on
its `contextSchema`.

Learn more in [Runtime and Tool
Context](/docs/ai-sdk-core/runtime-and-tool-context).

## ToolLoopAgent Class

The ToolLoopAgent class handles these three components. Here's an agent that uses multiple tools in a loop to accomplish a task:

```ts
import { ToolLoopAgent, tool } from 'ai';
import { z } from 'zod';

const weatherAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    weather: tool({
      description: 'Get the weather in a location (in Fahrenheit)',
      inputSchema: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
    convertFahrenheitToCelsius: tool({
      description: 'Convert temperature from Fahrenheit to Celsius',
      inputSchema: z.object({
        temperature: z.number().describe('Temperature in Fahrenheit'),
      }),
      execute: async ({ temperature }) => {
        const celsius = Math.round((temperature - 32) * (5 / 9));
        return { celsius };
      },
    }),
  },
});

const result = await weatherAgent.generate({
  prompt: 'What is the weather in San Francisco in celsius?',
});

console.log(result.text); // agent's final answer
console.log(result.steps); // steps taken by the agent
```

The agent automatically:

1. Calls the `weather` tool to get the temperature in Fahrenheit
2. Calls `convertFahrenheitToCelsius` to convert it
3. Generates a final text response with the result

The ToolLoopAgent handles the loop, context management, and stopping conditions.

## Why Use the ToolLoopAgent?

The ToolLoopAgent is the recommended approach for building agents with the AI SDK because it:

- **Reduces boilerplate** - Manages loops and message arrays
- **Improves reusability** - Define once, use throughout your application
- **Simplifies maintenance** - Single place to update agent configuration

For most use cases, start with the ToolLoopAgent. Use core functions (`generateText`, `streamText`) when you need explicit control over each step for complex structured workflows.

## HarnessAgent Class

Use `HarnessAgent` when you want to run a preconfigured established harness,
such as Claude Code, Codex, or Pi, instead of building the loop yourself around
a language model. Harnesses are a separate abstraction from providers and
models, but stream into AI SDK-compatible result and UI primitives.

Learn more in [Harnesses](/docs/ai-sdk-harnesses) and
[HarnessAgent](/docs/ai-sdk-harnesses/harness-agent).

The rest of this section focuses on `ToolLoopAgent`, which is the AI SDK agent
class for building your own model-and-tools loop.

## Terminal UI

Use `@ai-sdk/tui` to run a `ToolLoopAgent` in an interactive terminal UI. It is
useful for local agent development, demos, and internal tools where you want
prompt input, streamed responses, tool cards, reasoning sections, scrolling, and
tool approval prompts without building a custom interface.

```ts
import { runAgentTUI } from '@ai-sdk/tui';

await runAgentTUI({
  title: 'Weather Agent',
  agent: weatherAgent,
});
```

Learn more in [Terminal UI](/docs/agents/terminal-ui).

## Structured Workflows

Agents are flexible and powerful, but non-deterministic. When you need reliable, repeatable outcomes with explicit control flow, use core functions with structured workflow patterns combining:

- Conditional statements for explicit branching
- Standard functions for reusable logic
- Error handling for robustness
- Explicit control flow for predictability

[Explore workflow patterns](/docs/agents/workflows) to learn more about building structured, reliable systems.

## Next Steps

- **[Building Agents](/docs/agents/building-agents)** - Guide to creating agents with the ToolLoopAgent
- **[Runtime and Tool Context](/docs/ai-sdk-core/runtime-and-tool-context)** - How to pass shared agent state and per-tool context
- **[Workflow Patterns](/docs/agents/workflows)** - Structured patterns using core functions for complex workflows
- **[Loop Control](/docs/agents/loop-control)** - Execution control with stopWhen and prepareStep

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Policy-Based Tool Approvals
description: "Author tool authorization rules as code with @ai-sdk/policy-opa and Open Policy Agent."
url: "https://ai-sdk.dev/docs/agents/policy-tool-approvals"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

[Tool Approvals](/docs/agents/tool-approvals) let you approve or deny tool calls with a function in your agent setup. `@ai-sdk/policy-opa` moves those rules out of code and into [Open Policy Agent](https://www.openpolicyagent.org/) (OPA) policies written in `.rego`.

Use it when you want authorization to be:

- Written as a separate, reviewable artifact instead of a function buried in agent setup.
- Testable in CI with `opa test`, independent of the SDK.
- Editable without a code deploy (when served from a running OPA instance).

The package sits entirely on top of the public `toolApproval` callback. Nothing changes on the wire: the same `tool-approval-request` / `tool-approval-response` flow as built-in approvals.

## What you can enforce

OPA is a general policy engine. If you can express a rule as code over structured input, you can enforce it at the tool boundary. Common categories:

- Security and access: scopes, roles, permissions, tenant isolation, allowlists of tools, hosts, or paths.
- Business rules: who can do what, on which resources, under which conditions (for example, payments above a threshold need approval).
- Cost and usage: deny or require approval for calls that exceed a budget, token ceiling, or per-user quota.
- Compliance and change control: gate destructive or regulated actions on the right approver, environment, or time window.

The decision is not limited to the current call's arguments. The policy `input` also carries `messages`, the full model and tool-call history for the run, so a rule can factor in what already happened: prior tool calls, the sequence of actions so far, or a running total across the conversation. Examples:

- Require approval once a run has already performed N writes.
- Deny a second irreversible action (a push, a delete) in the same conversation.
- Track cumulative spend or token usage across steps and stop at a ceiling.

### Best fit: deterministic checks

Policy enforcement is strongest when the decision is deterministic and verifiable from structured fields, whether those fields come from the current call or the history in `messages`. The policy extracts a value and compares it, so the answer is the same every time.

- Strong fits: scopes, permissions, numeric thresholds, allowlists, time windows, and history-aware checks like counts, sequences, and running totals.
- Weak fits: content-based or semantic filtering such as "do not use bad language" or "block toxic content". These are best-effort, hard to verify, and easy to bypass. Use a dedicated moderation or classification step for them, and keep policy for the deterministic gate around it.

Rule of thumb: if you can point at a field in the `input` object and write an exact check on it, this is the right tool. If the rule depends on judging free-form meaning, do not lean on policy alone.

## Install

```sh
pnpm add @ai-sdk/policy-opa
# pick one (or both) OPA backends:
pnpm add @open-policy-agent/opa-wasm   # in-process WASM evaluation
pnpm add @open-policy-agent/opa        # HTTP client to a running OPA server
```

- The backends are optional peer dependencies.
- Only the backend you import is loaded.

## How it works

The policy is consulted before every tool dispatch and maps to one of the standard approval statuses:

- `allow` runs the tool.
- `deny` returns a denied result the model can reason about (no human needed).
- `requires-approval` pauses the run and waits for a human `tool-approval-response`.
- No matching rule normalizes to `not-applicable`, which the SDK treats as allow. Add `default decision := { "decision": "deny" }` to your policy to default-deny instead.

## Quick start

```ts
import { generateText } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';
import { opaPolicy, wasmPolicyClient } from '@ai-sdk/policy-opa';
import { readFile } from 'node:fs/promises';

// 1. Load the compiled policy bundle.
const wasm = await readFile('./policy.wasm');
const client = await wasmPolicyClient({ wasm });

// 2. Build the toolApproval configuration.
const toolApproval = opaPolicy({
  client,
  path: 'agent/call/decision',
});

// 3. Pass it to generateText (or streamText / ToolLoopAgent). Everything else is normal.
const result = await generateText({
  model: anthropic('claude-sonnet-5-5'),
  tools: { git, bash, queryLogs },
  toolApproval,
  prompt: 'find the failing test and push the fix',
});
```

## Writing the Rego policy

The policy emits a decision object. `reason` is optional and is surfaced back to the model (for `deny`) or to the human approver (for `requires-approval`). A manual approval request exposes it as `reason` in core results and as `approval.requestReason` in UI messages.

```txt
package agent.call

# Default to "not-applicable" so unmatched calls fall through.
# Use { decision: "deny" } to default-deny instead.
default decision := { "decision": "not-applicable" }

# Hard deny: pushes are never allowed automatically.
decision := { "decision": "deny", "reason": "pushes require human review" } {
  input.tool.name == "git"
  input.args.args[0] == "push"
}

# Auto-allow: read-only git operations.
decision := { "decision": "allow" } {
  input.tool.name == "git"
  input.args.args[0] in {"status", "log", "diff", "show"}
}

# Human-in-the-loop: kubectl by oncall.
decision := { "decision": "requires-approval", "reason": "kubectl by oncall" } {
  input.tool.name == "kubectl"
  input.runtimeContext.role == "sre-oncall"
}
```

Notes:

- The legacy boolean shape (`{ "allow": true | false, "reason": "..." }`) is also accepted, so existing rules migrate without rewriting.
- The default OPA `input` shape is `{ tool: { name }, args, messages, runtimeContext }`.
- Override the input shape with `toInput`:

```ts
opaPolicy({
  client,
  path: 'agent/call/decision',
  toInput: ({ toolCall, runtimeContext }) => ({
    action: toolCall.toolName,
    principal: runtimeContext.role,
    resource: toolCall.input,
  }),
});
```

### Test the policy in CI

OPA ships its own test framework. These tests run without the SDK, which is the main reason policy-as-code beats policy-in-application-code.

```txt
# policy_test.rego
package agent.call

test_push_denied {
  decision.decision == "deny" with input as {
    "tool": { "name": "git" },
    "args": { "args": ["push", "origin", "main"] }
  }
}
```

Run with `opa test policy.rego policy_test.rego`.

### Errors fail closed

- If the backend errors (server unreachable, WASM fault, misbuilt bundle), `opaPolicy` returns `denied` with the error message as the reason.
- If the backend returns a present but unrecognized decision, such as an unknown `decision` value or a non-boolean legacy `allow` value, `opaPolicy` returns `denied`.
- The error never rejects out of the callback and never aborts the run.
- A backend outage blocks the affected call rather than silently allowing it.
- This is distinct from a rule that returns no match: `null` or `undefined` normalizes to `not-applicable` (allow). Use a `default ... deny` rule if you want unmatched calls denied too.

## Loading the policy

### Option A: WASM (in-process)

Compile the `.rego` to WASM ahead of time:

```sh
opa build -t wasm -e 'agent/call/decision' -o bundle.tar.gz policy.rego
tar -xzf bundle.tar.gz /policy.wasm
```

```ts
import { wasmPolicyClient, opaPolicy } from '@ai-sdk/policy-opa';
import { readFile } from 'node:fs/promises';

const wasm = await readFile('./policy.wasm');
const client = await wasmPolicyClient({ wasm });
const toolApproval = opaPolicy({ client, path: 'agent/call/decision' });
```

- No network call per decision.
- Good fit when you ship the policy with the app, or fetch it from object storage at startup.
- Hot-reload means rebuilding the WASM and re-instantiating the client.

### Option B: HTTP (running OPA server)

```ts
import { httpPolicyClient, opaPolicy } from '@ai-sdk/policy-opa';

const client = httpPolicyClient({ url: 'http://localhost:8181' });
const toolApproval = opaPolicy({ client, path: 'agent/call/decision' });
```

- One HTTP round-trip per decision.
- Good fit when policies change frequently and you want hot-reload without redeploying, or when multiple services share one OPA.
- Pass `headers` for Styra DAS / EOPA authentication.

## Bring in external data and integrations

Policies are not limited to the tool-call input. OPA can decide based on external data such as role-to-permission mappings, IDP group memberships, entitlement lists, or allowlists, and that data can update without redeploying your app.

How OPA sources data is OPA's concern, not this package's. The common approaches:

- Static data at load time: pass `data` to `wasmPolicyClient({ wasm, data })`. It is handed to the bundle's `setData`, so the policy can read it under `data.*`.
- Bundles: point a running OPA server (the HTTP backend) at a bundle service so it periodically pulls fresh policy and data. Role mappings and IDP groups update without a code deploy.
- Lookups during evaluation: a policy can call out to an external service (for example an IDP or entitlements API) while it evaluates a decision.

Learn the details from OPA, not here:

- [Policy reference](https://www.openpolicyagent.org/docs/policy-reference)
- [External data](https://www.openpolicyagent.org/docs/external-data) (including connecting to an IDP)

This package only maps the resulting decision to the SDK's approval status. The `input` it passes is what your policy combines with whatever data OPA already has.

## Roll out safely with shadow mode

Do not ship a new policy straight to enforce. The first version almost always denies things you did not mean to. `shadow(approval, opts)` evaluates the policy and reports the decision via `onDecision`, but tells the SDK every call is approved until you flip `enforce: true`.

```ts
import { opaPolicy, shadow, wasmPolicyClient } from '@ai-sdk/policy-opa';

const client = await wasmPolicyClient({ wasm });

const toolApproval = shadow(
  opaPolicy({ client, path: 'agent/call/decision' }),
  {
    enforce: process.env.ENFORCE_POLICY === 'true',
    onDecision: event => {
      logger.info('policy.decision', {
        tool: event.toolCall.toolName,
        decision: event.decision.type,
        reason: event.decision.reason,
        enforced: event.enforced,
        wouldBlock: event.decision.type === 'denied',
      });
    },
  },
);
```

Recommended rollout:

1. Write the policy and test it with `opa test`.
2. Wrap it in `shadow(...)` with `enforce: false` (the default) and wire `onDecision` to your logs/metrics.
3. Run in your real environment. Inspect events where `decision.type` is `denied` or `user-approval`: those are the calls the policy would have changed.
4. Fix the policy and iterate.
5. When the only `denied` / `user-approval` events are ones you want, set `enforce: true`.

Telemetry semantics:

- `onDecision` is fire-and-forget. A slow or throwing logger cannot block or break enforcement, and thrown errors are swallowed.
- The contract is enforcement first, observability second.
- For the opposite (enforcement waits for the audit log), log from inside the underlying `toolApproval` instead of through `shadow`.

## Send decisions to your observability platform

`onDecision` is not only for rollout. Keep `shadow` in place with `enforce: true` and every decision flows to your logging, metrics, or tracing stack while the policy stays load-bearing.

```ts
const toolApproval = shadow(
  opaPolicy({ client, path: 'agent/call/decision' }),
  {
    enforce: true, // enforcing AND observing
    onDecision: event => {
      metrics.increment('agent.policy.decision', {
        tool: event.toolCall.toolName,
        decision: event.decision.type, // approved | denied | user-approval | not-applicable
        enforced: String(event.enforced),
      });
    },
  },
);
```

- Each `PolicyDecisionEvent` carries the tool call, the policy `decision` (type and reason), `enforced`, and `effective` (what the SDK acted on). Compare `decision` against `effective` to spot drift.
- `onDecision` is fire-and-forget; see the telemetry semantics under shadow mode.

When you use the HTTP backend, OPA can also emit [decision logs](https://www.openpolicyagent.org/docs/management-decision-logs) natively, shipping every evaluation to a remote service for a full audit trail without any application code.

## Capability scoping at the model boundary

`opaCapabilityMiddleware` enforces policy earlier than `toolApproval`: before the model is even told a tool exists. This is defense in depth, plus it saves tokens and improves jailbreak rejection.

```ts
import { wrapLanguageModel } from 'ai';
import { wasmPolicyClient, opaCapabilityMiddleware } from '@ai-sdk/policy-opa';

const client = await wasmPolicyClient({ wasm });

const wrappedModel = wrapLanguageModel({
  model: anthropic('claude-sonnet-5-5'),
  middleware: opaCapabilityMiddleware({ client, path: 'agent/tools/allowed' }),
});
```

- The rule at `path` returns a `string[]` of allowed tool names, or `{ tools: string[] }`.
- Tools not in the allowlist are dropped before the model sees them.
- Fails closed: on a malformed response or evaluator error, `params.tools` is set to `undefined` so the model is told it has no tools. For fail-open, write the fallback in Rego.

## Scoping a discovered tool surface

When tools come from MCP discovery or a plugin registry, you cannot write per-tool rules ahead of time, and any tool you forgot is silently allowed. `wrapMcpTools` makes the approval total over the discovered surface by routing unmatched tools to a configurable default.

```ts
import { opaPolicy, wasmPolicyClient, wrapMcpTools } from '@ai-sdk/policy-opa';

const discovered = await mcpClient.tools();
const client = await wasmPolicyClient({ wasm });

const { tools, toolApproval } = wrapMcpTools(
  discovered,
  opaPolicy({ client, path: 'agent/call/decision' }),
  { default: 'user-approval' }, // anything OPA does not match needs a human
);

await generateText({ model, tools, toolApproval, prompt });
```

Defaults for uncovered tools:

- `'user-approval'` (default): require a human. Good when you trust the source but want a safety net.
- `'denied'`: hard allowlist mode. The policy enumerates what is allowed; everything else is rejected.
- `'approved'`: allow. Only when the discovery source is fully trusted.

Despite the name, it works on any `Record<string, Tool>`.

## Allow-all when no policy is configured

`optionalOpaPolicy` returns `undefined` when `client` is `undefined`, which is the same as not passing `toolApproval` (the SDK approves every call). Useful for local dev or CI where the policy file is absent.

```ts
import { optionalOpaPolicy, wasmPolicyClient } from '@ai-sdk/policy-opa';
import { readFile } from 'node:fs/promises';

const wasm = process.env.POLICY_WASM_PATH
  ? await readFile(process.env.POLICY_WASM_PATH)
  : undefined;

const client = wasm ? await wasmPolicyClient({ wasm }) : undefined;

const toolApproval = optionalOpaPolicy({ client, path: 'agent/call/decision' });
```

- `POLICY_WASM_PATH` unset: `toolApproval` is `undefined`, all calls allowed, OPA modules never loaded.
- `POLICY_WASM_PATH` set: policy loads, enforcement on.
- For stricter behavior (refuse to start without a policy), use `opaPolicy` directly and let the missing bytes throw at startup.

## Transitive enforcement: composite tools

`toolApproval` only fires when the model calls a tool directly. A coarse dispatcher tool (a `bash` tool that can run `git push`, an HTTP tool, an MCP proxy) lets the model bypass a per-action rule by routing through it.

The fix lives inside the dispatcher's `toolApproval` entry: parse the dispatcher input down to a logical `(name, args)` pair, then evaluate it against the same rule the direct tool uses.

```ts
const bashApproval = opaPolicy({
  client,
  path: 'agent/action/decision',
  toInput: ({ toolCall }) => {
    const { command } = toolCall.input as { command: string };
    const [bin, ...rest] = command.split(/\s+/);
    return { kind: bin, args: rest };
  },
});
```

Guidance:

- Keep the matching logic in one place: either a shared Rego helper rule (both approvals call `opaPolicy` with the same `path`) or a shared TypeScript predicate.
- Deny anything you cannot reduce to a clean invocation. Shell input is adversarial to parse, so "cannot prove it is safe" means deny.
- Honest limitation: this gates dispatch at the model's call boundary. It does not stop a tool that, once approved, performs extra side effects beyond what its input describes. For that, run untrusted execution in an out-of-band sandbox (Vercel Sandbox, Firecracker, containers) and treat the sandbox as the trust boundary.

Worked examples for SQL, HTTP, MCP, browser, and shell dispatchers live in the [package README](https://github.com/vercel/ai/tree/main/packages/policy-opa).

## Example application

The [ai-sdk-slackbot](https://github.com/vercel-labs/ai-sdk-slackbot) example wires this package into a real agent end to end. It shows:

- Every tool call gated by Rego in `policies/decision.rego`, editable without touching TypeScript.
- Real tools under policy: a web search scoped to a domain, a weather tool with a city allowlist, and a `bash` tool restricted to a read-only command allowlist (transitive enforcement).
- Switching backends with one env var (`POLICY_MODE`): in-process WASM by default, or a live OPA HTTP server for hot-reloading policies in dev.
- Policy code tested independently with OPA-native tests in `policies/decision_test.rego`.

See [`policies/`](https://github.com/vercel-labs/ai-sdk-slackbot/tree/main/policies) and [`lib/policy/load.ts`](https://github.com/vercel-labs/ai-sdk-slackbot/blob/main/lib/policy/load.ts) for how the policy is loaded and applied.

## API reference

Everything is exported from the package root, `@ai-sdk/policy-opa`.

Engine-neutral core:

- `shadow(approval, opts?)`: wrap any approval so it is evaluated and reported but not enforced until `opts.enforce: true`. Recommended starting point for any new policy.
- `wrapMcpTools(tools, approval, opts?)`: make an approval total over a discovered tool set. `opts.default` controls uncovered tools.
- `PolicyClient`: the `evaluate(path, input)` interface every backend implements. The seam for non-OPA engines.
- Helper types: `PolicyDecision`, `WrappedMcpTools`, `PolicyDecisionEvent`.

OPA backend and adapters:

- `wasmPolicyClient({ wasm, data? })`: async; loads a compiled OPA WASM bundle in-process.
- `httpPolicyClient({ url, headers? })`: sync; client against a running OPA server.
- `opaPolicy({ client, path, toInput? })`: returns a `toolApproval` configuration. Fails closed.
- `optionalOpaPolicy({ client, path, toInput? })`: like `opaPolicy` but returns `undefined` when `client` is `undefined`.
- `opaCapabilityMiddleware({ client, path, toInput? })`: a `LanguageModelV4Middleware` that narrows `params.tools` to an allowlist. Fails closed.
- `normalizeOpaDecision(result)`: standalone result normalization, for users who call OPA themselves.

## Related

- [Tool Approvals](/docs/agents/tool-approvals): the underlying `toolApproval` callback this package plugs into.
- [Building Agents](/docs/agents/building-agents)
- [ai-sdk-slackbot example](https://github.com/vercel-labs/ai-sdk-slackbot): a full agent with policy-gated tools.
- [Open Policy Agent documentation](https://www.openpolicyagent.org/docs/)

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Subagents
description: Delegate context-heavy tasks to specialized subagents while keeping the main agent focused.
url: "https://ai-sdk.dev/docs/agents/subagents"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

A subagent is an agent that a parent agent can invoke. The parent delegates work via a tool, and the subagent executes autonomously before returning a result.

## How It Works

1. **Define a subagent** with its own model, instructions, and tools
2. **Create a tool that calls it** for the main agent to use
3. **Subagent runs independently with its own context window**
4. **Return a result** (optionally streaming progress to the UI)
5. **Control what the model sees** using `toModelOutput` to summarize

## When to Use Subagents

Subagents add latency and complexity. Use them when the benefits outweigh the costs:

| Use Subagents When                              | Avoid Subagents When           |
| ----------------------------------------------- | ------------------------------ |
| Tasks require exploring large amounts of tokens | Tasks are simple and focused   |
| You need to parallelize independent research    | Sequential processing suffices |
| Context would grow beyond model limits          | Context stays manageable       |
| You want to isolate tool access by capability   | All tools can safely coexist   |

## Why Use Subagents?

### Offloading Context-Heavy Tasks

Some tasks require exploring large amounts of information—reading files, searching codebases, or researching topics. Running these in the main agent consumes context quickly, making the agent less coherent over time.

With subagents, you can:

- Spin up a dedicated agent that uses hundreds of thousands of tokens
- Have it return only a focused summary (perhaps 1,000 tokens)
- Keep your main agent's context clean and coherent

The subagent does the heavy lifting while the main agent stays focused on orchestration.

### Parallelizing Independent Work

For tasks like exploring a codebase, you can spawn multiple subagents to research different areas simultaneously. Each returns a summary, and the main agent synthesizes the findings—without paying the context cost of all that exploration.

### Specialized Orchestration

A less common but valid pattern is using a main agent purely for orchestration, delegating to specialized subagents for different types of work. For example:

- An exploration subagent with read-only tools for researching codebases
- A coding subagent with file editing tools
- An integration subagent with tools for a specific platform or API

This creates a clear separation of concerns, though context offloading and parallelization are the more common motivations for subagents.

## Basic Subagent Without Streaming

The simplest subagent pattern requires no special machinery. Your main agent has a tool that calls another agent in its `execute` function:

```ts
import { ToolLoopAgent, tool } from 'ai';
import { z } from 'zod';

// Define a subagent for research tasks
const researchSubagent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: `You are a research agent.
Summarize your findings in your final response.`,
  tools: {
    read: readFileTool, // defined elsewhere
    search: searchTool, // defined elsewhere
  },
});

// Create a tool that delegates to the subagent
const researchTool = tool({
  description: 'Research a topic or question in depth.',
  inputSchema: z.object({
    task: z.string().describe('The research task to complete'),
  }),
  execute: async ({ task }, { abortSignal }) => {
    const result = await researchSubagent.generate({
      prompt: task,
      abortSignal,
    });
    return result.text;
  },
});

// Main agent uses the research tool
const mainAgent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: 'You are a helpful assistant that can delegate research tasks.',
  tools: {
    research: researchTool,
  },
});
```

This works well when you don't need to show the subagent's progress in the UI. The tool call blocks until the subagent completes, then returns the final text response.

### Handling Cancellation

When the user cancels a request, the `abortSignal` propagates to the subagent. Always pass it through to ensure cleanup:

```ts
execute: async ({ task }, { abortSignal }) => {
  const result = await researchSubagent.generate({
    prompt: task,
    abortSignal, // Cancels subagent if main request is aborted
  });
  return result.text;
},
```

If you abort the signal, the subagent stops executing and throws an `AbortError`. The main agent's tool execution fails, which stops the main loop.

To avoid errors about incomplete tool calls in subsequent messages, use `convertToModelMessages` with `ignoreIncompleteToolCalls`:

```ts
import { convertToModelMessages } from 'ai';

const modelMessages = await convertToModelMessages(messages, {
  ignoreIncompleteToolCalls: true,
});
```

This filters out tool calls that don't have corresponding results. Learn more in the [convertToModelMessages](/docs/reference/ai-sdk-ui/convert-to-model-messages) reference.

## Streaming Subagent Progress

When you want to show incremental progress as the subagent works, use [**preliminary tool results**](/docs/ai-sdk-core/tools-and-tool-calling#preliminary-tool-results). This pattern uses a generator function that yields partial updates to the UI.

### How Preliminary Tool Results Work

Change your `execute` function from a regular function to an async generator (`async function*`). Each `yield` sends a preliminary result to the frontend:

```ts
execute: async function* ({ /* input */ }) {
  // ... do work ...
  yield partialResult;
  // ... do more work ...
  yield updatedResult;
}
```

### Building the Complete Message

Each `yield` **replaces** the previous output entirely (it does not append). This means you need a way to accumulate the subagent's response into a complete message that grows over time.

The `readUIMessageStream` utility handles this. It reads each chunk from the stream and builds an ever-growing `UIMessage` containing all parts received so far:

```ts
import { readUIMessageStream, toUIMessageStream, tool } from 'ai';
import { z } from 'zod';

const researchTool = tool({
  description: 'Research a topic or question in depth.',
  inputSchema: z.object({
    task: z.string().describe('The research task to complete'),
  }),
  execute: async function* ({ task }, { abortSignal }) {
    // Start the subagent with streaming
    const result = await researchSubagent.stream({
      prompt: task,
      abortSignal,
    });

    // Each iteration yields a complete, accumulated UIMessage
    for await (const message of readUIMessageStream({
      stream: toUIMessageStream({ stream: result.stream }),
    })) {
      yield message;
    }
  },
});
```

Each yielded `message` is a complete `UIMessage` containing all the subagent's parts up to that point (text, tool calls, and tool results). The frontend simply replaces its display with each new message.

## Controlling What the Model Sees

Here's where subagents become powerful for context management. The full `UIMessage` with all the subagent's work is stored in the message history and displayed in the UI. But you can control what the main agent's model actually sees using `toModelOutput`.

### How It Works

The `toModelOutput` function maps the tool's output to the tokens sent to the model:

```ts
const researchTool = tool({
  description: 'Research a topic or question in depth.',
  inputSchema: z.object({
    task: z.string().describe('The research task to complete'),
  }),
  execute: async function* ({ task }, { abortSignal }) {
    const result = await researchSubagent.stream({
      prompt: task,
      abortSignal,
    });

    for await (const message of readUIMessageStream({
      stream: toUIMessageStream({ stream: result.stream }),
    })) {
      yield message;
    }
  },
  toModelOutput: ({ output: message }) => {
    // Extract just the final text as a summary
    const lastTextPart = message?.parts.findLast(p => p.type === 'text');
    return {
      type: 'text',
      value: lastTextPart?.text ?? 'Task completed.',
    };
  },
});
```

With this setup:

- **Users see**: The full subagent execution—every tool call, every intermediate step
- **The model sees**: Just the final summary text

The subagent might use 100,000 tokens exploring and reasoning, but the main agent only consumes the summary. This keeps the main agent coherent and focused.

### Write Subagent Instructions for Summarization

For `toModelOutput` to extract a useful summary, your subagent must produce one. Add explicit instructions like this:

```ts
const researchSubagent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: `You are a research agent. Complete the task autonomously.

IMPORTANT: When you have finished, write a clear summary of your findings as your final response.
This summary will be returned to the main agent, so include all relevant information.`,
  tools: {
    read: readFileTool,
    search: searchTool,
  },
});
```

Without this instruction, the subagent might not produce a comprehensive summary. It could simply say "Done", leaving `toModelOutput` with nothing useful to extract.

## Rendering Subagents in the UI (with useChat)

To display streaming progress, check the tool part's `state` and `preliminary` flag.

### Tool Part States

| State              | Description                                |
| ------------------ | ------------------------------------------ |
| `input-streaming`  | Tool input being generated                 |
| `input-available`  | Tool ready to execute                      |
| `output-available` | Tool produced output (check `preliminary`) |
| `output-error`     | Tool execution failed                      |

### Detecting Streaming vs Complete

```tsx
const hasOutput = part.state === 'output-available';
const isStreaming = hasOutput && part.preliminary === true;
const isComplete = hasOutput && !part.preliminary;
```

### Type Safety for Subagent Output

Export types alongside your agents for use in UI components:

```ts title="lib/agents.ts"
import { ToolLoopAgent, InferAgentUIMessage } from 'ai';

export const mainAgent = new ToolLoopAgent({
  // ... configuration with researchTool
});

// Export the main agent message type for the chat UI
export type MainAgentMessage = InferAgentUIMessage<typeof mainAgent>;
```

### Render Messages and Subagent Output

This example uses the types defined above to render both the main agent's messages and the subagent's streamed output:

```tsx
'use client';

import { useChat } from '@ai-sdk/react';
import type { MainAgentMessage } from '@/lib/agents';

export function Chat() {
  const { messages } = useChat<MainAgentMessage>();

  return (
    <div>
      {messages.map(message =>
        message.parts.map((part, i) => {
          switch (part.type) {
            case 'text':
              return <p key={i}>{part.text}</p>;
            case 'tool-research':
              return (
                <div>
                  {part.state !== 'input-streaming' && (
                    <div>Research: {part.input.task}</div>
                  )}
                  {part.state === 'output-available' && (
                    <div>
                      {part.output.parts.map((nestedPart, i) => {
                        switch (nestedPart.type) {
                          case 'text':
                            return <p key={i}>{nestedPart.text}</p>;
                          default:
                            return null;
                        }
                      })}
                    </div>
                  )}
                </div>
              );
            default:
              return null;
          }
        }),
      )}
    </div>
  );
}
```

## Caveats

### No Tool Approvals in Subagents

Subagent tools cannot use approval flows such as `toolApproval` (or the
deprecated `needsApproval`). All tools must execute automatically without user
confirmation.

### Subagent Context is Isolated

Each subagent invocation starts with a fresh context window. This is one of the key benefits of subagents: they don't inherit the accumulated context from the main agent, which is exactly what allows them to do heavy exploration without bloating the main conversation.

If you need to give a subagent access to the conversation history, the `messages` are available in the tool's execute function alongside `abortSignal`:

```ts
execute: async ({ task }, { abortSignal, messages }) => {
  const result = await researchSubagent.generate({
    messages: [
      ...messages, // The main agent's conversation history
      { role: 'user', content: task }, // The specific task for this invocation
    ],
    abortSignal,
  });
  return result.text;
},
```

Use this sparingly since passing full history defeats some of the context isolation benefits.

### Streaming Adds Complexity

The basic pattern (no streaming) is simpler to implement and debug. Only add streaming when you need to show real-time progress in the UI.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Terminal UI
description: Run AI SDK agents in an interactive terminal UI.
url: "https://ai-sdk.dev/docs/agents/terminal-ui"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

The `@ai-sdk/tui` package lets you run a local `ToolLoopAgent` or connect to a
remote agent through a `ChatTransport` in an interactive terminal interface.
It is useful for local development, demos, and internal tools where a terminal
experience is enough and you do not want to build a custom UI.

The terminal UI handles prompt input, streamed assistant responses, markdown
rendering, tool cards, reasoning sections, scrolling, and tool approval prompts.

## Installation

Install `@ai-sdk/tui` alongside `ai` and the provider package you use:

```
pnpm add @ai-sdk/tui ai @ai-sdk/openai
```

## Running an Agent

Create a `ToolLoopAgent` and pass it to `runAgentTUI`:

```ts
import { openai } from '@ai-sdk/openai';
import { runAgentTUI } from '@ai-sdk/tui';
import { ToolLoopAgent, tool } from 'ai';
import { z } from 'zod';

const agent = new ToolLoopAgent({
  model: openai('gpt-6-astra'),
  instructions:
    'You are a helpful terminal assistant. Answer in markdown and use tools when they help.',
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      inputSchema: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72,
      }),
    }),
  },
});

await runAgentTUI({
  title: 'Weather Agent',
  agent,
});
```

`runAgentTUI` runs until the user exits with `Esc` or `Ctrl+C`.

## Connecting to a Remote Agent

Pass a `ChatTransport` instead of an `agent` to connect the terminal UI to a
remote AI SDK UI message endpoint:

```ts
import { runAgentTUI } from '@ai-sdk/tui';
import { DefaultChatTransport } from 'ai';

await runAgentTUI({
  title: 'Remote Agent',
  transport: new DefaultChatTransport({
    api: 'https://example.com/api/chat',
  }),
});
```

The transport controls the endpoint, authentication, request body, and other
remote communication behavior. The terminal UI keeps its internal chat id and
message history private to the transport contract.

## Sandbox

Pass a sandbox session with the `sandbox` option when your agent tools need an
execution environment:

```ts
import { createJustBashNetworkSandboxSession } from '@ai-sdk/sandbox-just-bash';

const sandboxSession = await createJustBashNetworkSandboxSession({
  cwd: '/home/user',
});

try {
  await runAgentTUI({
    title: 'Sandbox Agent',
    agent,
    sandbox: sandboxSession.restricted(),
  });
} finally {
  await sandboxSession.destroy();
}
```

The terminal UI forwards the sandbox to every agent call as
`experimental_sandbox`. Tool description functions and tool `execute` functions
can read it from their options and delegate command or file operations to it.
Add the sandbox description to your agent instructions if the model should know
details such as the working directory, public hostname, or exposed ports.

## Display Options

You can control how tool calls, reasoning, and response statistics are shown:

```ts
await runAgentTUI({
  title: 'Weather Agent',
  agent,
  tools: 'auto-collapsed',
  reasoning: 'collapsed',
  responseStatistics: 'outputTokensPerSecond',
  contextSize: 200_000,
});
```

Settings:

- `tools`: Controls tool call rendering. Use `"full"` to show tool input and
  output, `"collapsed"` to show only tool cards, `"auto-collapsed"` to show the
  latest tool expanded until another visible section appears, or `"hidden"` to
  omit tool calls. Defaults to `"auto-collapsed"`.
- `reasoning`: Controls reasoning rendering. Use `"full"` to show reasoning,
  `"collapsed"` to show only reasoning cards, `"auto-collapsed"` to show the
  latest reasoning expanded until another visible section appears, or `"hidden"`
  to omit reasoning. Defaults to `"auto-collapsed"`.
- `responseStatistics`: Use `"outputTokensPerSecond"` to show output token
  throughput or `"outputTokenCount"` to show output token count. Defaults to
  `"outputTokensPerSecond"`.
- `contextSize`: When provided, the terminal UI shows total token usage as a
  percentage of the model context window.

## Tool Approvals

`runAgentTUI` supports `ToolLoopAgent` tool approval flows. When an agent emits a
manual approval request, the terminal UI prompts the user to approve or deny the
tool call before the agent continues.

```ts
const agent = new ToolLoopAgent({
  model: openai('gpt-6-astra'),
  tools: { weather },
  toolApproval: {
    weather: ({ location }) =>
      location.toLowerCase().includes('san francisco')
        ? 'approved'
        : 'user-approval',
  },
});

await runAgentTUI({ title: 'Weather Agent', agent });
```

## Compatibility

When using the `agent` option, the agent must be runnable directly from terminal
user input. It must not require per-call options and must not use structured
output, because the terminal UI cannot infer those values from a free-form
prompt. Use a `transport` for remote agents that need custom request handling.

Use `agent.generate()` or `agent.stream()` directly for examples or apps that
need fixed prompts, call options, structured output, custom result inspection, or
custom stream processing.

## Controls

- `Enter`: submit prompt
- `y` / `n`: approve or deny tool calls
- `Up` / `Down`: scroll transcript
- `PageUp` / `PageDown`: scroll transcript by a full page
- `Ctrl+L`: repaint
- `Esc` / `Ctrl+C`: exit

## Next Steps

- [Building Agents](/docs/agents/building-agents) for creating `ToolLoopAgent`
  instances
- [Tool Approvals](/docs/agents/tool-approvals) for configuring human review of
  tool calls
- [runAgentTUI API Reference](/docs/reference/ai-sdk-tui/run-agent-tui) for
  detailed parameter documentation

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Tool Approvals
description: Add manual and automatic approval flows to ToolLoopAgent tools.
url: "https://ai-sdk.dev/docs/agents/tool-approvals"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

By default, tools with an `execute` function run automatically when the model calls them. Use `toolApproval` on `ToolLoopAgent` to review, approve, or deny selected tool calls before they execute.

`toolApproval` is useful for tools that can modify data, spend money, execute code, send messages, access private data, or perform any other sensitive action.

`toolApproval` applies to tools executed by the AI SDK. Provider-executed
tools run provider-side and do not use AI SDK tool approvals.

## Statuses

Every approval rule returns one of these statuses, either as a string or as an object with a `type` field:

- `'not-applicable'`: execute the tool normally without approval metadata. This is the default.
- `'approved'`: record an automatic approval, then execute the tool.
- `'denied'`: record an automatic denial and return a denied tool output.
- `'user-approval'`: emit an approval request and wait for an explicit response.

For automatic approvals and denials, use the object form when you want to include a reason:

```ts
toolApproval: {
  deleteFile: {
    type: 'denied',
    reason: 'Deleting files is disabled in this workspace',
  },
}
```

Approval functions can also return `undefined`, which is treated the same as `'not-applicable'`.

## Require Approval for a Tool

Use a per-tool map when each tool has a simple policy.

```ts {13-15}
import { ToolLoopAgent, tool } from 'ai';
import { z } from 'zod';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    runCommand: tool({
      inputSchema: z.object({ command: z.string() }),
      execute: async ({ command }) => runCommand(command),
    }),
  },
  toolApproval: {
    runCommand: 'user-approval',
  },
});
```

When `runCommand` is called, the agent returns a `tool-approval-request` instead of executing the tool.

## Decide Based on Tool Input

Use a per-tool approval function when the decision depends on the parsed tool input. The function receives the typed input plus `toolCallId`, `messages`, `toolContext`, and `runtimeContext`.

```ts {17-24}
import { ToolLoopAgent, tool } from 'ai';
import { z } from 'zod';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    processPayment: tool({
      inputSchema: z.object({
        amount: z.number(),
        recipient: z.string(),
      }),
      execute: async ({ amount, recipient }) =>
        processPayment({ amount, recipient }),
    }),
  },
  toolApproval: {
    processPayment: async ({ amount }, { runtimeContext }) => {
      if (runtimeContext.role !== 'admin') {
        return { type: 'denied', reason: 'Only admins can send payments' };
      }
      return amount > 1000 ? 'user-approval' : undefined;
    },
  },
});
```

In this example, non-admin users are denied automatically, large payments require manual approval, and smaller admin payments execute normally.

## Use One Policy for All Tools

Pass a function directly as `toolApproval` when approval depends on the full tool call, shared state across tools, or the complete tool set. This is called a `GenericToolApprovalFunction`.

```ts {13-16,18-20,22-23}
const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: {
    readFile: tool({
      inputSchema: z.object({ path: z.string() }),
      execute: async ({ path }) => readFile(path),
    }),
    deleteFile: tool({
      inputSchema: z.object({ path: z.string() }),
      execute: async ({ path }) => deleteFile(path),
    }),
  },
  toolApproval: ({ toolCall }) => {
    if (toolCall.dynamic) {
      return 'user-approval';
    }

    if (toolCall.toolName === 'deleteFile') {
      return 'user-approval';
    }

    return undefined;
  },
});
```

The generic function receives:

- `toolCall`: the full tool call, including `toolName`, `toolCallId`, `input`, and whether it is dynamic.
- `tools`: all tools available to the model.
- `toolsContext`: context for all tools.
- `messages`: the messages sent to the model for the step that produced the tool call.
- `runtimeContext`: the call's shared runtime context.

## Configure Approval per Request

Because `toolApproval` is an agent setting, you can also return it from `prepareCall`. This is useful when approval policy depends on call options, tenant policy, or user permissions.

```ts {7-17}
import { ToolLoopAgent, tool } from 'ai';
import { z } from 'zod';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  callOptionsSchema: z.object({
    canRunCommands: z.boolean(),
  }),
  prepareCall: ({ options, ...settings }) => ({
    ...settings,
    toolApproval: {
      runCommand: options.canRunCommands
        ? 'user-approval'
        : { type: 'denied', reason: 'Command access is disabled' },
    },
  }),
  tools: {
    runCommand: tool({
      inputSchema: z.object({ command: z.string() }),
      execute: async ({ command }) => runCommand(command),
    }),
  },
});
```

## Handle Manual Approvals

Manual approval requires two calls:

1. Call `agent.generate()` or `agent.stream()` with `toolApproval`.
2. Read the `tool-approval-request` from the result or UI stream.
3. Ask the user or your approval system for a decision.
4. Add a `tool-approval-response` to the messages.
5. Call the agent again with the updated messages.

```ts {5-6,10-18,21-24,26}
import { type ModelMessage, type ToolApprovalResponse } from 'ai';

const messages: ModelMessage[] = [{ role: 'user', content: 'Delete temp.txt' }];

const result = await agent.generate({ messages });
messages.push(...result.responseMessages);

const approvalResponses: ToolApprovalResponse[] = [];

for (const part of result.content) {
  if (part.type === 'tool-approval-request' && !part.isAutomatic) {
    approvalResponses.push({
      type: 'tool-approval-response',
      approvalId: part.approvalId,
      approved: true,
      reason: 'User confirmed the file can be deleted',
    });
  }
}

messages.push({
  role: 'tool',
  content: approvalResponses,
});

const finalResult = await agent.generate({ messages });
```

If approved, the tool runs on the second call. If denied, the model receives the denial and can respond without the tool result.

When a tool execution is denied, consider adding an instruction such as "When
a tool execution is not approved, do not retry it" to prevent repeated
approval requests for the same action.

## Use with `useChat`

When streaming an agent to a chat UI, approval requests appear as tool parts with `state: 'approval-requested'`. Respond with `addToolApprovalResponse`.

```tsx {7-9,17-40}
'use client';

import { useChat } from '@ai-sdk/react';
import { lastAssistantMessageIsCompleteWithApprovalResponses } from 'ai';

export default function Chat() {
  const { messages, addToolApprovalResponse } = useChat({
    sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithApprovalResponses,
  });

  return messages.map(message =>
    message.parts.map(part => {
      if (part.type !== 'tool-runCommand') {
        return null;
      }

      if (part.state === 'approval-requested' && !part.approval.isAutomatic) {
        return (
          <div key={part.toolCallId}>
            {part.approval.requestReason && (
              <p>{part.approval.requestReason}</p>
            )}
            <button
              onClick={() =>
                addToolApprovalResponse({
                  id: part.approval.id,
                  approved: true,
                })
              }
            >
              Approve
            </button>
            <button
              onClick={() =>
                addToolApprovalResponse({
                  id: part.approval.id,
                  approved: false,
                })
              }
            >
              Deny
            </button>
          </div>
        );
      }
    }),
  );
}
```

Only call `addToolApprovalResponse` for manual approvals. Automatic approvals and denials already include approval state in the stream.
When a manual approval status includes a reason, it is available as
`part.approval.requestReason`. A reason supplied with
`addToolApprovalResponse` is stored separately as `part.approval.reason`.

### Schema transforms and persisted approvals

Approval requests preserve the original schema input in `inputSchemaInput` when
it differs from the input presented for approval. Keep this field when persisting
`responseMessages` or UI messages; the UI message conversion functions preserve it
automatically.

On continuation, the SDK reconstructs the transformed input and checks that it
matches the approved input. It never replaces the approved input with a different
value. Older or projected histories that omit the original input are rejected as
invalid tool input if revalidation fails or would change the approved value.

The original input is included in persisted messages and UI approval streams,
including fields removed by schema transforms. A transform that removes a field
does not redact it from approval metadata.

## Security Considerations

### Trust model

In the standard `useChat` pattern, the server rebuilds the conversation from the messages the client sends each turn. The server does not persist conversation state between requests. This means the message history is client-controlled input.

Tool approvals reconstructed from this history are re-validated before execution: the tool input is checked against the tool's schema, and the approval policy is re-evaluated. However, without additional protection, a client that crafts a valid-looking approval for a schema-conforming input can bypass the human-in-the-loop step.

If your tools perform sensitive operations (modifying data, spending money, calling external APIs, accessing private resources), use `experimental_toolApprovalSecret` to cryptographically bind approvals to the server that issued them.

### Signing approvals with `experimental_toolApprovalSecret`

When you provide a secret, the server HMAC-signs each approval request at issuance and verifies the signature when the approval is replayed. A forged or tampered approval is rejected before the tool executes. Configure the secret on `ToolLoopAgent` (or pass it directly to `generateText` or `streamText`).

```ts {6}
const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  tools: { deleteFile, runQuery },
  toolApproval: { deleteFile: 'user-approval', runQuery: 'user-approval' },
  experimental_toolApprovalSecret: process.env.TOOL_APPROVAL_SECRET,
});

const result = await agent.generate({
  messages,
});
```

The signature binds the approval to the exact tool name, tool call ID, and input arguments. Changing any of these after signing invalidates the approval.

**Setting up the secret:**

1. Generate a high-entropy random string (at least 32 bytes):
   ```bash
   openssl rand -base64 32
   ```
2. Store it as an environment variable accessible to all server instances:
   ```
   TOOL_APPROVAL_SECRET=your-generated-secret-here
   ```
3. Pass it to `ToolLoopAgent`, `generateText`, or `streamText` via `experimental_toolApprovalSecret`.

Every serverless instance that might handle a request needs the same secret, since one instance signs the approval and a different instance may verify it on the next turn.

**Behavior when configured:**

- Approval requests without a valid signature are rejected (fail-closed)
- No secret configured: approvals work as before (backward compatible)
- The secret is never sent to the client or included in the stream

`WorkflowAgent` also supports `experimental_toolApprovalSecret`. It signs in
a workflow step before writing the durable approval request. Pass an
environment variable reference, such as
`{ environmentVariable: 'TOOL_APPROVAL_SECRET' }`, so the raw secret is read
only inside signing and verification steps. Only the signature is persisted
and sent to the client.

## Related APIs

- Use `toolApproval` with `ToolLoopAgent`, `generateText`, and `streamText`.
- Author approval rules as code with [Policy-Based Tool Approvals](/docs/agents/policy-tool-approvals) (`@ai-sdk/policy-opa`).
- Use `needsApproval` only with [`WorkflowAgent`](/docs/agents/workflow-agent), where approvals suspend and resume durable workflow execution.
- Subagent tools cannot use `toolApproval`; see [Subagents](/docs/agents/subagents#no-tool-approvals-in-subagents).

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: WorkflowAgent
description: "Build durable, resumable agents with the WorkflowAgent from @ai-sdk/workflow."
url: "https://ai-sdk.dev/docs/agents/workflow-agent"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

The `WorkflowAgent` from `@ai-sdk/workflow` is designed for building **durable, resumable agents** that run inside a [workflow](https://vercel.com/docs/workflow). It provides the same agent loop as the [`ToolLoopAgent`](/docs/agents/building-agents), but adds automatic state persistence, tool schema serialization, and built-in tool approval flows that survive workflow step boundaries.

## Why Durable Agents?

A standard `ToolLoopAgent` runs entirely in memory — if the process crashes, all progress is lost. For production agents that make multiple tool calls, this creates problems:

- **Statefulness** — Long-running agent loops need to persist state across process boundaries
- **Resumability** — If a step fails, you want to retry from the last checkpoint, not restart from scratch
- **Human-in-the-loop** — Tools that require user approval need to pause the agent and resume later
- **Observability** — Each tool call runs as a discrete workflow step, visible in dashboards

`WorkflowAgent` solves these by running inside a workflow, where each tool execution is a durable step with automatic retries.

## When to Use WorkflowAgent vs ToolLoopAgent

|                         | ToolLoopAgent             | WorkflowAgent                                    |
| ----------------------- | ------------------------- | ------------------------------------------------ |
| **Package**             | `ai`                      | `@ai-sdk/workflow`                               |
| **Runtime**             | In-memory                 | Workflow                                         |
| **Durability**          | Lost on crash             | Survives restarts                                |
| **Tool retries**        | Manual                    | Automatic (via workflow steps)                   |
| **Human approval**      | Built-in                  | Built-in + survives suspension                   |
| **`generate()` method** | Available                 | Available with durable non-streaming model calls |
| **`stream()` method**   | Available                 | Primary API                                      |
| **Stream output**       | `streamText` return value | `writable` parameter with `ModelCallStreamPart`  |

For simpler use cases that don't need durability, use [`ToolLoopAgent`](/docs/agents/building-agents) from the `ai` package.

## Installation

```bash
npm install @ai-sdk/workflow workflow@5
```

`@ai-sdk/workflow` requires Workflow 5, as well as the `ai` package and `zod` peer dependencies. The `workflow` package provides the Workflow DevKit runtime (`getWritable`, `'use workflow'`, `'use step'`).

## Creating a WorkflowAgent

Define an agent by instantiating the `WorkflowAgent` class with a model, instructions, and tools:

```ts
import { WorkflowAgent } from '@ai-sdk/workflow';
import { tool } from 'ai';
import { z } from 'zod';

const agent = new WorkflowAgent({
  model: 'anthropic/claude-sonnet-5.5',
  instructions: 'You are a helpful assistant.',
  tools: {
    weather: tool({
      description: 'Get weather for a location',
      inputSchema: z.object({
        location: z.string(),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72,
      }),
    }),
  },
});
```

### Model Resolution

The `model` parameter accepts two forms:

```ts
// String — AI Gateway model ID
new WorkflowAgent({ model: 'anthropic/claude-sonnet-5.5' });

// Provider instance
import { openai } from '@ai-sdk/openai';
new WorkflowAgent({ model: openai('gpt-6-astra') });
```

## Using the Agent in a Workflow

`WorkflowAgent` is designed to run inside a workflow function. The key integration points are:

1. Mark your function with `'use workflow'`
2. Pass `getWritable()` to the agent's `stream()` method
3. Start the workflow from your API route

### End-to-End Example

```ts title="workflow/agent-chat.ts"
import { WorkflowAgent, type ModelCallStreamPart } from '@ai-sdk/workflow';
import { convertToModelMessages, tool, type UIMessage } from 'ai';
import { getWritable } from 'workflow';
import { z } from 'zod';

export async function chat(messages: UIMessage[]) {
  'use workflow';

  const modelMessages = await convertToModelMessages(messages);

  const agent = new WorkflowAgent({
    model: 'anthropic/claude-sonnet-5.5',
    instructions: 'You are a flight booking assistant.',
    tools: {
      searchFlights: tool({
        description: 'Search for available flights',
        inputSchema: z.object({
          origin: z.string(),
          destination: z.string(),
          date: z.string(),
        }),
        execute: searchFlightsStep,
      }),
      bookFlight: tool({
        description: 'Book a specific flight',
        inputSchema: z.object({
          flightId: z.string(),
          passengerName: z.string(),
        }),
        execute: bookFlightStep,
      }),
    },
  });

  const result = await agent.stream({
    messages: modelMessages,
    writable: getWritable<ModelCallStreamPart>(),
  });

  return { messages: result.messages };
}
```

```ts title="app/api/chat/route.ts"
import { createModelCallToUIChunkTransform } from '@ai-sdk/workflow';
import { createUIMessageStreamResponse, type UIMessage } from 'ai';
import { start } from 'workflow/api';
import { chat } from '@/workflow/agent-chat';

export async function POST(request: Request) {
  const { messages }: { messages: UIMessage[] } = await request.json();

  const run = await start(chat, [messages]);

  return createUIMessageStreamResponse({
    stream: run.readable.pipeThrough(createModelCallToUIChunkTransform()),
  });
}
```

### Message Conversion

`WorkflowAgent.stream()` expects `ModelMessage[]`, not `UIMessage[]`. When receiving messages from the client (via `useChat`), convert them first:

```ts
import { convertToModelMessages, type UIMessage } from 'ai';

export async function chat(messages: UIMessage[]) {
  'use workflow';

  const modelMessages = await convertToModelMessages(messages);

  const result = await agent.stream({
    messages: modelMessages,
    // ...
  });
}
```

### Writable Streams

Unlike `ToolLoopAgent` where you consume the returned stream, `WorkflowAgent` writes raw `ModelCallStreamPart` chunks to a `writable` stream provided by the workflow runtime via `getWritable()`. At the response boundary, use `createModelCallToUIChunkTransform()` to convert these into `UIMessageChunk` objects for the client:

```ts
import { createModelCallToUIChunkTransform } from '@ai-sdk/workflow';
import { createUIMessageStreamResponse } from 'ai';

// Convert raw model stream parts → UI message chunks
return createUIMessageStreamResponse({
  stream: run.readable.pipeThrough(createModelCallToUIChunkTransform()),
});
```

The transform also forwards `reset-step` events emitted by `WorkflowAgent` on
retries.
Clients remove partial parts from the failed model-call step before processing
the retried output.

## Resumable Streaming with WorkflowChatTransport

Workflow functions can time out or be interrupted by network failures. `WorkflowChatTransport` is a [`ChatTransport`](/docs/ai-sdk-ui/transport) implementation that handles these interruptions automatically — it detects when a stream ends without a `finish` event and reconnects to resume from where it left off.

```tsx title="app/page.tsx"
'use client';

import { useChat } from '@ai-sdk/react';
import { WorkflowChatTransport } from '@ai-sdk/workflow/client';
import { useMemo } from 'react';

export default function Chat() {
  const transport = useMemo(
    () =>
      new WorkflowChatTransport({
        api: '/api/chat',
        maxConsecutiveErrors: 5,
      }),
    [],
  );

  const { messages, sendMessage } = useChat({ transport });

  // ... render chat UI
}
```

The transport requires your POST endpoint to return an `x-workflow-run-id` response header, and a GET endpoint at `{api}/{runId}/stream` for reconnection:

```ts title="app/api/chat/route.ts"
import { createModelCallToUIChunkTransform } from '@ai-sdk/workflow';
import { createUIMessageStreamResponse, type UIMessage } from 'ai';
import { start } from 'workflow/api';
import { chat } from '@/workflow/agent-chat';

export async function POST(request: Request) {
  const { messages }: { messages: UIMessage[] } = await request.json();
  const run = await start(chat, [messages]);

  return createUIMessageStreamResponse({
    stream: run.readable.pipeThrough(createModelCallToUIChunkTransform()),
    headers: {
      'x-workflow-run-id': run.runId,
    },
  });
}
```

```ts title="app/api/chat/[runId]/stream/route.ts"
import { createModelCallToUIChunkTransform } from '@ai-sdk/workflow';
import { createUIMessageStreamResponse } from 'ai';
import type { NextRequest } from 'next/server';
import { getRun } from 'workflow/api';

export async function GET(
  request: NextRequest,
  { params }: { params: Promise<{ runId: string }> },
) {
  const { runId } = await params;
  const startIndex = Number(
    new URL(request.url).searchParams.get('startIndex') ?? '0',
  );
  if (!Number.isSafeInteger(startIndex) || startIndex < 0) {
    return Response.json(
      { error: 'startIndex must be a non-negative safe integer' },
      { status: 400 },
    );
  }

  const run = await getRun(runId);
  const readable = run
    .getReadable({ startIndex: 0 })
    .pipeThrough(
      createModelCallToUIChunkTransform({ uiStartIndex: startIndex }),
    );

  return createUIMessageStreamResponse({
    stream: readable,
    headers: {
      'x-workflow-run-id': runId,
    },
  });
}
```

`WorkflowChatTransport` counts `UIMessageChunk` objects, while the durable
`WorkflowAgent` stream stores raw `ModelCallStreamPart` objects. Replay the raw
stream from index `0` and apply the non-negative UI cursor in
`createModelCallToUIChunkTransform()` as shown above. Negative start indexes
require a durable stream that already stores `UIMessageChunk` objects and
cannot be used with this raw-to-UI conversion.

For the full API reference, see [`WorkflowChatTransport`](/docs/reference/ai-sdk-workflow/workflow-chat-transport).

## Tools as Workflow Steps

Mark tool execute functions with `'use step'` to make them durable workflow steps. This gives each tool call:

- **Automatic retries** — Failed tool calls are retried automatically (default: 3 attempts)
- **Persistence** — Results survive process restarts
- **Observability** — Each tool call appears as a discrete step in the workflow dashboard

```ts
async function searchFlightsStep(input: {
  origin: string;
  destination: string;
  date: string;
}) {
  'use step';
  const response = await fetch(`https://api.flights.example/search?...`);
  return response.json();
}

async function bookFlightStep(input: {
  flightId: string;
  passengerName: string;
}) {
  'use step';
  const response = await fetch('https://api.flights.example/book', {
    method: 'POST',
    body: JSON.stringify(input),
  });
  return response.json();
}
```

Tools without `'use step'` still work but run as regular in-memory functions without durability guarantees.

## Tool Approval

For `WorkflowAgent`, human approval is configured on the tool definition with
`needsApproval`. This is specific to `WorkflowAgent`; for `generateText`,
`streamText`, and `ToolLoopAgent`, use `toolApproval` instead. When a workflow
tool has `needsApproval` set, the agent pauses and emits an approval request to
the writable stream. The workflow suspends until the user approves or denies:

```ts
const agent = new WorkflowAgent({
  model: 'anthropic/claude-sonnet-5.5',
  tools: {
    bookFlight: tool({
      description: 'Book a flight',
      inputSchema: z.object({
        flightId: z.string(),
        passengerName: z.string(),
      }),
      needsApproval: true, // Always require approval
      execute: bookFlightStep,
    }),
    cancelBooking: tool({
      description: 'Cancel a booking',
      inputSchema: z.object({ bookingId: z.string() }),
      // Conditional approval based on input
      needsApproval: async input => {
        return input.bookingId.startsWith('VIP-');
      },
      execute: cancelBookingStep,
    }),
  },
});
```

Because the workflow is durable, the approval request survives process restarts — the user can approve hours later and the agent will resume.

### Signed Tool Approvals

Client-supplied message history can be modified before it is replayed to the
workflow. For tools that perform sensitive operations, configure
`experimental_toolApprovalSecret` to authenticate approval requests:

```ts {4}
const agent = new WorkflowAgent({
  model: 'anthropic/claude-sonnet-5.5',
  experimental_toolApprovalSecret: {
    environmentVariable: 'TOOL_APPROVAL_SECRET',
  },
  tools: {
    bookFlight: tool({
      description: 'Book a flight',
      inputSchema: z.object({ flightId: z.string() }),
      needsApproval: true,
      execute: bookFlightStep,
    }),
  },
});
```

The agent HMAC-signs the approval ID, tool call ID, tool name, and validated
input. When approved message history is replayed, a missing or invalid
signature prevents the tool from executing. The signature is preserved through
the durable model-call stream, `createModelCallToUIChunkTransform()`,
`addToolApprovalResponse()`, and `convertToModelMessages()`.

Use a high-entropy secret of at least 32 bytes and make the same secret
available to every worker that can issue or resume an approval. Keep old keys
available while approvals signed with them are pending; changing the secret
invalidates those pending approvals.

`WorkflowAgent` passes only the environment variable name into signing and
verification steps. Each step reads the secret from its local environment, and
the raw value is never included in step arguments, durable stream parts,
callbacks, or telemetry events. Configure the environment variable on every
worker, and do not put the secret value in `runtimeContext` or `toolsContext`.

You can also provide `experimental_toolApprovalSecret` to `agent.stream()`.
The stream-level value overrides the constructor default.

## Loop Control

Unlike `ToolLoopAgent`, `WorkflowAgent` does not apply a default step limit.
When `stopWhen` is omitted, it continues until the model stops calling tools or
another natural termination condition is met.

A model that repeatedly calls tools can make an unlimited number of model
calls. Configure an explicit stop condition when you need to bound execution
time and cost.

Control how many steps the agent can take:

```ts
import { isStepCount } from 'ai';

const result = await agent.stream({
  messages,
  stopWhen: isStepCount(10), // Stop after 10 LLM calls
});
```

Omitting `stopWhen` already lets `WorkflowAgent` continue until it has finished
calling tools. You can make that intent explicit with `isLoopFinished()`:

```ts
import { isLoopFinished } from 'ai';

const result = await agent.stream({
  messages,
  stopWhen: isLoopFinished(),
});
```

`isLoopFinished()` is equivalent to omitting `stopWhen` for `WorkflowAgent`.
Use it with caution because a model that keeps calling tools can run
indefinitely and incur significant costs. See
[`isLoopFinished()`](/docs/reference/ai-sdk-core/loop-finished).

## Structured Output

Parse agent responses into typed objects using `Output`:

```ts
import { Output } from '@ai-sdk/workflow';
import { z } from 'zod';

const result = await agent.stream({
  messages,
  output: Output.object({
    schema: z.object({
      sentiment: z.enum(['positive', 'neutral', 'negative']),
      summary: z.string(),
    }),
  }),
});

console.log(result.output); // { sentiment: 'positive', summary: '...' }
```

## Configuration Options

`WorkflowAgent` accepts the same generation settings as `ToolLoopAgent` (`temperature`, `maxOutputTokens`, `topP`, etc.) plus workflow-specific options.

### runtimeContext and toolsContext

Pass server-side state through the agent loop without putting it into the prompt. Use these instead of the previous `experimental_context` option.

- `runtimeContext` is shared agent state that flows through `prepareStep`, lifecycle callbacks, and `onEnd`. Treat it as immutable; return a new value from `prepareStep` to update it for the current and subsequent steps.
- `toolsContext` is a per-tool map keyed by tool name. Each tool's `execute` only sees its own validated entry as `context`. Tools that declare a `contextSchema` validate their entry against the schema before execution.

```ts title="workflow/agent-chat.ts"
import { WorkflowAgent } from '@ai-sdk/workflow';
import { tool } from 'ai';
import { z } from 'zod';

const agent = new WorkflowAgent({
  model: 'anthropic/claude-sonnet-4-6',
  tools: {
    weather: tool({
      description: 'Get the weather for a city.',
      inputSchema: z.object({ city: z.string() }),
      contextSchema: z.object({
        defaultUnit: z.enum(['celsius', 'fahrenheit']),
      }),
      execute: async ({ city }, { context }) => ({
        city,
        unit: context.defaultUnit,
      }),
    }),
  },

  // Shared agent state — available in `prepareStep`, lifecycle callbacks, and `onEnd`.
  runtimeContext: {
    tenantId: 'tenant_123',
    requestId: 'req_abc',
    plan: 'enterprise',
  },

  // Per-tool context — each tool sees only its own validated entry.
  toolsContext: {
    weather: { defaultUnit: 'celsius' },
  },

  prepareStep: ({ runtimeContext }) => {
    if (runtimeContext.plan === 'enterprise') {
      return { temperature: 0.2 };
    }
    return {};
  },
});
```

`runtimeContext` and `toolsContext` can also be passed per-call to `stream()`, where they override the constructor-level defaults.

Because `WorkflowAgent` runs inside the Workflow runtime, context values may be persisted and replayed across workflow and step boundaries. Keep `runtimeContext`, `toolsContext`, and any context values returned from `prepareStep` serializable. Use plain data such as strings, numbers, booleans, arrays, plain objects, dates, URLs, maps, sets, and other Workflow-supported structured data. Do not put functions, class instances, symbols, `WeakMap`, `WeakSet`, database clients, or SDK clients in context. Pass identifiers or configuration data instead, and recreate non-serializable resources inside step functions.

This differs from `ToolLoopAgent`, which runs in memory and can carry richer JavaScript values for the lifetime of a single process. With `WorkflowAgent`, treating context as durable data keeps workflow replay and step execution reliable.

### experimental\_sandbox

Pass a sandbox session when tools need an execution environment. The sandbox is
available to tool descriptions and `execute` functions as
`experimental_sandbox`, and to `prepareStep`, where you can override it for the
current step:

```ts
const agent = new WorkflowAgent({
  model: 'anthropic/claude-sonnet-5.5',
  tools: {
    shell: tool({
      description: 'Run a shell command in the sandbox.',
      inputSchema: z.object({ command: z.string() }),
      execute: async ({ command }, { experimental_sandbox }) => {
        if (!experimental_sandbox) {
          throw new Error('Sandbox is not available');
        }

        return experimental_sandbox.run({ command });
      },
    }),
  },
  experimental_sandbox: sandbox,
});

await agent.stream({
  messages,
  writable: getWritable(),
  experimental_sandbox: requestSandbox, // Overrides the constructor default.
});
```

`experimental_sandbox` is a live runtime handle, not durable context. Do not
store it in `runtimeContext` or `toolsContext`. If a tool runs as a separate
workflow step, pass serializable sandbox identifiers or configuration and
reattach inside that step.

### prepareCall

Called once before the agent loop starts. Use it to transform model, instructions, or other settings based on runtime context:

```ts
const agent = new WorkflowAgent({
  model: 'anthropic/claude-sonnet-5.5',
  prepareCall: async ({ model, tools, messages }) => {
    return {
      instructions: `Current time: ${new Date().toISOString()}`,
    };
  },
});
```

### prepareStep

Called before each step (LLM call). Use it to modify settings, manage context, or inject messages dynamically:

```ts
const agent = new WorkflowAgent({
  model: 'anthropic/claude-sonnet-4-6',
  prepareStep: async ({ stepNumber, experimental_sandbox }) => {
    if (stepNumber > 5) {
      return { toolChoice: 'none' }; // Force text response after 5 steps
    }
    if (experimental_sandbox) {
      return { temperature: 0.2 };
    }
    return {};
  },
});
```

Both `prepareCall` and `prepareStep` can also be passed per-call in `stream()`.

## Lifecycle Callbacks

Agents provide lifecycle callbacks for logging, observability, and custom telemetry. All callbacks can be defined in the constructor (agent-wide) or in `stream()` (per-call). When both are provided, both fire (constructor first):

```ts
const agent = new WorkflowAgent({
  model: 'anthropic/claude-sonnet-5.5',

  onStart({ messages }) {
    console.log(`Agent started with ${messages.length} messages`);
  },

  onStepStart({ stepNumber }) {
    console.log(`Step ${stepNumber} starting`);
  },

  onToolExecutionStart({ toolCall }) {
    console.log(`Calling tool: ${toolCall.toolName}`);
  },

  onToolExecutionEnd({ toolCall, success, durationMs }) {
    console.log(`Tool finished: ${toolCall.toolName}`, {
      success,
      durationMs,
    });
  },

  onStepEnd({ usage, finishReason }) {
    console.log('Step done:', { finishReason });
  },

  onEnd({ steps, totalUsage }) {
    console.log(`Completed in ${steps.length} steps`);
  },
});
```

For concrete tool sets, `WorkflowAgentToolExecutionStartEvent` and
`WorkflowAgentToolExecutionEndEvent` preserve the relationship between each
tool name and its input, context, and output types. TypeScript narrows the
nested `toolCall` directly, but use `Extract` or a user-defined type guard when
you need to narrow the correlated `toolContext` or `output` fields by tool name.

Tool input callbacks (`onInputStart`, `onInputDelta`, and
`onInputAvailable`) are also preserved by `WorkflowAgent`. The model call runs
inside a durable step, while callback functions remain in the workflow
context because arbitrary functions cannot cross the step boundary. As a
result, `WorkflowAgent` records the callback events during the model step and
replays them in order immediately after that step completes, before tool
execution and step lifecycle callbacks. They do not run concurrently with
model generation and cannot provide in-flight cancellation or backpressure.
Each callback receives its tool's `toolsContext` entry after
`contextSchema` validation.

For highly fragmented tool inputs, `onInputDelta` replay data is part of the
durable model-step result. Only configure `onInputDelta` when each generated
delta is needed; omit it to avoid retaining delta replay data.

The deprecated `experimental_onStart` and `experimental_onStepStart` names
remain available for backwards compatibility. When both the stable and
experimental name are provided in the same constructor or `stream()` call, the
stable callback is used.

## Type Inference

Infer the UI message type for type-safe client components:

```ts
import { WorkflowAgent, InferWorkflowAgentUIMessage } from '@ai-sdk/workflow';

const myAgent = new WorkflowAgent({
  // ... configuration
});

export type MyAgentUIMessage = InferWorkflowAgentUIMessage<typeof myAgent>;
```

## Migrating from `DurableAgent`

`WorkflowAgent` replaces the Workflow DevKit's [`DurableAgent`](https://workflow-sdk.dev/docs/api-reference/workflow-ai/durable-agent). The two share the same core idea — a durable agent loop that runs inside a workflow — but `WorkflowAgent` moves the class into the AI SDK, tightens typing, and introduces first-class tool approval. If you are using `DurableAgent` today, follow the steps below to switch.

### Change the import and class name

`DurableAgent` was exported from `workflow/ai`. `WorkflowAgent` is exported from `@ai-sdk/workflow`, alongside its helpers.

```diff
- import { DurableAgent } from 'workflow/ai';
+ import { WorkflowAgent, type ModelCallStreamPart } from '@ai-sdk/workflow';

- const agent = new DurableAgent({
+ const agent = new WorkflowAgent({
    model: 'anthropic/claude-sonnet-5.5',
    instructions: 'You are a helpful assistant.',
    tools: { /* ... */ },
  });
```

Install the new package alongside `workflow`:

```bash
npm install @ai-sdk/workflow workflow@5
```

### Write `ModelCallStreamPart`, not `UIMessageChunk`

`DurableAgent` wrote `UIMessageChunk` objects directly to the writable returned by `getWritable()`. `WorkflowAgent` writes the lower-level `ModelCallStreamPart` shape and leaves the conversion to a transform at the response boundary. This keeps the durable stream provider-shaped and avoids baking a UI protocol into the workflow payload.

```diff
  // Inside the workflow
  await agent.stream({
    messages,
-   writable: getWritable<UIMessageChunk>(),
+   writable: getWritable<ModelCallStreamPart>(),
  });
```

```diff
  // Inside the route handler
+ import { createModelCallToUIChunkTransform } from '@ai-sdk/workflow';

  return createUIMessageStreamResponse({
-   stream: run.readable,
+   stream: run.readable.pipeThrough(createModelCallToUIChunkTransform()),
  });
```

### Replace `maxSteps` with `stopWhen`

`DurableAgent` accepted `maxSteps` directly. `WorkflowAgent` uses the AI SDK's shared `stopWhen` conditions so the same stop logic works across `ToolLoopAgent`, `generateText`, and `streamText`.

```diff
+ import { isStepCount } from 'ai';

  await agent.stream({
    messages,
-   maxSteps: 10,
+   stopWhen: isStepCount(10),
  });
```

See [Loop Control](/docs/agents/loop-control) for the full list of stop conditions.

### Replace `experimental_output` with `output`

```diff
+ import { Output } from '@ai-sdk/workflow';

  await agent.stream({
    messages,
-   experimental_output: Output.object({ schema }),
+   output: Output.object({ schema }),
  });
```

The returned value is now on `result.output` (previously `result.experimental_output`).

### WorkflowAgent: Use `needsApproval` for human-in-the-loop tools

With `DurableAgent`, tool approval was implemented by calling a Hook from inside the tool's `execute` function. `WorkflowAgent` makes approval a first-class tool property — the agent emits the approval request, suspends the workflow, and resumes automatically when the user responds.

```diff
  bookFlight: tool({
    description: 'Book a flight',
    inputSchema: z.object({ flightId: z.string() }),
+   needsApproval: true,
-   execute: async (input) => {
-     const approved = await waitForApprovalHook(input);
-     if (!approved) throw new Error('Denied');
-     return bookFlightStep(input);
-   },
+   execute: bookFlightStep,
  }),
```

`needsApproval` also accepts an async function so you can decide per-input
whether approval is required (see [Tool Approval](#tool-approval) above).

### `uiMessages` / `collectUIMessages` is gone

`DurableAgent.stream()` returned accumulated `uiMessages` when `collectUIMessages: true` was set. `WorkflowAgent.stream()` returns `ModelMessage[]` on `result.messages` instead.

For persistence, store `UIMessage[]` as your source of truth and call [`convertToModelMessages`](/docs/reference/ai-sdk-ui/convert-to-model-messages) before passing them to the agent — this is the pattern described in [Chatbot Message Persistence](/docs/ai-sdk-ui/chatbot-message-persistence). There is no built-in `ModelMessage` → `UIMessage` conversion, so avoid persisting `result.messages` as your only copy if you need to render the conversation in the UI later.

```diff
  const result = await agent.stream({
    messages,
    writable: getWritable<ModelCallStreamPart>(),
-   collectUIMessages: true,
  });

- return { uiMessages: result.uiMessages };
+ return { messages: result.messages };
```

### Non-streaming generation

Use `agent.generate()` when you need the completed result without streaming:

```ts
export async function summarize(modelId: string, text: string) {
  'use workflow';

  const agent = new WorkflowAgent({
    model: modelId,
    instructions: 'Summarize the text briefly.',
  });
  const result = await agent.generate({ prompt: text });
  return { text: result.text, usage: result.usage };
}
```

Generation uses the provider's `doGenerate()` method and the same durable tool loop as streaming. It defaults to a maximum of 20 steps, while `stream()` remains unbounded unless you set `stopWhen`. Generation failures reject; stream model errors retain their existing `result.error` behavior.

The generated result uses core's final-step text and all-step content, tool results, warnings, and usage semantics. `responseMessages` contains generated messages only. `output` defaults to text and supports constructor or per-call output specifications. End callbacks run before complete-output parsing.

Use result getters inside the workflow, then return selected serializable fields through `run.returnValue`. Returning the whole result class is not supported. See the [generate reference](/docs/reference/ai-sdk-workflow/workflow-agent#generate) for options and remaining differences.

#### Approval round trips without streaming

Both `generate()` and `stream()` return approval requests even when no writable is provided. For generation, keep `result.responseMessages`, display the requested tool and input to the user, then append their decision to the next call:

```ts
const pending = await agent.generate({ messages });
// Persist these messages and collect the user's decision outside the workflow.
const continuation = [...messages, ...pending.responseMessages];

// In a later workflow invocation, using the same tool definitions:
const resumed = await agent.generate({
  messages: [
    ...continuation,
    {
      role: 'tool',
      content: [{ type: 'tool-approval-response', approvalId, approved }],
    },
  ],
});
```

Use the `approvalId` from the returned `tool-approval-request` and the user's `approved` decision. Preserve the request signature and original tool-call input when storing the conversation. Configure `experimental_toolApprovalSecret` with the same environment-variable reference for both invocations. Secrets are resolved inside durable signing and verification steps.

Local tools execute only after approval validation; denied tools produce an execution-denied result. Provider-executed approvals are forwarded to the provider. Tools that wait on Workflow hooks can also suspend and resume during generation.

#### Deadlines, cancellation, and replay

Inside a workflow, `timeout` sets an absolute deadline for model calls. Each model step checks it before dispatch, before SDK retry attempts, and after the provider returns. The same deadline remains in effect after a tool suspends and resumes. SDK `maxRetries` governs model retries; the durable model step does not add another retry layer. Expiration before dispatch rejects with `TimeoutError`; expiration during SDK retry backoff rejects with `AbortError`. Neither starts a new retry.

A model deadline does not cancel a tool waiting on a Workflow hook. The run can remain suspended after that deadline. Once the hook resumes, any subsequent model call fails before dispatch if the deadline has expired. To stop the suspended run, use the Workflow runtime's `run.cancel()` API. Cancellation does not undo external side effects that already happened.

Workflow reuses completed durable model and tool steps on replay. Put external side effects inside step functions and use the external service's idempotency mechanism where necessary: completed-step replay does not guarantee exactly-once effects for an interrupted attempt.

Generated files and result getters are available inside workflow code. Return the data the caller needs, for example:

```ts
return {
  text: result.text,
  files: result.files.map(file => ({
    data: file.uint8Array,
    mediaType: file.mediaType,
  })),
  usage: result.usage,
  responseMessages: result.responseMessages,
};
```

The generated result class itself is not a supported `run.returnValue` shape. Request and response bodies are omitted unless explicitly enabled with `include`; available response headers, timestamps, IDs and provider metadata are retained. Workflow uses its existing callback shapes and preparation rules, and reports model response timing rather than complete tool/step timing. It does not implement ToolLoopAgent's `callOptionsSchema` abstraction or core's live streaming readers and HTTP response helpers.

### Replace `experimental_context` with `runtimeContext` and `toolsContext`

`WorkflowAgent` no longer accepts `experimental_context`. Split the value into shared agent state (`runtimeContext`) and per-tool state (`toolsContext`); each tool's `execute` then receives only its own validated entry as `context`. See [runtimeContext and toolsContext](#runtimecontext-and-toolscontext) for the full shape.

```diff
  const agent = new WorkflowAgent({
    model: 'anthropic/claude-sonnet-5.5',
    tools: { weather: weatherTool },
-   experimental_context: { tenantId: 'tenant_123', apiKey: 'sk-...' },
+   runtimeContext: { tenantId: 'tenant_123' },
+   toolsContext: { weather: { apiKey: 'sk-...' } },
  });
```

### Everything else

Other options carry over with the same names: `prepareStep`, `onStart`, `onStepStart`, `onStepEnd`, `onEnd`, `onError`, `toolChoice`, `activeTools`, `timeout`, `repairToolCall`, `experimental_sandbox`, and the usual generation settings (`temperature`, `maxOutputTokens`, `topP`, …). `WorkflowAgent` additionally adds `prepareCall` (runs once before the loop) and the `onToolExecutionStart` / `onToolExecutionEnd` lifecycle callbacks documented above. The deprecated `experimental_onStart` and `experimental_onStepStart` aliases remain available for backwards compatibility.

## Next Steps

- [WorkflowAgent API Reference](/docs/reference/ai-sdk-workflow/workflow-agent) for detailed parameter documentation
- [WorkflowChatTransport API Reference](/docs/reference/ai-sdk-workflow/workflow-chat-transport) for stream reconnection options
- [Building Agents](/docs/agents/building-agents) for the in-memory `ToolLoopAgent` alternative
- [Loop Control](/docs/agents/loop-control) for advanced stop conditions

## Tool execution concurrency

Set `toolCallConcurrency` to limit how many local tool calls execute concurrently
within a step. Use `1` when calls must execute sequentially in model order:

```ts
const agent = new WorkflowAgent({
  model,
  tools,
  toolCallConcurrency: 1,
});
```

The limit applies to both `generate()` and `stream()`, including approved tool
calls resumed from message history. It must be a positive integer. By default,
all local calls execute concurrently; provider-executed tools are unaffected.
You can also return the setting from `prepareCall` or pass it to `generate()` or
`stream()`. Explicit call options take precedence over `prepareCall`, which takes
precedence over the constructor default.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Workflow Patterns
description: Learn workflow patterns for building reliable agents with the AI SDK.
url: "https://ai-sdk.dev/docs/agents/workflows"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Combine the building blocks from the [overview](/docs/agents/overview) with these patterns to add structure and reliability to your agents:

- [Sequential Processing](#sequential-processing-chains) - Steps executed in order
- [Parallel Processing](#parallel-processing) - Independent tasks run simultaneously
- [Evaluation/Feedback Loops](#evaluator-optimizer) - Results checked and improved iteratively
- [Orchestration](#orchestrator-worker) - Coordinating multiple components
- [Routing](#routing) - Directing work based on context

## Choose Your Approach

Consider these key factors:

- **Flexibility vs Control** - How much freedom does the LLM need vs how tightly you must constrain its actions?
- **Error Tolerance** - What are the consequences of mistakes in your use case?
- **Cost Considerations** - More complex systems typically mean more LLM calls and higher costs
- **Maintenance** - Simpler architectures are easier to debug and modify

**Start with the simplest approach that meets your needs**. Add complexity only when required by:

1. Breaking down tasks into clear steps
2. Adding tools for specific capabilities
3. Implementing feedback loops for quality control
4. Introducing multiple agents for complex workflows

Let's look at examples of these patterns in action.

## Patterns with Examples

These patterns, adapted from [Anthropic's guide on building effective agents](https://www.anthropic.com/research/building-effective-agents), serve as building blocks you can combine to create comprehensive workflows. Each pattern addresses specific aspects of task execution. Combine them thoughtfully to build reliable solutions for complex problems.

## Sequential Processing (Chains)

The simplest workflow pattern executes steps in a predefined order. Each step's output becomes input for the next step, creating a clear chain of operations. Use this pattern for tasks with well-defined sequences, like content generation pipelines or data transformation processes.

```ts
import { generateText, Output } from 'ai';
import { z } from 'zod';

async function generateMarketingCopy(input: string) {
  const model = "anthropic/claude-sonnet-5.5";

  // First step: Generate marketing copy
  const { text: copy } = await generateText({
    model,
    prompt: `Write persuasive marketing copy for: ${input}. Focus on benefits and emotional appeal.`,
  });

  // Perform quality check on copy
  const { output: qualityMetrics } = await generateText({
    model,
    output: Output.object({
      schema: z.object({
        hasCallToAction: z.boolean(),
        emotionalAppeal: z.number().min(1).max(10),
        clarity: z.number().min(1).max(10),
      }),
    }),
    prompt: `Evaluate this marketing copy for:
    1. Presence of call to action (true/false)
    2. Emotional appeal (1-10)
    3. Clarity (1-10)

    Copy to evaluate: ${copy}`,
  });

  // If quality check fails, regenerate with more specific instructions
  if (
    !qualityMetrics.hasCallToAction ||
    qualityMetrics.emotionalAppeal < 7 ||
    qualityMetrics.clarity < 7
  ) {
    const { text: improvedCopy } = await generateText({
      model,
      prompt: `Rewrite this marketing copy with:
      ${!qualityMetrics.hasCallToAction ? '- A clear call to action' : ''}
      ${qualityMetrics.emotionalAppeal < 7 ? '- Stronger emotional appeal' : ''}
      ${qualityMetrics.clarity < 7 ? '- Improved clarity and directness' : ''}

      Original copy: ${copy}`,
    });
    return { copy: improvedCopy, qualityMetrics };
  }

  return { copy, qualityMetrics };
}
```

## Routing

This pattern lets the model decide which path to take through a workflow based on context and intermediate results. The model acts as an intelligent router, directing the flow of execution between different branches of your workflow. Use this when handling varied inputs that require different processing approaches. In the example below, the first LLM call's results determine the second call's model size and system prompt.

```ts
import { generateText, Output } from 'ai';
import { z } from 'zod';

async function handleCustomerQuery(query: string) {
  const model = "anthropic/claude-sonnet-5.5";

  // First step: Classify the query type
  const { output: classification } = await generateText({
    model,
    output: Output.object({
      schema: z.object({
        reasoning: z.string(),
        type: z.enum(['general', 'refund', 'technical']),
        complexity: z.enum(['simple', 'complex']),
      }),
    }),
    prompt: `Classify this customer query:
    ${query}

    Determine:
    1. Query type (general, refund, or technical)
    2. Complexity (simple or complex)
    3. Brief reasoning for classification`,
  });

  // Route based on classification
  // Set model and system prompt based on query type and complexity
  const { text: response } = await generateText({
    model:
      classification.complexity === 'simple'
        ? 'openai/gpt-6-luna'
        : 'openai/gpt-6-astra',
    instructions: {
      general:
        'You are an expert customer service agent handling general inquiries.',
      refund:
        'You are a customer service agent specializing in refund requests. Follow company policy and collect necessary information.',
      technical:
        'You are a technical support specialist with deep product knowledge. Focus on clear step-by-step troubleshooting.',
    }[classification.type],
    prompt: query,
  });

  return { response, classification };
}
```

## Parallel Processing

Break down tasks into independent subtasks that execute simultaneously. This pattern uses parallel execution to improve efficiency while maintaining the benefits of structured workflows. For example, analyze multiple documents or process different aspects of a single input concurrently (like code review).

```ts
import { generateText, Output } from 'ai';
import { z } from 'zod';

// Example: Parallel code review with multiple specialized reviewers
async function parallelCodeReview(code: string) {
  const model = "anthropic/claude-sonnet-5.5";

  // Run parallel reviews
  const [securityReview, performanceReview, maintainabilityReview] =
    await Promise.all([
      generateText({
        model,
        instructions:
          'You are an expert in code security. Focus on identifying security vulnerabilities, injection risks, and authentication issues.',
        output: Output.object({
          schema: z.object({
            vulnerabilities: z.array(z.string()),
            riskLevel: z.enum(['low', 'medium', 'high']),
            suggestions: z.array(z.string()),
          }),
        }),
        prompt: `Review this code:
      ${code}`,
      }),

      generateText({
        model,
        instructions:
          'You are an expert in code performance. Focus on identifying performance bottlenecks, memory leaks, and optimization opportunities.',
        output: Output.object({
          schema: z.object({
            issues: z.array(z.string()),
            impact: z.enum(['low', 'medium', 'high']),
            optimizations: z.array(z.string()),
          }),
        }),
        prompt: `Review this code:
      ${code}`,
      }),

      generateText({
        model,
        instructions:
          'You are an expert in code quality. Focus on code structure, readability, and adherence to best practices.',
        output: Output.object({
          schema: z.object({
            concerns: z.array(z.string()),
            qualityScore: z.number().min(1).max(10),
            recommendations: z.array(z.string()),
          }),
        }),
        prompt: `Review this code:
      ${code}`,
      }),
    ]);

  const reviews = [
    { ...securityReview.output, type: 'security' },
    { ...performanceReview.output, type: 'performance' },
    { ...maintainabilityReview.output, type: 'maintainability' },
  ];

  // Aggregate results using another model instance
  const { text: summary } = await generateText({
    model,
    instructions: 'You are a technical lead summarizing multiple code reviews.',
    prompt: `Synthesize these code review results into a concise summary with key actions:
    ${JSON.stringify(reviews, null, 2)}`,
  });

  return { reviews, summary };
}
```

## Orchestrator-Worker

A primary model (orchestrator) coordinates the execution of specialized workers. Each worker optimizes for a specific subtask, while the orchestrator maintains overall context and ensures coherent results. This pattern excels at complex tasks requiring different types of expertise or processing.

```ts
import { generateText, Output } from 'ai';
import { z } from 'zod';

async function implementFeature(featureRequest: string) {
  // Orchestrator: Plan the implementation
  const { output: implementationPlan } = await generateText({
    model: "anthropic/claude-sonnet-5.5",
    output: Output.object({
      schema: z.object({
        files: z.array(
          z.object({
            purpose: z.string(),
            filePath: z.string(),
            changeType: z.enum(['create', 'modify', 'delete']),
          }),
        ),
        estimatedComplexity: z.enum(['low', 'medium', 'high']),
      }),
    }),
    instructions:
      'You are a senior software architect planning feature implementations.',
    prompt: `Analyze this feature request and create an implementation plan:
    ${featureRequest}`,
  });

  // Workers: Execute the planned changes
  const fileChanges = await Promise.all(
    implementationPlan.files.map(async file => {
      // Each worker is specialized for the type of change
      const workerSystemPrompt = {
        create:
          'You are an expert at implementing new files following best practices and project patterns.',
        modify:
          'You are an expert at modifying existing code while maintaining consistency and avoiding regressions.',
        delete:
          'You are an expert at safely removing code while ensuring no breaking changes.',
      }[file.changeType];

      const { output: change } = await generateText({
        model: "anthropic/claude-sonnet-5.5",
        output: Output.object({
          schema: z.object({
            explanation: z.string(),
            code: z.string(),
          }),
        }),
        instructions: workerSystemPrompt,
        prompt: `Implement the changes for ${file.filePath} to support:
        ${file.purpose}

        Consider the overall feature context:
        ${featureRequest}`,
      });

      return {
        file,
        implementation: change,
      };
    }),
  );

  return {
    plan: implementationPlan,
    changes: fileChanges,
  };
}
```

## Evaluator-Optimizer

Add quality control to workflows with dedicated evaluation steps that assess intermediate results. Based on the evaluation, the workflow proceeds, retries with adjusted parameters, or takes corrective action. This creates robust workflows capable of self-improvement and error recovery.

```ts
import { generateText, Output } from 'ai';
import { z } from 'zod';

async function translateWithFeedback(text: string, targetLanguage: string) {
  let currentTranslation = '';
  let iterations = 0;
  const MAX_ITERATIONS = 3;

  // Initial translation
  const { text: translation } = await generateText({
    model: "anthropic/claude-sonnet-5.5",
    instructions: 'You are an expert literary translator.',
    prompt: `Translate this text to ${targetLanguage}, preserving tone and cultural nuances:
    ${text}`,
  });

  currentTranslation = translation;

  // Evaluation-optimization loop
  while (iterations < MAX_ITERATIONS) {
    // Evaluate current translation
    const { output: evaluation } = await generateText({
      model: "anthropic/claude-sonnet-5.5",
      output: Output.object({
        schema: z.object({
          qualityScore: z.number().min(1).max(10),
          preservesTone: z.boolean(),
          preservesNuance: z.boolean(),
          culturallyAccurate: z.boolean(),
          specificIssues: z.array(z.string()),
          improvementSuggestions: z.array(z.string()),
        }),
      }),
      instructions: 'You are an expert in evaluating literary translations.',
      prompt: `Evaluate this translation:

      Original: ${text}
      Translation: ${currentTranslation}

      Consider:
      1. Overall quality
      2. Preservation of tone
      3. Preservation of nuance
      4. Cultural accuracy`,
    });

    // Check if quality meets threshold
    if (
      evaluation.qualityScore >= 8 &&
      evaluation.preservesTone &&
      evaluation.preservesNuance &&
      evaluation.culturallyAccurate
    ) {
      break;
    }

    // Generate improved translation based on feedback
    const { text: improvedTranslation } = await generateText({
      model: "anthropic/claude-sonnet-5.5",
      instructions: 'You are an expert literary translator.',
      prompt: `Improve this translation based on the following feedback:
      ${evaluation.specificIssues.join('\n')}
      ${evaluation.improvementSuggestions.join('\n')}

      Original: ${text}
      Current Translation: ${currentTranslation}`,
    });

    currentTranslation = improvedTranslation;
    iterations++;
  }

  return {
    finalTranslation: currentTranslation,
    iterationsRequired: iterations,
  };
}
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Harness Adapters
description: Learn about the available AI SDK harness adapters.
url: "https://ai-sdk.dev/docs/ai-sdk-harnesses/harness-adapters"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Harness adapters connect `HarnessAgent` to a specific agent runtime. They are
the harness equivalent of AI SDK model providers: each adapter wraps one runtime
and normalizes its sessions, stream events, tools, usage, lifecycle state, and
configuration into the harness contract.

## AI SDK Harness Adapters

The AI SDK includes the following harness adapters:

- [Claude Code](/providers/ai-sdk-harnesses/claude-code) (`@ai-sdk/harness-claude-code`)
- [Cline](/providers/ai-sdk-harnesses/cline) (`@ai-sdk/harness-cline`)
- [Codex](/providers/ai-sdk-harnesses/codex) (`@ai-sdk/harness-codex`)
- [Cursor](/providers/ai-sdk-harnesses/cursor) (`@ai-sdk/harness-cursor`)
- [Deep Agents](/providers/ai-sdk-harnesses/deepagents) (`@ai-sdk/harness-deepagents`)
- [fx](/providers/ai-sdk-harnesses/fx) (`@ai-sdk/harness-fx`)
- [GitHub Copilot](/providers/ai-sdk-harnesses/github-copilot) (`@ai-sdk/harness-github-copilot`)
- [Grok Build](/providers/ai-sdk-harnesses/grok-build) (`@ai-sdk/harness-grok-build`)
- [OpenCode](/providers/ai-sdk-harnesses/opencode) (`@ai-sdk/harness-opencode`)
- [Pi](/providers/ai-sdk-harnesses/pi) (`@ai-sdk/harness-pi`)

### Coming Soon

- Amp (`@ai-sdk/harness-amp`)
- Goose (`@ai-sdk/harness-goose`)
- Mastra (`@ai-sdk/harness-mastra`)

## Adapter Capabilities

| Adapter                                                      | Runtime location | Custom tools | Custom skills | Structured output | Built-in tool approval | Built-in tool filtering | Image input | History access |
| ------------------------------------------------------------ | ---------------- | ------------ | ------------- | ----------------- | ---------------------- | ----------------------- | ----------- | -------------- |
| [Claude Code](/providers/ai-sdk-harnesses/claude-code)       | Sandbox bridge   | ✓            | ✓             | ✓                 | ✓                      | ✓                       | ✓           | ✗              |
| [Cline](/providers/ai-sdk-harnesses/cline)                   | Host process     | ✓            | ✓             | ✓                 | ✓                      | ✓                       | ✓           | ✗              |
| [Codex](/providers/ai-sdk-harnesses/codex)                   | Sandbox bridge   | ✓            | ✓             | ✓                 | ✗                      | ✗                       | ✓           | ✗              |
| [Cursor](/providers/ai-sdk-harnesses/cursor)                 | Sandbox via ACP  | ✓            | ✓             | ✗                 | ✗                      | ✗                       | ✓           | ✗              |
| [Deep Agents](/providers/ai-sdk-harnesses/deepagents)        | Sandbox bridge   | ✓            | ✓             | ✓                 | ✓                      | ✓ via auto-rejection    | ✓           | ✗              |
| [fx](/providers/ai-sdk-harnesses/fx)                         | Host process     | ✓            | ✓             | ✓                 | ✓                      | ✓                       | ✓           | ✗              |
| [GitHub Copilot](/providers/ai-sdk-harnesses/github-copilot) | Sandbox via ACP  | ✓            | ✓             | ✗                 | ✗                      | ✗                       | ✓           | ✗              |
| [Grok Build](/providers/ai-sdk-harnesses/grok-build)         | Sandbox via ACP  | ✓            | ✓             | ✓                 | ✗                      | ✗                       | ✓           | ✗              |
| [OpenCode](/providers/ai-sdk-harnesses/opencode)             | Sandbox bridge   | ✓            | ✓             | ✓                 | ✓                      | ✓                       | ✓           | ✗              |
| [Pi](/providers/ai-sdk-harnesses/pi)                         | Host process     | ✓            | ✓             | ✗                 | ✓                      | ✓                       | ✓           | ✗              |

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: HarnessAgent
description: Create and run AI SDK HarnessAgent sessions.
url: "https://ai-sdk.dev/docs/ai-sdk-harnesses/harness-agent"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

`HarnessAgent` is an AI SDK `Agent` implementation backed by a harness adapter.
It gives you `generate()` and `stream()` methods that return AI SDK-compatible
results while a preconfigured harness powers these results.

## Installation

Install the core harness package, a harness adapter, and a sandbox adapter:

```bash
pnpm add @ai-sdk/harness @ai-sdk/harness-claude-code @ai-sdk/sandbox-vercel
```

Bridge-backed harnesses such as Claude Code and Codex require using a real network sandbox
like `@ai-sdk/sandbox-vercel`. Host-runtime harnesses such as Pi can also run with
`@ai-sdk/sandbox-just-bash` because they do not need a sandbox-exposed port.

## Create an Agent

```ts
import { HarnessAgent } from '@ai-sdk/harness/agent';
import { claudeCode } from '@ai-sdk/harness-claude-code';

export const agent = new HarnessAgent({
  harness: claudeCode,
  model: 'claude-sonnet-4-6',
  instructions:
    'You are a careful coding assistant. Prefer small changes and explain tradeoffs.',
});
```

Construct the agent at module scope. It holds configuration, not a live session.
Live state belongs to `HarnessAgentSession`.
Create a sandbox session separately and pass it to `agent.createSession()`:

```ts
import { createVercelNetworkSandboxSession } from '@ai-sdk/sandbox-vercel';

const sandboxSession = await createVercelNetworkSandboxSession({
  runtime: 'node24',
  ports: [4000],
  template: await agent.getSandboxTemplate(),
});
```

Set `model` to select the model that the harness runtime uses. Model identifiers
are specific to each harness. When omitted, the harness uses its default model.
The model is applied per turn, so `prepareCall` can replace it between turns.

To use this agent, ensure environment variables with sandbox and harness credentials
are set.

## Run a Turn

```ts
const session = await agent.createSession({ sandboxSession });

let exitCode = 0;
try {
  const result = await agent.generate({
    session,
    prompt: 'Create a short TODO.md for this repository.',
  });

  console.log(result.text);
} catch (err) {
  exitCode = 1;
  console.error(err);
} finally {
  await session.destroy();
  await sandboxSession.destroy();
  process.exit(exitCode);
}
```

`generate()` drains the turn and returns a `GenerateTextResult`.

Use `stream()` for incremental output:

```ts
const session = await agent.createSession({ sandboxSession });

let exitCode = 0;
try {
  const result = await agent.stream({
    session,
    prompt: 'Create a short TODO.md for this repository.',
  });

  for await (const part of result.stream) {
    if (part.type === 'text-delta') {
      process.stdout.write(part.text);
    }
  }
} catch (err) {
  exitCode = 1;
  console.error(err);
} finally {
  await session.destroy();
  await sandboxSession.destroy();
  process.exit(exitCode);
}
```

### Image Inputs

For harness adapters that support images, you can include an image as a file part
in the latest user message. Set `mediaType` to the image's MIME type:

```ts
import { readFileSync } from 'node:fs';

const result = await agent.generate({
  session,
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'Describe the image in one sentence.',
        },
        {
          type: 'file',
          mediaType: 'image/png',
          data: readFileSync('image.png'),
        },
      ],
    },
  ],
});
```

For image input support, see the
[adapter capability table](/docs/ai-sdk-harnesses/harness-adapters#adapter-capabilities).

## Lifecycle Callbacks

Configure lifecycle callbacks on `HarnessAgent` to observe agent calls, model
steps, and tool executions:

```ts
const agent = new HarnessAgent({
  harness: claudeCode,
  tools: { weather },
  onStart: event => console.log('call started', event.callId),
  onStepStart: event => console.log('step started', event.stepNumber),
  onLanguageModelCallStart: event =>
    console.log('model call started', event.modelId),
  onLanguageModelCallEnd: event =>
    console.log('model call ended', event.finishReason),
  onToolExecutionStart: event =>
    console.log('tool started', event.toolCall.toolName),
  onToolExecutionEnd: event => console.log('tool ended', event.toolOutput.type),
  onStepEnd: step => console.log('step ended', step.stepNumber),
  onEnd: event => console.log('call ended', event.finishReason),
});
```

Callbacks configured on individual `generate()` and `stream()` calls are
invoked in addition to settings callbacks, with settings callbacks invoked
first. Callback errors are ignored and do not change agent execution.

Harness runtimes execute their built-in tools internally. For those tools,
`onToolExecutionStart` and `onToolExecutionEnd` describe the logical tool
lifecycle after the runtime reports the result. The callbacks are still
delivered before the tool result is published to the result stream.

## Runtime Context

Set `runtimeContext` on `HarnessAgent` to make application context available to
every turn. Use `prepareCall` to replace it for each new prompt turn. The
prepared context is available through `generate()` and `stream()` and remains
the same during in-memory continuations with `continueGenerate()` and
`continueStream()`. Lifecycle callbacks receive the full context, and it is
also available on each result's `finalStep.runtimeContext`.

```ts
const agent = new HarnessAgent({
  harness: claudeCode,
  runtimeContext: {
    conversationId: 'conversation-123',
    userId: 'user-456',
  },
  telemetry: {
    includeRuntimeContext: {
      conversationId: true,
    },
  },
  onEnd: ({ runtimeContext }) => {
    console.log(runtimeContext.userId);
  },
});
```

Runtime context is excluded from telemetry by default. Use
`telemetry.includeRuntimeContext` to allowlist individual top-level properties;
only properties set to `true` are sent to telemetry integrations. Lifecycle
callbacks and results still receive the full context, so avoid storing secrets
that those consumers should not access.

## Generate Structured Output

Set `output` when constructing `HarnessAgent` to require the same typed output
on every turn. The agent converts the output specification to JSON Schema for
the harness adapter, validates the completed response, and returns the parsed
value through `result.output`.

```ts
import { HarnessAgent } from '@ai-sdk/harness/agent';
import { Output } from 'ai';
import { z } from 'zod';

const agent = new HarnessAgent({
  harness: claudeCode,
  output: Output.object({
    schema: z.object({
      recipe: z.object({
        name: z.string(),
        ingredients: z.array(
          z.object({
            name: z.string(),
            amount: z.number(),
            unit: z.enum(['oz', 'fl oz', 'cup', 'gallon']),
          }),
        ),
        steps: z.array(z.string()),
      }),
    }),
  }),
});

const session = await agent.createSession({ sandboxSession });
try {
  const result = await agent.generate({
    session,
    prompt: 'Generate a lasagna recipe.',
  });
  console.dir(result.output, { depth: Infinity });
} finally {
  await session.destroy();
}
```

With `stream()`, read `partialOutputStream` for incrementally parsed values and
await `result.output` for the validated final value. Structured data also remains
available as JSON in the normal text and stream surfaces; adapters do not add it
to the `finish` part.

Harness structured output requires a schema. Schema-less `Output.json()` and
adapters or runtime configurations that cannot enforce the schema throw
`HarnessCapabilityUnsupportedError`; see the
[adapter capability table](/docs/ai-sdk-harnesses/harness-adapters#adapter-capabilities).

## Messages and History

A harness session owns its native conversation history. When you pass `messages`
or a message-array `prompt`, `HarnessAgent` takes the latest user message as the
fresh input for the turn. It does not replay the full prior conversation into
the harness.

A trailing tool message is handled differently: tool approval responses and
client-provided tool results continue the unfinished harness turn that produced
the corresponding request or tool call.

This is different from model calls, where the application usually sends the full
message history. In chat routes, persist and resume the harness session instead
of relying on message replay.

### Reading the Runtime's Own History

The runtime's conversation can grow outside your process — the same
conversation continued interactively in the agent's own CLI, for instance —
and the live event stream never sees those exchanges. Adapters that can reach
their runtime's store expose them through `session.readHistory()`:

```ts
const { messages, cursor } = await session.readHistory();
// Messages use role/content shapes. Adapter-specific details, including the
// runtime's raw record, are available under harnessMetadata.

// Later: read only what was recorded since.
const delta = await session.readHistory({ since: cursor });
```

History reads are adapter-specific; see the adapter capability table before
calling this method.
`readHistory()` throws `HarnessCapabilityUnsupportedError` when the adapter
does not implement history reads, and `HarnessHistoryUnavailableError` when it
does but cannot reach the runtime's store from this environment (for example,
the store lives inside a remote sandbox). A conversation with no recorded
messages yet is not an error; it resolves to an empty `messages` array.

## Session Lifecycle

End every session explicitly:

- `session.destroy()` stops the runtime and discards resumability.
- `session.detach()` parks the runtime, returns resume state, and leaves a
  supplied sandbox running for a later attach. If the turn is unfinished, the
  resume state includes the continuation state.
- `session.stop()` saves resume state, then stops the runtime. If
  the turn is unfinished, the resume state includes the continuation state.
- `session.suspendTurn()` is for advanced active-turn continuation across a
  process boundary.
- `session.hasUnfinishedTurn()` reports whether the current turn must be
  continued or suspended before the session accepts a new prompt.

Use `destroy()` for one-off scripts and tests. Use `detach()` or `stop()` for
HTTP routes that need multi-turn continuity.

## Change Settings Between Turns

Use `callOptionsSchema` and `prepareCall` to derive `model`, `skills`,
`instructions`, `tools`, and `runtimeContext` for each new turn. This follows the same
call-options pattern as `ToolLoopAgent`:

```ts
import { HarnessAgent } from '@ai-sdk/harness/agent';
import { tool } from 'ai';
import { z } from 'zod';

const getPolicy = tool({
  description: 'Look up the active project policy.',
  inputSchema: z.object({}),
  execute: async () => 'Keep public APIs backward compatible.',
});

const agent = new HarnessAgent({
  harness: claudeCode,
  tools: { getPolicy },
  callOptionsSchema: z.object({
    area: z.enum(['frontend', 'backend']),
    enablePolicyTool: z.boolean(),
    useCheaperModel: z.boolean(),
  }),
  prepareCall: ({ options, ...call }) => ({
    ...call,
    model: options.useCheaperModel ? 'claude-haiku-4-5' : undefined,
    instructions: `Work as the ${options.area} specialist.`,
    runtimeContext: { area: options.area },
    skills: [options.area === 'frontend' ? frontendSkill : backendSkill],
    tools: options.enablePolicyTool ? { getPolicy } : undefined,
  }),
});

const session = await agent.createSession({ sandboxSession });
try {
  await agent.generate({
    session,
    prompt: 'Review the current implementation.',
    options: {
      area: 'frontend',
      enablePolicyTool: false,
      useCheaperModel: false,
    },
  });

  await agent.generate({
    session,
    prompt: 'Now review the API contract.',
    options: {
      area: 'backend',
      enablePolicyTool: true,
      useCheaperModel: true,
    },
  });
} finally {
  await session.destroy();
}
```

`prepareCall` runs for a new prompt after its custom `options` have been
validated. Its settings are then fixed for that whole turn. If the turn pauses
for a tool result, approval, stop condition, or process handoff, its
continuation reuses the same settings and does not call `prepareCall` again.
This prevents settings from changing mid-turn.

Changing `model` does not create a new harness session. Each adapter switches or
reconfigures its runtime before the next prompt while preserving the session's
conversation history.

The Codex adapter starts a fresh native Codex thread when its skills,
instructions, or tool catalog changes because `codex exec resume` retains the
original native thread bootstrap. The harness session remains usable, but prior
native conversation context does not carry across that settings-change boundary.
Turns with unchanged settings continue the existing native thread.

Pass `abortSignal` directly to `generate()` or `stream()`; it is already a
per-call setting and is not part of `prepareCall`. `output` also stays fixed on
the agent because its response format is tied to the agent's output schema.

When you pass `sandboxSession` to `agent.createSession()`, the caller retains
ownership of that sandbox. `session.stop()` and `session.destroy()` still end
the harness runtime but do not stop or destroy the supplied sandbox session.
In this case, the agent does not need a `sandbox` provider in its constructor.

```ts
const session = await agent.createSession({
  sessionId: chatId,
  sandboxSession,
});

try {
  const result = await agent.stream({ session, messages });

  for await (const part of result.stream) {
    if (part.type === 'text-delta') {
      process.stdout.write(part.text);
    }
  }

  const resumeState = await session.detach();
  await persistResumeState({ chatId, resumeState });
} catch (error) {
  await session.destroy();
  throw error;
}
```

## Resuming

Persist the opaque resume state. In this example, the harness `sessionId` is
`chatId`, and the sandbox ID is derived from the same known ID. If your
application cannot derive the sandbox ID, persist `sandboxSession.id` alongside
the harness `sessionId` and resume state instead. Reattach the sandbox before
asking the agent to resume. When you choose `sandboxId` on creation, it names a **new**
live sandbox and never looks one up. Reusing that ID with the creator fails
if the named sandbox exists; only the resume function performs a lookup.

```ts
const resumeState = await loadResumeState({ chatId });
const sandboxId = `chat-${chatId}`;

const sandboxSession = resumeState
  ? await resumeVercelNetworkSandboxSession({ sandboxId })
  : await createVercelNetworkSandboxSession({
      sandboxId,
      ports: [4000],
      template: await agent.getSandboxTemplate(),
    });

const session = await agent.createSession(
  resumeState
    ? { sessionId: chatId, resumeFrom: resumeState, sandboxSession }
    : { sessionId: chatId, sandboxSession },
);
```

`HarnessAgent` validates that the resume state was produced by the same harness
adapter before handing it to the runtime. If the resume state includes an
unfinished turn, call `continueStream()` or `continueGenerate()` before sending a
new prompt. The Vercel resume function reattaches whether the sandbox is still
running after `detach()` or stopped with a restorable snapshot. After
`sandboxSession.destroy()` deletes it, that ID cannot be resumed.

## Continue a Suspended Turn

For advanced workflows that must hand off an active turn across a process
boundary, suspend the turn and persist the continuation state:

```ts
if (session.hasUnfinishedTurn()) {
  const continuationState = await session.suspendTurn();
  await persistContinuationState({ chatId, continuationState });
}
```

When you only have raw continuation state from `suspendTurn()`, resume with
`continueFrom`, then continue the turn without sending a new prompt:

```ts
const session = await agent.createSession({
  sessionId: chatId,
  continueFrom: continuationState,
  sandboxSession,
  // Rebind this when the suspended turn used host-only tool context:
  toolsContext,
  runtimeContext,
});

const result = await agent.continueStream({ session });
```

Use `continueStream()` for incremental output, or `continueGenerate()` to drain
the continued turn and return a `GenerateTextResult`.

`toolsContext` and `runtimeContext` are intentionally not stored in
continuation state because they can contain credentials or non-serializable
host objects. When a suspended turn used context derived by `prepareCall`,
pass the same values to `createSession` to rebind them in the process that
continues the turn. After a process resume without rebinding, the continuation
uses the agent's constructor-level `runtimeContext` and `toolsContext`.

## Stop After a Harness Step

Use `stopWhen` to opt into semantic step boundaries. Predicates run after real
harness tool steps that can continue into another model step, and receive the
completed steps from the current invocation. When a predicate matches, the
returned result finishes while the underlying turn remains unfinished. A
terminal text-only step instead consumes the turn's `finish` event and finishes
naturally.

```ts
import { isStepCount } from 'ai';

const steppedAgent = new HarnessAgent({
  harness: claudeCode,
  stopWhen: isStepCount(1),
});

const session = await steppedAgent.createSession({ sandboxSession });
const result = await steppedAgent.generate({
  session,
  prompt: 'Create a short TODO.md for this repository.',
});

if (session.hasUnfinishedTurn()) {
  const continueFrom = await session.suspendTurn();
  await persistContinuationState({ chatId, continuationState: continueFrom });
}
```

`stopWhen` has no default. When omitted, `HarnessAgent` continues running until
the turn naturally finishes or pauses for host input, preserving the behavior
of agents without step control. Pass one predicate or an array; matching any
predicate finishes the current result slice. Resume a stopped turn with
`createSession({ continueFrom })` and `continueStream()` or
`continueGenerate()`.

## Prepare the Sandbox

Use `sandboxConfig` to prepare the sandbox before the harness starts.

`sandboxConfig.onBootstrap` runs during sandbox template creation, after the
harness adapter's own bootstrap and before snapshot-capable providers publish a
snapshot. Use it for expensive setup that should be reused by future sessions.
When you provide `onBootstrap`, also provide `bootstrapHash`; change the hash
whenever the bootstrap output should invalidate the reusable snapshot.

`sandboxConfig.onSession` runs after each sandbox session is acquired and its
working directory exists, including resumed sessions. Use it for per-session
files or lightweight configuration.

```ts
const agent = new HarnessAgent({
  harness: claudeCode,
  sandboxConfig: {
    workDir: 'repo',
    bootstrapHash: 'ripgrep-v1',
    onBootstrap: async ({ session, abortSignal }) => {
      const result = await session.run({
        command:
          'command -v rg >/dev/null || (apt-get update && apt-get install -y ripgrep)',
        abortSignal,
      });
      if (result.exitCode !== 0) {
        throw new Error(`Failed to install ripgrep: ${result.stderr}`);
      }
    },
    onSession: async ({ session, sessionWorkDir, abortSignal }) => {
      await session.writeTextFile({
        path: `${sessionWorkDir}/README.md`,
        content: 'Session notes for the harness.',
        abortSignal,
      });
    },
  },
});
```

`workDir` is optional. When provided, it must be relative to the sandbox's
default working directory and is used as the session working directory. Set it
to `'.'` to use the sandbox's default working directory itself. When omitted,
regular sessions use the default `<harnessId>-<sessionId>` directory, while
`onBootstrap` receives the sandbox's default working directory.

## Prepare Reusable Sandboxes

Use `agent.getSandboxTemplate()` to resolve one harness's bootstrap recipe and
`sandboxConfig.onBootstrap` before creating a sandbox. The template's identity
is known before sandbox creation. Vercel persists the prepared snapshot and
creates a live sandbox from it; identical calls reuse that snapshot:

```ts
const sandboxSession = await createVercelNetworkSandboxSession({
  runtime: 'node24',
  ports: [4000],
  template: await agent.getSandboxTemplate(),
});

const session = await agent.createSession({ sandboxSession });
try {
  await agent.generate({ session, prompt: 'Inspect this repository.' });
} finally {
  await session.destroy();
  await sandboxSession.destroy();
}
```

For multiple harnesses sharing a caller bootstrap hook, build one template from
all adapters. The last adapter wins for duplicate `harnessId` values, and the
recipe identities are sorted before hashing. `onSession` is not part of the
template and runs separately on every acquired session.

```ts
import { createHarnessSandboxTemplate } from '@ai-sdk/harness/agent';

const template = await createHarnessSandboxTemplate({
  harnesses: [claudeCode, codex],
  sandboxConfig,
});
console.log(template?.identity);

const sandboxSession = await createVercelNetworkSandboxSession({
  source: { type: 'snapshot', snapshotId: baseSnapshotId },
  ports: [4000],
  template,
});
```

The derived template includes the source snapshot ID in its cache key, so
repeated calls with the same source and template reuse the prepared snapshot.
Pin mutable Git revisions, image tags, and source URLs when relying on cached
templates. `sandboxId` names the live sandbox, not its reusable template.
The existing native `name` option still names the live sandbox; if both are
supplied, their values must match.

## Settings

`HarnessAgent` accepts these main settings:

- `harness`: the adapter instance.
- `model`: optional harness-specific model identifier. When omitted, the
  harness uses its default model.
- `id`: optional stable agent identifier.
- `instructions`: instructions appended to the runtime's system or developer
  prompt when supported, or prepended to the user prompt otherwise.
- `headers`: additional headers sent with model requests. Headers are fixed at
  construction time. `authorization`, `x-api-key`, `user-agent`, and
  `x-client-app` are not allowed.
- `callOptionsSchema` and `prepareCall`: validate custom call options and derive
  model, skills, instructions, and tools for each new turn.
- `output`: typed output specification applied to every turn.
- `stopWhen`: condition(s) for finishing a result slice after a completed
  harness tool step that can continue into another model step.
- `tools`: AI SDK tools executed by the host when the harness calls them.
- `runtimeContext`: application context forwarded to lifecycle callbacks and
  results for every turn; `prepareCall` can replace it for a new prompt turn.
- `activeTools`: allowlist of built-in and host-executed tools the harness can
  call.
- `inactiveTools`: denylist of built-in and host-executed tools the harness
  cannot call.
- `skills`: instruction bundles surfaced by the adapter.
- `permissionMode`: built-in tool permission mode.
- `toolApproval`: approval status map for host-executed tools.
- `sandboxConfig`: sandbox working-directory and lifecycle hook configuration.
- `telemetry`, `debug`, and `onLog`: observability and diagnostics. Use
  `telemetry.includeRuntimeContext` to allowlist runtime context properties for
  telemetry.

Telemetry reports each turn's resolved model, instructions, and active
host-defined tools. Skills are adapter context and do not have a corresponding
AI SDK telemetry field, so they are not included in standard telemetry events.

Adapter-specific settings belong on the adapter factory, for example
`createCodex({ reasoningEffort: 'high' })`.

## Custom Sandbox Orchestration

When your application creates and manages sandboxes itself, prepare the network
sandbox session first and then pass that same session to `agent.createSession()`.
The agent does not create the sandbox and does not stop or destroy the
caller-owned sandbox.

```ts
import { HarnessAgent } from '@ai-sdk/harness/agent';
import { claudeCode } from '@ai-sdk/harness-claude-code';
import { createVercelNetworkSandboxSessionFromNativeSandbox } from '@ai-sdk/sandbox-vercel';
import { Sandbox } from '@vercel/sandbox';

const sandbox = await Sandbox.create({
  runtime: 'node24',
  ports: [4000],
});
const sandboxSession =
  createVercelNetworkSandboxSessionFromNativeSandbox(sandbox);

const agent = new HarnessAgent({ harness: claudeCode });
const session = await agent.createSession({ sandboxSession });

try {
  const result = await agent.stream({
    session,
    prompt: 'Create a short TODO.md for this repository.',
  });

  for await (const part of result.stream) {
    if (part.type === 'text-delta') {
      process.stdout.write(part.text);
    }
  }
} finally {
  await session.destroy();
  await sandboxSession.destroy();
}
```

### Basic Sandbox Sessions Without Network Control

The following example demonstrates how the basic-session API works. If your
project can expose a full network sandbox session, passing that session is
strongly recommended. Pass a restricted basic session only when your project
cannot expose the full network session.

A basic sandbox session only exposes filesystem and process APIs. The agent
still leaves the sandbox lifecycle to the caller. For a bridge-backed harness,
configure the bridge port and its externally reachable endpoint on the adapter
because the basic session cannot resolve them.

```ts
import { HarnessAgent } from '@ai-sdk/harness/agent';
import { createClaudeCode } from '@ai-sdk/harness-claude-code';
import {
  createVercelNetworkSandboxSessionFromNativeSandbox,
  createVercelSandboxSessionFromNativeSandbox,
} from '@ai-sdk/sandbox-vercel';
import { Sandbox } from '@vercel/sandbox';

const sandbox = await Sandbox.create({
  runtime: 'node24',
  ports: [4000],
});
const sandboxSession =
  createVercelNetworkSandboxSessionFromNativeSandbox(sandbox);
const portEndpoint = await sandboxSession.getPortEndpoint({
  port: 4000,
  protocol: 'ws',
});
const restrictedSandboxSession =
  createVercelSandboxSessionFromNativeSandbox(sandbox);
const claudeCode = createClaudeCode({ port: 4000, portEndpoint });

const agent = new HarnessAgent({ harness: claudeCode });
const session = await agent.createSession({
  sandboxSession: restrictedSandboxSession,
});

try {
  const result = await agent.stream({
    session,
    prompt: 'Create a short TODO.md for this repository.',
  });

  for await (const part of result.stream) {
    if (part.type === 'text-delta') {
      process.stdout.write(part.text);
    }
  }
} finally {
  await session.destroy();
  await sandboxSession.destroy();
}
```

## Next Steps

- [Tools](/docs/ai-sdk-harnesses/tools) for built-in and host-executed tools.
- [Skills](/docs/ai-sdk-harnesses/skills) for reusable instruction bundles.
- [Harness adapters](/docs/ai-sdk-harnesses/harness-adapters) for adapter-specific
  settings.
- [Workflow utilities](/docs/ai-sdk-harnesses/workflow-utilities) for durable
  long-running turns.
- [UI](/docs/ai-sdk-harnesses/ui) for `useChat` integration.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Overview
description: Learn what AI SDK harnesses are and how they fit into the AI SDK.
url: "https://ai-sdk.dev/docs/ai-sdk-harnesses/overview"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

The AI SDK harness abstraction lets you run established agent harnesses through a
single AI SDK surface. A harness is a complete agent runtime, such as Claude
Code, Codex, or Pi. It owns capabilities that are larger than a model call:
workspace access, built-in coding tools, native session state, compaction,
permission flows, and runtime-specific configuration. Harness adapters can run
the runtime in a sandbox, but their execution environments differ by adapter;
review the adapter's runtime location and security guidance before use.

The AI SDK harness abstraction is separate from the provider/model abstraction.
Providers expose models to AI SDK Core functions such as `generateText` and
`streamText`. Harnesses expose agent runtimes to `HarnessAgent`.

Using a harness does not mean giving up customization. `HarnessAgent` lets you
provide your own instructions, skills, AI SDK tools, permission settings,
sandbox setup hooks, and adapter-specific configuration while preserving the
runtime behavior that makes each harness powerful.

The two abstractions are decoupled, but they use compatible primitives where
possible. Harness output is projected into AI SDK stream and response types, so
surfaces that consume AI SDK model streams can also consume harness streams. For
example, you can pass a `HarnessAgent` stream to `toUIMessageStream` and render
it with `useChat`. The API is designed to feel familiar to AI SDK users, but
harness runtimes have different concepts from lower-level language models.
Configuration follows AI SDK patterns where they fit and diverges where the
harness runtime has different behavior.

Harness packages are **experimental**. Expect breaking changes between
releases as this early API gets further refined.

## When to Use a Harness

Use a harness when you want an existing agent runtime to drive the task:

- Coding agents that can inspect and modify a sandboxed workspace.
- Agent runtimes with their own built-in tools and permission model.
- Multi-turn sessions where the runtime owns conversation history.
- Workflows that should preserve native harness behavior instead of re-creating
  it with a tool loop.

Use providers and models when you want direct control over the model call, the
tool loop, model settings, structured output, or a custom agent architecture.

## Core Concepts

Harnesses have four primary pieces:

- `HarnessAgent`: the AI SDK agent implementation you use in application code.
- Harness adapter: the package that connects to a runtime, such as
  `@ai-sdk/harness-claude-code`.
- Sandbox provider: the isolated filesystem and process environment where the
  harness runs.
- Session: the live conversation and workspace state for a harness run.

## Compatible Streams

`HarnessAgent.generate()` returns an AI SDK `GenerateTextResult`.
`HarnessAgent.stream()` returns an AI SDK `StreamTextResult`.

That means you can consume familiar fields:

- `result.text`
- `result.stream`
- `result.steps`
- `result.usage`
- `result.responseMessages`

Harness-specific events are translated into compatible stream parts. Text,
reasoning, tool calls, tool results, usage, and finish reasons use the same AI
SDK shapes where possible. Events without a first-class AI SDK part, such as
workspace file changes and compaction, are surfaced as dynamic
provider-executed tool parts.

## Sessions Matter

Unlike a language model call, a harness session owns state. The session carries
the harness runtime, sandbox, working directory, native conversation history,
and pending approvals.

Create a session before running turns:

```ts
const session = await agent.createSession();

try {
  const result = await agent.generate({
    session,
    prompt: 'Inspect the repository and summarize the test setup.',
  });

  console.log(result.text);
} finally {
  await session.destroy();
}
```

For server routes, use a stable `sessionId` and persist the resume state
returned by `session.detach()` or `session.stop()`.

## Next Steps

- [HarnessAgent](/docs/ai-sdk-harnesses/harness-agent) for the main API.
- [Skills](/docs/ai-sdk-harnesses/skills) for reusable instruction bundles.
- [Harness adapters](/docs/ai-sdk-harnesses/harness-adapters) for the implemented
  runtimes.
- [Workflow utilities](/docs/ai-sdk-harnesses/workflow-utilities) for durable
  long-running turns.
- [UI](/docs/ai-sdk-harnesses/ui) for `useChat` routes.
- [Terminal UI](/docs/ai-sdk-harnesses/terminal-ui) for `@ai-sdk/tui`.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Skills
description: Use skills with AI SDK harnesses.
url: "https://ai-sdk.dev/docs/ai-sdk-harnesses/skills"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

[Skills](https://agentskills.io/) are reusable instruction bundles that can
be useful for project conventions, workflow guidance, domain-specific procedures,
or any other instructions that should be discoverable by the underlying harness
runtime. You can configure skills for a `HarnessAgent` or replace them between
completed turns.

## Define Skills

Pass skills to `HarnessAgent` with the `skills` setting:

```ts
const agent = new HarnessAgent({
  harness: claudeCode,
  skills: [
    {
      name: 'careful-refactors',
      description: 'Make small, low-risk code changes.',
      content:
        'Prefer minimal diffs. Preserve public APIs. Before editing, read references/checklist.md and follow it.',
      files: [
        {
          path: 'references/checklist.md',
          content:
            '# Refactor checklist\n\n- Identify the smallest useful change.\n- Preserve public APIs.\n- Run the narrowest relevant test.',
        },
      ],
    },
  ],
});
```

Each skill has:

- `name`: stable identifier for the skill.
- `description`: short model-facing summary.
- `content`: full instruction content.
- `files`: optional additional text files bundled with the skill.

Additional files use skill-relative POSIX paths, for example
`reference.md`, `references/codes.md`, or `templates/config.json`. Reference
those paths from `content` when the agent should read them.

## When to Use Skills

Use skills for reusable instructions that should be available on demand, instead of
always being loaded into the agent's context like regular `instructions`.

Use `instructions` for broad agent behavior and current-session priorities.

Skills can be changed between completed turns using `callOptionsSchema` and
`prepareCall`. See [Change Settings Between Turns](/docs/ai-sdk-harnesses/harness-agent#change-settings-between-turns)
for details.

## Related

- [HarnessAgent](/docs/ai-sdk-harnesses/harness-agent)
- [Harness tools](/docs/ai-sdk-harnesses/tools)
- [Harness adapters](/docs/ai-sdk-harnesses/harness-adapters)

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Terminal UI
description: "Use @ai-sdk/tui with HarnessAgent."
url: "https://ai-sdk.dev/docs/ai-sdk-harnesses/terminal-ui"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

`@ai-sdk/tui` can render harness streams, tool calls, reasoning sections, and
approval prompts in a terminal. Because `HarnessAgent` requires a session for
`generate` and `stream` calls, wrap it with a small `AgentTUIAgent` adapter that injects one
session for the lifetime of the terminal UI.

## Installation

```bash
pnpm add @ai-sdk/tui @ai-sdk/harness @ai-sdk/harness-codex @ai-sdk/sandbox-vercel
```

## Example

```ts title='tui.ts'
import { HarnessAgent, type HarnessAgentSession } from '@ai-sdk/harness/agent';
import { codex } from '@ai-sdk/harness-codex';
import { createVercelNetworkSandboxSession } from '@ai-sdk/sandbox-vercel';
import { runAgentTUI, type AgentTUIAgent } from '@ai-sdk/tui';

const agent = new HarnessAgent({
  harness: codex,
});

function createTUIAgent({
  agent,
  session,
}: {
  agent: HarnessAgent<any, any, any>;
  session: HarnessAgentSession;
}): AgentTUIAgent {
  return {
    version: 'agent-v1',
    id: agent.id,
    tools: agent.tools,
    generate(request) {
      return agent.generate({
        ...request,
        session,
      } as Parameters<typeof agent.generate>[0]);
    },
    stream(request) {
      return agent.stream({
        ...request,
        session,
      } as Parameters<typeof agent.stream>[0]);
    },
  } as AgentTUIAgent;
}

const sandboxSession = await createVercelNetworkSandboxSession({
  runtime: 'node24',
  ports: [4000],
  template: await agent.getSandboxTemplate(),
});
const session = await agent.createSession({ sandboxSession });

try {
  await runAgentTUI({
    title: 'Codex',
    agent: createTUIAgent({ agent, session }),
    tools: 'auto-collapsed',
    reasoning: 'collapsed',
  });
} finally {
  await session.destroy();
  await sandboxSession.destroy();
}
```

The terminal UI runs until the user exits with `Esc` or `Ctrl+C`.

Use one session per terminal run. For long-lived terminal tools, persist the
state from `session.detach()` or `session.stop()` if you need to resume later.

## Related

- [Terminal UI](/docs/agents/terminal-ui)
- [runAgentTUI API Reference](/docs/reference/ai-sdk-tui/run-agent-tui)
- [HarnessAgent](/docs/ai-sdk-harnesses/harness-agent)

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Tools
description: Use tools with AI SDK harnesses.
url: "https://ai-sdk.dev/docs/ai-sdk-harnesses/tools"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Harnesses have three tool surfaces:

- Built-in tools exposed by the underlying harness runtime, such as file reads,
  edits, shell commands, and web search.
- AI SDK tools that you pass to `HarnessAgent` with the `tools` setting.
- External MCP tools configured through the harness adapter's `mcpServers`
  setting.

This page covers harness-specific behavior. For general AI SDK tool concepts,
schemas, tool results, and `tool()` usage, see [Tools](/docs/foundations/tools).

## Built-in Tools

Each adapter declares the built-in tools its runtime can call natively.
`HarnessAgent` merges those built-ins with your host-defined tools and exposes
the combined tool set through `agent.tools`.

```ts
const agent = new HarnessAgent({
  harness: claudeCode,
});

agent.tools.bash;
agent.tools.read;
agent.tools.write;
```

Built-in calls are executed by the harness runtime, not by your application
process. Stream parts use `providerExecuted: true` when the runtime already
performed the call.

Adapters use common names where possible:

- `read`
- `write`
- `edit`
- `bash`
- `grep`
- `glob`
- `webSearch`

Some runtimes also expose native tools without a common cross-harness name.
Those appear under their native names.

## Host-Executed Tools

Pass AI SDK tools to `HarnessAgent` the same way you do for a `ToolLoopAgent`:

```ts
import { HarnessAgent } from '@ai-sdk/harness/agent';
import { claudeCode } from '@ai-sdk/harness-claude-code';
import { tool } from 'ai';
import { z } from 'zod';

const weather = tool({
  description: 'Get the current temperature for a city.',
  inputSchema: z.object({
    city: z.string(),
  }),
  execute: async ({ city }) => {
    const temperatures: Record<string, number> = {
      Paris: 12,
      Tokyo: 18,
      Reykjavik: 3,
    };

    return { city, celsius: temperatures[city] ?? 20 };
  },
});

const agent = new HarnessAgent({
  harness: claudeCode,
  tools: { weather },
});
```

When the harness calls `weather`, `HarnessAgent` executes the tool in your host
process, then submits the result back to the harness runtime.

Host-executed tools can declare a `contextSchema` and receive turn-scoped
context through `toolsContext`:

```ts
const lookupAccount = tool({
  inputSchema: z.object({}),
  contextSchema: z.object({ userId: z.string() }),
  execute: async (_, { context }) => {
    return loadAccount(context.userId);
  },
});

const agent = new HarnessAgent({
  harness: claudeCode,
  tools: { lookupAccount },
  toolsContext: {
    lookupAccount: { userId: 'user-123' },
  },
});
```

Use `prepareCall` to replace `toolsContext` when the value depends on custom
call options. The type of `toolsContext` follows the tool set: it is rejected
when no tool declares a context schema, required when a tool requires a
context object, and optional when every context object is optional.
`HarnessAgent` validates each entry against the tool's `contextSchema` before
execution and exposes the configured map in step results and lifecycle
callbacks.

Tool context remains host-only and is not serialized into suspended-turn state.
When recreating a session for an unfinished turn, rebind it explicitly:

```ts
const session = await agent.createSession({
  sessionId,
  continueFrom,
  sandboxSession,
  toolsContext: {
    lookupAccount: { userId: 'user-123' },
  },
});
```

Missing or invalid context fails validation before the host tool executes. The
validation error returned to the harness runtime is intentionally generic so
host-only context values and schema details are not disclosed to the model.

## Client-Side Tools

Omit `execute` when a browser, user interaction, or another external process
provides the tool result:

```ts
const weather = tool({
  description: 'Get the current temperature for a city.',
  inputSchema: z.object({ city: z.string() }),
});
```

When the harness calls a tool without `execute`, the returned result slice ends
after the tool-call step while the underlying turn waits for a result.
`session.hasUnfinishedTurn()` remains `true`, and `session.suspendTurn()`
includes the pending tool call in its serializable continuation state.

In UI flows, pass the model messages produced after `addToolOutput` to the next
`stream()` or `generate()` call. `HarnessAgent` extracts the trailing tool
result and continues the paused turn automatically.

For direct agent calls, provide the raw result to `continueStream()` or
`continueGenerate()`:

```ts
const continued = await agent.continueStream({
  session,
  toolResultContinuations: [
    {
      toolCallId,
      output: { city: 'Paris', celsius: 12 },
    },
  ],
});
```

Set `isError: true` when the external tool failed. To continue in another
process, call `suspendTurn()`, recreate the session with `continueFrom`, and
then pass the result to `continueStream()` or `continueGenerate()`.

## Tool Filtering

Use `activeTools` or `inactiveTools` on `HarnessAgent` to control which tools the
harness can call. Both settings accept tool names from the combined tool set:
the built-in tools declared by the harness adapter and the AI SDK tools passed
with `tools`.

`activeTools` is an allowlist:

```ts
const agent = new HarnessAgent({
  harness: claudeCode,
  tools: { weather },
  activeTools: ['weather'],
});
```

`inactiveTools` is a denylist:

```ts
const agent = new HarnessAgent({
  harness: claudeCode,
  tools: { weather },
  inactiveTools: ['bash', 'write'],
});
```

Pass either `activeTools` or `inactiveTools`, not both. The TypeScript settings
type prevents combining them, and `HarnessAgent` also throws at runtime when
both are specified.

For host-executed tools, inactive tools are not passed to the underlying
harness runtime. If the runtime still attempts to call one, `HarnessAgent`
returns an execution-denied tool result.

For built-in tools, support depends on the harness adapter. Some adapters can
filter built-ins natively. Others enforce filtering through their built-in tool
approval mechanism by denying inactive built-in calls before they execute,
without emitting approval request or response stream parts. Adapters that
support neither mechanism throw when you filter built-in tools.

## Sandbox in Tool Execution

Host-executed tools receive the session sandbox through the same
`experimental_sandbox` execution option used by AI SDK tools elsewhere. The
value is a restricted sandbox session, so tools can read, write, and run
commands without being able to stop the network sandbox or change its network
policy.

```ts
const inspectFile = tool({
  description: 'Read a file from the harness workspace.',
  inputSchema: z.object({
    path: z.string(),
  }),
  execute: async ({ path }, { experimental_sandbox }) => {
    return {
      content: await experimental_sandbox?.readTextFile({ path }),
    };
  },
});
```

## Tool Approvals

Harnesses distinguish built-in tool permissions from host-executed tool
approvals.

Use `permissionMode` for adapter-native built-ins:

```ts
const agent = new HarnessAgent({
  harness: pi,
  permissionMode: 'allow-edits',
});
```

Available values are:

- `allow-all`: allow built-in reads, edits, and shell commands. This is the
  default.
- `allow-edits`: allow reads and edits, but request approval for shell commands
  when the adapter supports built-in approvals.
- `allow-reads`: allow reads, but request approval for edits and shell commands
  when the adapter supports built-in approvals.

Use `toolApproval` for host-executed tools:

```ts
const agent = new HarnessAgent({
  harness: claudeCode,
  tools: { weather },
  toolApproval: {
    weather: 'user-approval',
  },
});
```

`toolApproval` accepts the same status values as AI SDK tool approval status
objects: `not-applicable`, `approved`, `user-approval`, and `denied`.

When approval is required, the stream pauses after a `tool-approval-request`.
Continue the same session by sending a tool approval response message. In UI
flows, `useChat` sends those messages for you when you add the approval result.
In direct agent code, pass the approval response as `messages` on the next
`stream()` or `generate()` call.

## Built-in Approval Support

Most adapters can pause built-in tool calls for approval. Adapters that do not
support it will error if an unsupported tool approval mode is specified.

Host-executed tool approvals are handled by `HarnessAgent`, so they work across
adapters.

## File Changes and Compaction

Some harness events are not ordinary tool calls. For UI compatibility,
`HarnessAgent` projects them as dynamic, provider-executed tool parts:

- `fileChange`: emitted for opaque workspace file mutations.
- `compaction`: emitted when the runtime compacts context.

Check `part.dynamic` before assuming a tool part belongs to your typed tool set.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: UI
description: Use AI SDK harnesses with useChat.
url: "https://ai-sdk.dev/docs/ai-sdk-harnesses/ui"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Harness streams are compatible with AI SDK UI message streams. You can use
`useChat()` on the client and stream `HarnessAgent` output from a server route.

The important difference from model-based chat routes is session management.
A harness owns its conversation state, so the route should resume or create a
`HarnessAgentSession` for the chat id instead of replaying the whole UI message
history into a model.

## Client

```tsx title='app/page.tsx'
'use client';

import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';
import { useState } from 'react';

export default function Page() {
  const [input, setInput] = useState('');
  const { error, messages, sendMessage, status } = useChat({
    id: 'example-chat',
    transport: new DefaultChatTransport({
      api: '/api/chat',
    }),
  });

  return (
    <>
      {messages.map(message => (
        <div key={message.id}>
          <strong>{message.role === 'user' ? 'You: ' : 'AI: '}</strong>
          {message.parts.map((part, index) => {
            if (part.type === 'text') {
              return <span key={index}>{part.text}</span>;
            }

            if (part.type.startsWith('tool-') || part.type === 'dynamic-tool') {
              return <pre key={index}>{JSON.stringify(part, null, 2)}</pre>;
            }

            return null;
          })}
        </div>
      ))}

      {error && <div>{error.message}</div>}

      <form
        onSubmit={event => {
          event.preventDefault();
          if (input.trim()) {
            sendMessage({ text: input });
            setInput('');
          }
        }}
      >
        <input
          value={input}
          onChange={event => setInput(event.target.value)}
          disabled={status !== 'ready'}
        />
        <button type="submit" disabled={status !== 'ready'}>
          Send
        </button>
      </form>
    </>
  );
}
```

## Agent

Define the `HarnessAgent` on the server:

```ts title='app/api/chat/agent.ts'
import { HarnessAgent } from '@ai-sdk/harness/agent';
import { claudeCode } from '@ai-sdk/harness-claude-code';

export const agent = new HarnessAgent({
  harness: claudeCode,
  instructions: 'You are a helpful coding assistant.',
});
```

## Session Store

Persist the opaque resume state returned by `session.detach()`. If the
turn paused for approval or was otherwise interrupted, that resume state carries
the continuation state internally. The chat ID can also be the harness
`sessionId`. Here we derive a stable `sandboxId` from that known chat id. If
you do not derive one, persist the returned `sandboxSession.id` separately.

```ts title='app/api/chat/session-store.ts'
import type { HarnessAgentResumeSessionState } from '@ai-sdk/harness/agent';
import {
  createVercelNetworkSandboxSession,
  resumeVercelNetworkSandboxSession,
} from '@ai-sdk/sandbox-vercel';

const states: Record<string, HarnessAgentResumeSessionState | undefined> = {};

export async function resumeOrCreateSession({
  agent,
  chatId,
}: {
  agent: typeof import('./agent').agent;
  chatId: string;
}) {
  const resumeFrom = states[chatId];
  const sandboxId = `harness-${chatId}`;
  const sandboxSession = resumeFrom
    ? await resumeVercelNetworkSandboxSession({ sandboxId })
    : await createVercelNetworkSandboxSession({
        sandboxId,
        runtime: 'node24',
        ports: [4000],
        template: await agent.getSandboxTemplate(),
      });
  const session = await agent.createSession({
    sessionId: chatId,
    resumeFrom,
    sandboxSession,
  });
  return { session, sandboxSession };
}

export async function detachAndPersist({
  chatId,
  session,
}: {
  chatId: string;
  session: Awaited<ReturnType<typeof resumeOrCreateSession>>['session'];
}) {
  states[chatId] = await session.detach();
}
```

Use durable storage instead of an in-memory map in production. The creator
always creates a new sandbox and fails on a conflicting `sandboxId`; only the
resume function reattaches an existing one.

## Route

Convert UI messages to model messages, run the harness turn, and convert the
result stream back to a UI message stream:

```ts title='app/api/chat/route.ts'
import { agent } from './agent';
import { detachAndPersist, resumeOrCreateSession } from './session-store';
import { getHarnessErrorMessage } from '@ai-sdk/harness/agent';
import {
  convertToModelMessages,
  createUIMessageStream,
  createUIMessageStreamResponse,
  toUIMessageStream,
  type UIMessage,
} from 'ai';

export async function POST(request: Request) {
  const body: {
    id?: string;
    messages: UIMessage[];
  } = await request.json();

  if (!body.id) {
    return new Response('Missing chat ID', { status: 400 });
  }

  const chatId = body.id;
  const messages = await convertToModelMessages(body.messages);

  return createUIMessageStreamResponse({
    stream: createUIMessageStream({
      execute: async ({ writer }) => {
        const { session } = await resumeOrCreateSession({ agent, chatId });
        const result = await agent.stream({ session, messages });

        writer.merge(
          toUIMessageStream({
            stream: result.stream,
            onError: getHarnessErrorMessage,
            onEnd: async () => {
              await detachAndPersist({ chatId, session });
            },
          }),
        );
      },
      onError: getHarnessErrorMessage,
    }),
  });
}
```

Creating the UI message stream before acquiring the session ensures sandbox,
bootstrap, and harness startup failures are sent as UI error parts instead of
becoming generic HTTP errors. `getHarnessErrorMessage` preserves reviewed,
client-safe harness messages and masks unknown server errors.

Do not use `createAgentUIStreamResponse` directly with `HarnessAgent` unless you
wrap the agent to inject the required session. `HarnessAgent.stream()` requires
`session` on every call.

## Detach or Stop

Use `session.detach()` when you want to park the harness runtime and keep the
sandbox warm for the next request. Bridge-backed adapters can usually reattach
or replay efficiently. If the turn is unfinished, `detach()` includes the turn
continuation state in the returned resume state.

`session.stop()` saves resume state and stops the harness runtime, but does not
stop a supplied sandbox. The caller owns that network session and should call
its `destroy()` method when the chat ends, not while a later request needs to
reattach. Preserve the chosen `sandboxId` across processes.

## Rendering Harness Parts

Harness output contains the same UI message part shapes used by AI SDK model
streams:

- `text` and `reasoning` parts for generated content.
- typed tool parts such as `tool-bash`, `tool-read`, or a host tool like
  `tool-weather`.
- `dynamic-tool` parts for dynamic events such as `fileChange` and
  `compaction`.

Render typed harness built-ins the same way you render normal AI SDK tool
parts. Check `part.state` for `input-streaming`, `input-available`, and
`output-available`.

## Type-Safe Tool Parts

Until `HarnessAgent` session options are part of the base `Agent` call
parameters, infer UI tools from `agent.tools`:

```ts
import type { InferUITools, UIMessage } from 'ai';
import { agent } from './agent';

export type HarnessMessage = UIMessage<
  unknown,
  never,
  InferUITools<typeof agent.tools>
>;
```

Then use `useChat<HarnessMessage>()` on the client.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Workflow Utilities
description: Run HarnessAgent turns as durable Workflow DevKit workflows.
url: "https://ai-sdk.dev/docs/ai-sdk-harnesses/workflow-utilities"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

`@ai-sdk/workflow-harness` provides helpers for running `HarnessAgent` turns
inside a [workflow](https://vercel.com/docs/workflow).

The package provides a serializable state machine and runners for time-sliced
and semantic agent step turns. You call the appropriate runner from your own
`'use workflow'` and `'use step'` functions.

The core harness and workflow files are framework-independent. The HTTP handlers
shown below use Next.js as one example; adapt them to your runtime and Workflow
SDK integration. If you use Next.js, ensure you have
[configured your project for Workflow](https://workflow-sdk.dev/docs/getting-started/next)
before following the examples.

## Installation

```bash
pnpm add @ai-sdk/workflow-harness workflow
```

In addition to the workflow specific packages, install the core harness package,
a harness adapter, and a sandbox adapter as shown in
[HarnessAgent](/docs/ai-sdk-harnesses/harness-agent).

## Configuring the Harness Agent

The agent can be configured in the usual way, for the most part. When using
semantic agent steps, set `stopWhen` to `isStepCount(1)` so one call to
`stream()` completes one agent step. Omit it when using time slices.

```ts title='harness-workflow/agent.ts' {13-17}
import { HarnessAgent } from '@ai-sdk/harness/agent';
import { claudeCode } from '@ai-sdk/harness-claude-code';
import { isStepCount } from 'ai';

export const agent = new HarnessAgent({
  harness: claudeCode,
  instructions: 'You are a helpful coding assistant.',
  /*
   * Only needed for semantic agent-step workflows.
   * Omit this for time-sliced workflows.
   */
  stopWhen: isStepCount(1),
});
```

## Using Semantic Agent Steps

Semantic agent steps persist the harness turn after each agent step. Configure
the shared agent with the highlighted `stopWhen` option shown above, then call
`runHarnessAgentStep()` from a Workflow step.

### Defining the Agent Step

Keep the Workflow step in its own module and import the agent dynamically inside
the step body. This keeps the agent, sandbox adapter, and other Node.js
dependencies out of the workflow bundle.

```ts title='harness-workflow/agent-step.ts'
import {
  runHarnessAgentStep,
  type HarnessWorkflowState,
} from '@ai-sdk/workflow-harness';

export async function agentStep(
  state: HarnessWorkflowState,
): Promise<HarnessWorkflowState> {
  'use step';

  const { agent } = await import('./agent');
  const {
    createVercelNetworkSandboxSession,
    resumeVercelNetworkSandboxSession,
  } = await import('@ai-sdk/sandbox-vercel');
  const sandboxId = `harness-${state.sessionId}`;
  const isFirstStep = state.resumeFrom == null && state.continueFrom == null;
  const sandboxSession = isFirstStep
    ? await createVercelNetworkSandboxSession({
        sandboxId,
        runtime: 'node24',
        ports: [4000],
        template: await agent.getSandboxTemplate(),
      })
    : await resumeVercelNetworkSandboxSession({ sandboxId });

  return runHarnessAgentStep({
    agent,
    state,
    sandboxSession,
  });
}
```

### Defining the Semantic Agent Step Workflow

Create the workflow state and keep scheduling `agentStep()` while the agent has
more work.

```ts title='harness-workflow/workflow.ts'
import { agentStep } from './agent-step';
import {
  createHarnessWorkflowState,
  finalizeHarnessWorkflow,
  type HarnessWorkflowInput,
} from '@ai-sdk/workflow-harness';

export async function agentWorkflow(input: {
  messages: NonNullable<HarnessWorkflowInput['messages']>;
  sessionId: string;
}) {
  'use workflow';

  let state = createHarnessWorkflowState(input);

  do {
    state = await agentStep(state);
  } while (state.status === 'ready_for_next_step');

  return finalizeHarnessWorkflow(state);
}
```

This example covers the workflow execution foundation only. For multi-turn
conversations, you must persist the harness session's `resumeFrom` state
between workflow runs. See [Resume Persistence](#resume-persistence).

Each `ready_for_next_step` result carries `continueFrom`, which lets the next
Workflow step continue the same unfinished turn. When the turn finishes,
`finalizeHarnessWorkflow()` returns its result or throws if the workflow failed.

### Starting the Semantic Agent Step Workflow

Start the workflow from server-side code. This Next.js route converts the AI SDK
UI messages, starts `agentWorkflow()`, and returns the workflow's AI SDK UI
message stream. Passing the converted messages lets `HarnessAgent` distinguish a
new user turn from tool approval and tool result continuations.

```ts title='app/api/harness-workflow/route.ts'
import { agentWorkflow } from '../../../harness-workflow/workflow';
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  type UIMessage,
  type UIMessageChunk,
} from 'ai';
import { start } from 'workflow/api';

export async function POST(request: Request) {
  const body: {
    id?: string;
    messages: UIMessage[];
  } = await request.json();

  if (!body.id) {
    return new Response('Missing chat ID', { status: 400 });
  }

  const messages = await convertToModelMessages(body.messages);
  const run = await start(agentWorkflow, [
    {
      messages,
      sessionId: body.id,
    },
  ]);

  return createUIMessageStreamResponse({
    stream: run.readable as ReadableStream<UIMessageChunk>,
  });
}
```

The workflow state `sessionId` identifies the harness session. The step derives
`sandboxId` from it before first creating the sandbox and uses that same ID to
reattach on later steps. The state does not serialize a live sandbox. Keep
`agent.ts`, `agent-step.ts`, `workflow.ts`, and the route in separate modules so
the workflow bundle does not include Node-heavy agent, sandbox, or framework
dependencies.

## Using Time Slices

Time slices persist a long-running harness turn at wall-clock boundaries. Omit
the highlighted `stopWhen` option from the shared agent, then call
`runHarnessAgentTimeSlice()` from a Workflow step. It uses a 750-second budget
by default; pass `timeSliceSeconds` to choose a different budget.

### Defining the Time Slice Step

As with semantic agent steps, keep the Workflow step in its own module and

```ts title='harness-workflow/time-slice-step.ts'
import {
  runHarnessAgentTimeSlice,
  type HarnessWorkflowState,
} from '@ai-sdk/workflow-harness';

export async function timeSliceStep(
  state: HarnessWorkflowState,
): Promise<HarnessWorkflowState> {
  'use step';

  const { agent } = await import('./agent');
  const {
    createVercelNetworkSandboxSession,
    resumeVercelNetworkSandboxSession,
  } = await import('@ai-sdk/sandbox-vercel');
  const sandboxId = `harness-${state.sessionId}`;
  const isFirstStep = state.resumeFrom == null && state.continueFrom == null;
  const sandboxSession = isFirstStep
    ? await createVercelNetworkSandboxSession({
        sandboxId,
        runtime: 'node24',
        ports: [4000],
        template: await agent.getSandboxTemplate(),
      })
    : await resumeVercelNetworkSandboxSession({ sandboxId });

  return runHarnessAgentTimeSlice({
    agent,
    state,
    sandboxSession,
  });
}
```

### Defining the Time-Sliced Workflow

Create the workflow state and keep scheduling `timeSliceStep()` while the agent
has more work.

```ts title='harness-workflow/workflow.ts'
import { timeSliceStep } from './time-slice-step';
import {
  createHarnessWorkflowState,
  finalizeHarnessWorkflow,
  type HarnessWorkflowInput,
} from '@ai-sdk/workflow-harness';

export async function timeSliceWorkflow(input: {
  messages: NonNullable<HarnessWorkflowInput['messages']>;
  sessionId: string;
}) {
  'use workflow';

  let state = createHarnessWorkflowState(input);

  do {
    state = await timeSliceStep(state);
  } while (state.status === 'ready_for_next_step');

  return finalizeHarnessWorkflow(state);
}
```

This example covers the workflow execution foundation only. For multi-turn
conversations, you must persist the harness session's `resumeFrom` state
between workflow runs. See [Resume Persistence](#resume-persistence).

Each `ready_for_next_step` result carries `continueFrom`, which lets the next
Workflow step continue the same unfinished turn. When the turn finishes,
`finalizeHarnessWorkflow()` returns its result or throws if the workflow failed.

### Starting the Time-Sliced Workflow

Start the workflow from server-side code. As with semantic agent steps, pass the
converted messages so `HarnessAgent` can distinguish a new user turn from tool
approval and tool result continuations.

```ts title='app/api/harness-workflow/route.ts'
import { timeSliceWorkflow } from '../../../harness-workflow/workflow';
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  type UIMessage,
  type UIMessageChunk,
} from 'ai';
import { start } from 'workflow/api';

export async function POST(request: Request) {
  const body: {
    id?: string;
    messages: UIMessage[];
  } = await request.json();

  if (!body.id) {
    return new Response('Missing chat ID', { status: 400 });
  }

  const messages = await convertToModelMessages(body.messages);
  const run = await start(timeSliceWorkflow, [
    {
      messages,
      sessionId: body.id,
    },
  ]);

  return createUIMessageStreamResponse({
    stream: run.readable as ReadableStream<UIMessageChunk>,
  });
}
```

If you use Next.js, wrap your config with `withWorkflow()` so the Workflow SDK
transforms `'use workflow'` and `'use step'` modules. Other frameworks have
their own Workflow SDK setup.

```js title='next.config.js'
import { withWorkflow } from 'workflow/next';

const nextConfig = {};

export default withWorkflow(nextConfig);
```

The workflow state `sessionId` identifies the harness session. The step derives
`sandboxId` from it; otherwise, you must persist `sandboxSession.id` separately
to reattach across steps. Keep
`agent.ts`, `time-slice-step.ts`, `workflow.ts`, and the route in separate
modules so the workflow bundle does not include Node-heavy agent, sandbox, or
framework dependencies.

`destroyOnFinish` destroys the harness session, but never the supplied sandbox.
The caller can explicitly call `sandboxSession.destroy()` on the network
session when no later step needs to reattach. Never destroy it at a
time-slice boundary where a subsequent step must resume it.

## Resume Persistence

Workflow automatically persists the `HarnessWorkflowState` returned by each
step, including `continueFrom`, for the duration of the current workflow run.
To continue the native harness session across separate user-turn workflow runs,
persist the opaque `resumeFrom` state by `sessionId`.

The storage implementation is the same for semantic agent steps and time
slices. This example uses Workflow steps because filesystem access must stay out
of the workflow function itself. Use durable storage instead of local files in
production.

```ts title='harness-workflow/resume-store.ts'
import type { HarnessV1ResumeSessionState } from '@ai-sdk/harness';
import { safeParseJSON } from '@ai-sdk/provider-utils';

const RESUME_DIR = '.harness-sessions';

function fileName(sessionId: string): string {
  return `${sessionId.replace(/[^a-zA-Z0-9_-]/g, '_')}.json`;
}

export async function loadResumeStep(
  sessionId: string,
): Promise<HarnessV1ResumeSessionState | undefined> {
  'use step';

  const { readFile } = await import('node:fs/promises');
  const { join } = await import('node:path');

  let text: string;
  try {
    text = await readFile(
      join(process.cwd(), RESUME_DIR, fileName(sessionId)),
      'utf8',
    );
  } catch {
    return undefined;
  }

  const parsed = await safeParseJSON({ text });

  return parsed.success
    ? (parsed.value as unknown as HarnessV1ResumeSessionState)
    : undefined;
}

export async function persistResumeStep({
  sessionId,
  resumeState,
}: {
  sessionId: string;
  resumeState: HarnessV1ResumeSessionState | undefined;
}): Promise<void> {
  'use step';

  if (!resumeState) return;

  const { mkdir, writeFile } = await import('node:fs/promises');
  const { join } = await import('node:path');
  const dir = join(process.cwd(), RESUME_DIR);

  await mkdir(dir, { recursive: true });
  await writeFile(join(dir, fileName(sessionId)), JSON.stringify(resumeState));
}
```

Load the previous `resumeFrom` state before creating the workflow state, then
persist the updated value after the execution loop. The integration points are
the same for both workflow approaches.

### Semantic Agent Step Workflow

```ts title='harness-workflow/workflow.ts'
import { agentStep } from './agent-step';
import { loadResumeStep, persistResumeStep } from './resume-store';
import {
  createHarnessWorkflowState,
  finalizeHarnessWorkflow,
  type HarnessWorkflowInput,
} from '@ai-sdk/workflow-harness';

export async function agentWorkflow(input: {
  messages: NonNullable<HarnessWorkflowInput['messages']>;
  sessionId: string;
}) {
  'use workflow';

  const resumeFrom = await loadResumeStep(input.sessionId);
  let state = createHarnessWorkflowState({ ...input, resumeFrom });

  do {
    state = await agentStep(state);
  } while (state.status === 'ready_for_next_step');

  await persistResumeStep({
    sessionId: state.sessionId,
    resumeState: state.resumeFrom,
  });

  return finalizeHarnessWorkflow(state);
}
```

### Time-Sliced Workflow

```ts title='harness-workflow/workflow.ts'
import { loadResumeStep, persistResumeStep } from './resume-store';
import { timeSliceStep } from './time-slice-step';
import {
  createHarnessWorkflowState,
  finalizeHarnessWorkflow,
  type HarnessWorkflowInput,
} from '@ai-sdk/workflow-harness';

export async function timeSliceWorkflow(input: {
  messages: NonNullable<HarnessWorkflowInput['messages']>;
  sessionId: string;
}) {
  'use workflow';

  const resumeFrom = await loadResumeStep(input.sessionId);
  let state = createHarnessWorkflowState({ ...input, resumeFrom });

  do {
    state = await timeSliceStep(state);
  } while (state.status === 'ready_for_next_step');

  await persistResumeStep({
    sessionId: state.sessionId,
    resumeState: state.resumeFrom,
  });

  return finalizeHarnessWorkflow(state);
}
```

## Related

- [HarnessAgent](/docs/ai-sdk-harnesses/harness-agent)
- [Harness adapters](/docs/ai-sdk-harnesses/harness-adapters)
- [UI](/docs/ai-sdk-harnesses/ui)

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Handling Authentication
description: Learn how to authenticate with the AI SDK.
url: "https://ai-sdk.dev/docs/ai-sdk-rsc/authentication"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK RSC is currently experimental. We recommend using [AI SDK
UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).

The RSC API makes extensive use of [`Server Actions`](https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations) to power streaming values and UI from the server.

Server Actions are exposed as public, unprotected endpoints. As a result, you should treat Server Actions as you would public-facing API endpoints and ensure that the user is authorized to perform the action before returning any data.

```tsx title="app/actions.tsx"
'use server';

import { cookies } from 'next/headers';
import { createStreamableUI } from '@ai-sdk/rsc';
import { validateToken } from '../utils/auth';

export const getWeather = async () => {
  const token = cookies().get('token');

  if (!token || !validateToken(token)) {
    return {
      error: 'This action requires authentication',
    };
  }
  const streamableDisplay = createStreamableUI(null);

  streamableDisplay.update(<Skeleton />);
  streamableDisplay.done(<Weather />);

  return {
    display: streamableDisplay.value,
  };
};
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Error Handling
description: Learn how to handle errors with the AI SDK.
url: "https://ai-sdk.dev/docs/ai-sdk-rsc/error-handling"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK RSC is currently experimental. We recommend using [AI SDK
UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).

Two categories of errors can occur when working with the RSC API: errors while streaming user interfaces and errors while streaming other values.

## Handling UI Errors

To handle errors while generating UI, the [`streamableUI`](/docs/reference/ai-sdk-rsc/create-streamable-ui) object exposes an `error()` method.

```tsx title='app/actions.tsx'
'use server';

import { createStreamableUI } from '@ai-sdk/rsc';

export async function getStreamedUI() {
  const ui = createStreamableUI();

  (async () => {
    ui.update(<div>loading</div>);
    const data = await fetchData();
    ui.done(<div>{data}</div>);
  })().catch(e => {
    ui.error(<div>Error: {e.message}</div>);
  });

  return ui.value;
}
```

With this method, you can catch any error with the stream, and return relevant UI. On the client, you can also use a [React Error Boundary](https://react.dev/reference/react/Component#catching-rendering-errors-with-an-error-boundary) to wrap the streamed component and catch any additional errors.

```tsx title='app/page.tsx'
'use client';

import { getStreamedUI } from '@/actions';
import { useState } from 'react';
import { ErrorBoundary } from './ErrorBoundary';

export default function Page() {
  const [streamedUI, setStreamedUI] = useState(null);

  return (
    <div>
      <button
        onClick={async () => {
          const newUI = await getStreamedUI();
          setStreamedUI(newUI);
        }}
      >
        What does the new UI look like?
      </button>
      <ErrorBoundary>{streamedUI}</ErrorBoundary>
    </div>
  );
}
```

## Handling Other Errors

To handle other errors while streaming, you can return an error object that the receiver can use to determine why the failure occurred.

```tsx title='app/actions.tsx'
'use server';

import { createStreamableValue } from '@ai-sdk/rsc';
import { fetchData, emptyData } from '../utils/data';

export const getStreamedData = async () => {
  const streamableData = createStreamableValue<string>(emptyData);

  (async () => {
    const data1 = await fetchData();
    streamableData.update(data1);

    const data2 = await fetchData();
    streamableData.update(data2);

    const data3 = await fetchData();
    streamableData.done(data3);
  })().catch(e => {
    streamableData.error(e);
  });

  return { data: streamableData.value };
};
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Managing Generative UI State
description: Overview of the AI and UI states
url: "https://ai-sdk.dev/docs/ai-sdk-rsc/generative-ui-state"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK RSC is currently experimental. We recommend using [AI SDK
UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).

State is an essential part of any application. State is particularly important in AI applications as it is passed to large language models (LLMs) on each request to ensure they have the necessary context to produce a great generation. Traditional chatbots are text-based and have a structure that mirrors that of any chat application.

For example, in a chatbot, state is an array of `messages` where each `message` has:

- `id`: a unique identifier
- `role`: who sent the message (user/assistant/system/tool)
- `content`: the content of the message

This state can be rendered in the UI and sent to the model without any modifications.

With Generative UI, the model can now return a React component, rather than a plain text message. The client can render that component without issue, but that state can't be sent back to the model because React components aren't serialisable. So, what can you do?

**The solution is to split the state in two, where one (AI State) becomes a proxy for the other (UI State)**.

One way to understand this concept is through a Lego analogy. Imagine a 10,000 piece Lego model that, once built, cannot be easily transported because it is fragile. By taking the model apart, it can be easily transported, and then rebuilt following the steps outlined in the instructions pamphlet. In this way, the instructions pamphlet is a proxy to the physical structure. Similarly, AI State provides a serialisable (JSON) representation of your UI that can be passed back and forth to the model.

## What is AI and UI State?

The RSC API simplifies how you manage AI State and UI State, providing a robust way to keep them in sync between your database, server and client.

### AI State

AI State refers to the state of your application in a serialisable format that will be used on the server and can be shared with the language model.

For a chat app, the AI State is the conversation history (messages) between the user and the assistant. Components generated by the model would be represented in a JSON format as a tool alongside any necessary props. AI State can also be used to store other values and meta information such as `createdAt` for each message and `chatId` for each conversation. The LLM reads this history so it can generate the next message. This state serves as the source of truth for the current application state.

**Note**: AI state can be accessed/modified from both the server and the
client.

### UI State

UI State refers to the state of your application that is rendered on the client. It is a fully client-side state (similar to `useState`) that can store anything from JavaScript values to React elements. UI state is a list of actual UI elements that are rendered on the client.

**Note**

: UI State can only be accessed client-side.

## Using AI / UI State

### Creating the AI Context

AI SDK RSC simplifies managing AI and UI state across your application by providing several hooks. These hooks are powered by [ React context ](https://react.dev/reference/react/hooks#context-hooks) under the hood.

Notably, this means you do not have to pass the message history to the server explicitly for each request. You also can access and update your application state in any child component of the context provider. As you begin building [multistep generative interfaces](/docs/ai-sdk-rsc/multistep-interfaces), this will be particularly helpful.

To use `@ai-sdk/rsc` to manage AI and UI State in your application, you can create a React context using [`createAI`](/docs/reference/ai-sdk-rsc/create-ai):

```tsx title='app/actions.tsx'
// Define the AI state and UI state types
export type ServerMessage = {
  role: 'user' | 'assistant';
  content: string;
};

export type ClientMessage = {
  id: string;
  role: 'user' | 'assistant';
  display: ReactNode;
};

export const sendMessage = async (input: string): Promise<ClientMessage> => {
  "use server"
  ...
}
```

```tsx title='app/ai.ts'
import { createAI } from '@ai-sdk/rsc';
import { ClientMessage, ServerMessage, sendMessage } from './actions';

export type AIState = ServerMessage[];
export type UIState = ClientMessage[];

// Create the AI provider with the initial states and allowed actions
export const AI = createAI<AIState, UIState>({
  initialAIState: [],
  initialUIState: [],
  actions: {
    sendMessage,
  },
});
```

You must pass Server Actions to the

`actions`

object.

In this example, you define types for AI State and UI State, respectively.

Next, wrap your application with your newly created context. With that, you can get and set AI and UI State across your entire application.

```tsx title='app/layout.tsx'
import { type ReactNode } from 'react';
import { AI } from './ai';

export default function RootLayout({
  children,
}: Readonly<{ children: ReactNode }>) {
  return (
    <AI>
      <html lang="en">
        <body>{children}</body>
      </html>
    </AI>
  );
}
```

## Reading UI State in Client

The UI state can be accessed in Client Components using the [`useUIState`](/docs/reference/ai-sdk-rsc/use-ui-state) hook provided by the RSC API. The hook returns the current UI state and a function to update the UI state like React's `useState`.

```tsx title='app/page.tsx'
'use client';

import { useUIState } from '@ai-sdk/rsc';

export default function Page() {
  const [messages, setMessages] = useUIState();

  return (
    <ul>
      {messages.map(message => (
        <li key={message.id}>{message.display}</li>
      ))}
    </ul>
  );
}
```

## Reading AI State in Client

The AI state can be accessed in Client Components using the [`useAIState`](/docs/reference/ai-sdk-rsc/use-ai-state) hook provided by the RSC API. The hook returns the current AI state.

```tsx title='app/page.tsx'
'use client';

import { useAIState } from '@ai-sdk/rsc';

export default function Page() {
  const [messages, setMessages] = useAIState();

  return (
    <ul>
      {messages.map(message => (
        <li key={message.id}>{message.content}</li>
      ))}
    </ul>
  );
}
```

## Reading AI State on Server

The AI State can be accessed within any Server Action provided to the `createAI` context using the [`getAIState`](/docs/reference/ai-sdk-rsc/get-ai-state) function. It returns the current AI state as a read-only value:

```tsx title='app/actions.ts'
import { getAIState } from '@ai-sdk/rsc';

export async function sendMessage(message: string) {
  'use server';

  const history = getAIState();

  const response = await generateText({
    model: "anthropic/claude-sonnet-5.5",
    messages: [...history, { role: 'user', content: message }],
  });

  return response;
}
```

Remember, you can only access state within actions that have been passed to
the `createAI` context within the `actions` key.

## Updating AI State on Server

The AI State can also be updated from within your Server Action with the [`getMutableAIState`](/docs/reference/ai-sdk-rsc/get-mutable-ai-state) function. This function is similar to `getAIState`, but it returns the state with methods to read and update it:

```tsx title='app/actions.ts'
import { getMutableAIState } from '@ai-sdk/rsc';

export async function sendMessage(message: string) {
  'use server';

  const history = getMutableAIState();

  // Update the AI state with the new user message.
  history.update([...history.get(), { role: 'user', content: message }]);

  const response = await generateText({
    model: "anthropic/claude-sonnet-5.5",
    messages: history.get(),
  });

  // Update the AI state again with the response from the model.
  history.done([...history.get(), { role: 'assistant', content: response }]);

  return response;
}
```

It is important to update the AI State with new responses using `.update()`
and `.done()` to keep the conversation history in sync.

## Calling Server Actions from the Client

To call the `sendMessage` action from the client, you can use the [`useActions`](/docs/reference/ai-sdk-rsc/use-actions) hook. The hook returns all the available Actions that were provided to `createAI`:

```tsx title='app/page.tsx'
'use client';

import { useActions, useUIState } from '@ai-sdk/rsc';
import { AI } from './ai';

export default function Page() {
  const { sendMessage } = useActions<typeof AI>();
  const [messages, setMessages] = useUIState();

  const handleSubmit = async event => {
    event.preventDefault();

    setMessages(currentMessages => [
      ...currentMessages,
      {
        id: Date.now().toString(),
        role: 'user',
        display: event.target.message.value,
      },
    ]);

    const response = await sendMessage(event.target.message.value);

    setMessages(currentMessages => [...currentMessages, response]);
  };

  return (
    <>
      <ul>
        {messages.map(message => (
          <li key={message.id}>{message.display}</li>
        ))}
      </ul>
      <form onSubmit={handleSubmit}>
        <input type="text" name="message" />
        <button type="submit">Send</button>
      </form>
    </>
  );
}
```

When the user submits a message, the `sendMessage` action is called with the message content. The response from the action is then added to the UI state, updating the displayed messages.

Important! Don't forget to update the UI State after you call your Server
Action otherwise the streamed component will not show in the UI.

To learn more, check out this [example](/examples/next-app/state-management/ai-ui-states) on managing AI and UI state using `@ai-sdk/rsc`.

***

Next, you will learn how you can save and restore state with `@ai-sdk/rsc`.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Handling Loading State
description: Overview of handling loading state with AI SDK RSC
url: "https://ai-sdk.dev/docs/ai-sdk-rsc/loading-state"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK RSC is currently experimental. We recommend using [AI SDK
UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).

Given that responses from language models can often take a while to complete, it's crucial to be able to show loading state to users. This provides visual feedback that the system is working on their request and helps maintain a positive user experience.

There are three approaches you can take to handle loading state with the AI SDK RSC:

- Managing loading state similar to how you would in a traditional Next.js application. This involves setting a loading state variable in the client and updating it when the response is received.
- Streaming loading state from the server to the client. This approach allows you to track loading state on a more granular level and provide more detailed feedback to the user.
- Streaming loading component from the server to the client. This approach allows you to stream a React Server Component to the client while awaiting the model's response.

## Handling Loading State on the Client

### Client

Let's create a simple Next.js page that will call the `generateResponse` function when the form is submitted. The function will take in the user's prompt (`input`) and then generate a response (`response`). To handle the loading state, use the `loading` state variable. When the form is submitted, set `loading` to `true`, and when the response is received, set it back to `false`. While the response is being streamed, the input field will be disabled.

```tsx title='app/page.tsx'
'use client';

import { useState } from 'react';
import { generateResponse } from './actions';
import { readStreamableValue } from '@ai-sdk/rsc';

// Force the page to be dynamic and allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [generation, setGeneration] = useState<string>('');
  const [loading, setLoading] = useState<boolean>(false);

  return (
    <div>
      <div>{generation}</div>
      <form
        onSubmit={async e => {
          e.preventDefault();
          setLoading(true);
          const response = await generateResponse(input);

          let textContent = '';

          for await (const delta of readStreamableValue(response)) {
            textContent = `${textContent}${delta}`;
            setGeneration(textContent);
          }
          setInput('');
          setLoading(false);
        }}
      >
        <input
          type="text"
          value={input}
          disabled={loading}
          className="disabled:opacity-50"
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button>Send Message</button>
      </form>
    </div>
  );
}
```

### Server

Now let's implement the `generateResponse` function. Use the `streamText` function to generate a response to the input.

```typescript title='app/actions.ts'
'use server';

import { streamText } from 'ai';
import { createStreamableValue } from '@ai-sdk/rsc';

export async function generateResponse(prompt: string) {
  const stream = createStreamableValue();

  (async () => {
    const { textStream } = streamText({
      model: "anthropic/claude-sonnet-5.5",
      prompt,
    });

    for await (const text of textStream) {
      stream.update(text);
    }

    stream.done();
  })();

  return stream.value;
}
```

## Streaming Loading State from the Server

If you are looking to track loading state on a more granular level, you can create a new streamable value to store a custom variable and then read this on the frontend. Let's update the example to create a new streamable value for tracking loading state:

### Server

```typescript title='app/actions.ts' {9,22,25}
'use server';

import { streamText } from 'ai';
import { createStreamableValue } from '@ai-sdk/rsc';

export async function generateResponse(prompt: string) {
  const stream = createStreamableValue();
  const loadingState = createStreamableValue({ loading: true });

  (async () => {
    const { textStream } = streamText({
      model: "anthropic/claude-sonnet-5.5",
      prompt,
    });

    for await (const text of textStream) {
      stream.update(text);
    }

    stream.done();
    loadingState.done({ loading: false });
  })();

  return { response: stream.value, loadingState: loadingState.value };
}
```

### Client

```tsx title='app/page.tsx' {22,34-42}
'use client';

import { useState } from 'react';
import { generateResponse } from './actions';
import { readStreamableValue } from '@ai-sdk/rsc';

// Force the page to be dynamic and allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [generation, setGeneration] = useState<string>('');
  const [loading, setLoading] = useState<boolean>(false);

  return (
    <div>
      <div>{generation}</div>
      <form
        onSubmit={async e => {
          e.preventDefault();
          setLoading(true);
          const { response, loadingState } = await generateResponse(input);

          let textContent = '';

          // read both streamable values concurrently
          await Promise.all([
            (async () => {
              for await (const responseDelta of readStreamableValue(response)) {
                textContent = `${textContent}${responseDelta}`;
                setGeneration(textContent);
              }
            })(),
            (async () => {
              for await (const loadingDelta of readStreamableValue(
                loadingState,
              )) {
                if (loadingDelta) {
                  setLoading(loadingDelta.loading);
                }
              }
            })(),
          ]);
          setInput('');
          setLoading(false);
        }}
      >
        <input
          type="text"
          value={input}
          disabled={loading}
          className="disabled:opacity-50"
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button>Send Message</button>
      </form>
    </div>
  );
}
```

This allows you to provide more detailed feedback about the generation process to your users.

## Streaming Loading Components with `streamUI`

If you are using the [ `streamUI` ](/docs/reference/ai-sdk-rsc/stream-ui) function, you can stream the loading state to the client in the form of a React component. `streamUI` supports the usage of [ JavaScript generator functions ](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/function*), which allow you to yield some value (in this case a React component) while some other blocking work completes.

## Server

```ts
'use server';

import { openai } from '@ai-sdk/openai';
import { streamUI } from '@ai-sdk/rsc';

export async function generateResponse(prompt: string) {
  const result = await streamUI({
    model: openai('gpt-6-astra'),
    prompt,
    text: async function* ({ content }) {
      yield <div>loading...</div>;
      return <div>{content}</div>;
    },
  });

  return result.value;
}
```

Remember to update the file from `.ts` to `.tsx` because you are defining a
React component in the `streamUI` function.

## Client

```tsx
'use client';

import { useState } from 'react';
import { generateResponse } from './actions';
import { readStreamableValue } from '@ai-sdk/rsc';

// Force the page to be dynamic and allow streaming responses up to 30 seconds
export const maxDuration = 30;

export default function Home() {
  const [input, setInput] = useState<string>('');
  const [generation, setGeneration] = useState<React.ReactNode>();

  return (
    <div>
      <div>{generation}</div>
      <form
        onSubmit={async e => {
          e.preventDefault();
          const result = await generateResponse(input);
          setGeneration(result);
          setInput('');
        }}
      >
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button>Send Message</button>
      </form>
    </div>
  );
}
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Migrating from RSC to UI
description: Learn how to migrate from AI SDK RSC to AI SDK UI.
url: "https://ai-sdk.dev/docs/ai-sdk-rsc/migrating-to-ui"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

This guide helps you migrate from AI SDK RSC to AI SDK UI.

## Background

The AI SDK has two packages that help you build the frontend for your applications – [AI SDK UI](/docs/ai-sdk-ui) and [AI SDK RSC](/docs/ai-sdk-rsc).

We introduced support for using [React Server Components](https://react.dev/reference/rsc/server-components) (RSC) within the AI SDK to simplify building generative user interfaces for frameworks that support RSC.

However, given we're pushing the boundaries of this technology, AI SDK RSC currently faces significant limitations that make it unsuitable for stable production use.

- It is not possible to abort a stream using server actions. This will be improved in future releases of React and Next.js [(1122)](https://github.com/vercel/ai/issues/1122).
- When using `createStreamableUI` and `streamUI`, components remount on `.done()`, causing them to flicker [(2939)](https://github.com/vercel/ai/issues/2939).
- Many suspense boundaries can lead to crashes [(2843)](https://github.com/vercel/ai/issues/2843).
- Using `createStreamableUI` can lead to quadratic data transfer. You can avoid this using createStreamableValue instead, and rendering the component client-side.
- Closed RSC streams cause update issues [(3007)](https://github.com/vercel/ai/issues/3007).

Due to these limitations, AI SDK RSC is marked as experimental, and we do not recommend using it for stable production environments.

As a result, we strongly recommend migrating to AI SDK UI, which has undergone extensive development to provide a more stable and production grade experience.

In building [v0](https://v0.dev), we have invested considerable time exploring how to create the best chat experience on the web. AI SDK UI ships with many of these best practices and commonly used patterns like [language model middleware](/docs/ai-sdk-core/middleware), [multi-step tool calls](/docs/ai-sdk-core/tools-and-tool-calling#multi-step-calls-using-stopwhen), [attachments](/docs/ai-sdk-ui/chatbot#attachments), [telemetry](/docs/ai-sdk-core/telemetry), [provider registry](/docs/ai-sdk-core/provider-management#provider-registry), and many more. These features have been considerately designed into a neat abstraction that you can use to reliably integrate AI into your applications.

## Streaming Chat Completions

### Basic Setup

The `streamUI` function executes as part of a server action as illustrated below.

#### Before: Handle generation and rendering in a single server action

```tsx title="@/app/actions.tsx"
import { openai } from '@ai-sdk/openai';
import { getMutableAIState, streamUI } from '@ai-sdk/rsc';

export async function sendMessage(message: string) {
  'use server';

  const messages = getMutableAIState('messages');

  messages.update([...messages.get(), { role: 'user', content: message }]);

  const { value: stream } = await streamUI({
    model: openai('gpt-6-astra'),
    instructions: 'you are a friendly assistant!',
    messages: messages.get(),
    text: async function* ({ content, done }) {
      // process text
    },
    tools: {
      // tool definitions
    },
  });

  return stream;
}
```

#### Before: Call server action and update UI state

The chat interface calls the server action. The response is then saved using the `useUIState` hook.

```tsx title="@/app/page.tsx"
'use client';

import { useState, ReactNode } from 'react';
import { useActions, useUIState } from '@ai-sdk/rsc';

export default function Page() {
  const { sendMessage } = useActions();
  const [input, setInput] = useState('');
  const [messages, setMessages] = useUIState();

  return (
    <div>
      {messages.map(message => message)}

      <form
        onSubmit={async event => {
          event.preventDefault();
          const response: ReactNode = await sendMessage(input);
          setMessages(msgs => [...msgs, response]);
        }}
      >
        <input type="text" />
        <button type="submit">Submit</button>
      </form>
    </div>
  );
}
```

The `streamUI` function combines generating text and rendering the user interface. To migrate to AI SDK UI, you need to **separate these concerns** – streaming generations with `streamText` and rendering the UI with `useChat`.

#### After: Replace server action with route handler

The `streamText` function executes as part of a route handler and streams the response to the client. The `useChat` hook on the client decodes this stream and renders the response within the chat interface.

```ts title="@/app/api/chat/route.ts"
import {
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
} from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(request) {
  const { messages } = await request.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    instructions: 'you are a friendly assistant!',
    messages,
    tools: {
      // tool definitions
    },
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

#### After: Update client to use chat hook

```tsx title="@/app/page.tsx"
'use client';

import { useChat } from '@ai-sdk/react';

export default function Page() {
  const { messages, input, setInput, handleSubmit } = useChat();

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role}</div>
          <div>{message.content}</div>
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

### Parallel Tool Calls

In AI SDK RSC, `streamUI` does not support parallel tool calls. You will have to use a combination of `streamText`, `createStreamableUI` and `createStreamableValue`.

With AI SDK UI, `useChat` comes with built-in support for parallel tool calls. You can define multiple tools in the `streamText` and have them called in parallel. The `useChat` hook will then handle the parallel tool calls for you automatically.

### Multi-Step Tool Calls

In AI SDK RSC, `streamUI` does not support multi-step tool calls. You will have to use a combination of `streamText`, `createStreamableUI` and `createStreamableValue`.

With AI SDK UI, `useChat` comes with built-in support for multi-step tool calls. You can set `stopWhen` in the `streamText` function to define when the model should stop making tool calls. The `useChat` hook will then handle the multi-step tool calls for you automatically.

### Generative User Interfaces

The `streamUI` function uses `tools` as a way to execute functions based on user input and renders React components based on the function output to go beyond text in the chat interface.

#### Before: Render components within the server action and stream to client

```tsx title="@/app/actions.tsx"
import { z } from 'zod';
import { streamUI } from '@ai-sdk/rsc';
import { openai } from '@ai-sdk/openai';
import { getWeather } from '@/utils/queries';
import { Weather } from '@/components/weather';

const { value: stream } = await streamUI({
  model: openai('gpt-6-astra'),
  instructions: 'you are a friendly assistant!',
  messages,
  text: async function* ({ content, done }) {
    // process text
  },
  tools: {
    displayWeather: {
      description: 'Display the weather for a location',
      inputSchema: z.object({
        latitude: z.number(),
        longitude: z.number(),
      }),
      generate: async function* ({ latitude, longitude }) {
        yield <div>Loading weather...</div>;

        const { value, unit } = await getWeather({ latitude, longitude });

        return <Weather value={value} unit={unit} />;
      },
    },
  },
});
```

As mentioned earlier, `streamUI` generates text and renders the React component in a single server action call.

#### After: Replace with route handler and stream props data to client

The `streamText` function streams the props data as response to the client, while `useChat` decode the stream as `toolInvocations` and renders the chat interface.

```ts title="@/app/api/chat/route.ts"
import { z } from 'zod';
import { openai } from '@ai-sdk/openai';
import { getWeather } from '@/utils/queries';
import {
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
} from 'ai';

export async function POST(request) {
  const { messages } = await request.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    instructions: 'you are a friendly assistant!',
    messages,
    tools: {
      displayWeather: {
        description: 'Display the weather for a location',
        inputSchema: z.object({
          latitude: z.number(),
          longitude: z.number(),
        }),
        execute: async function ({ latitude, longitude }) {
          const props = await getWeather({ latitude, longitude });
          return props;
        },
      },
    },
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

#### After: Update client to use chat hook and render components using tool invocations

```tsx title="@/app/page.tsx"
'use client';

import { useChat } from '@ai-sdk/react';
import { Weather } from '@/components/weather';

export default function Page() {
  const { messages, input, setInput, handleSubmit } = useChat();

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role}</div>
          <div>{message.content}</div>

          <div>
            {message.toolInvocations.map(toolInvocation => {
              const { toolName, toolCallId, state } = toolInvocation;

              if (state === 'result') {
                const { result } = toolInvocation;

                return (
                  <div key={toolCallId}>
                    {toolName === 'displayWeather' ? (
                      <Weather weatherAtLocation={result} />
                    ) : null}
                  </div>
                );
              } else {
                return (
                  <div key={toolCallId}>
                    {toolName === 'displayWeather' ? (
                      <div>Loading weather...</div>
                    ) : null}
                  </div>
                );
              }
            })}
          </div>
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          type="text"
          value={input}
          onChange={event => {
            setInput(event.target.value);
          }}
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

### Handling Client Interactions

With AI SDK RSC, components streamed to the client can trigger subsequent generations by calling the relevant server action using the `useActions` hooks. This is possible as long as the component is a descendant of the `<AI/>` context provider.

#### Before: Use actions hook to send messages

```tsx title="@/app/components/list-flights.tsx"
'use client';

import { useActions, useUIState } from '@ai-sdk/rsc';

export function ListFlights({ flights }) {
  const { sendMessage } = useActions();
  const [_, setMessages] = useUIState();

  return (
    <div>
      {flights.map(flight => (
        <div
          key={flight.id}
          onClick={async () => {
            const response = await sendMessage(
              `I would like to choose flight ${flight.id}!`,
            );

            setMessages(msgs => [...msgs, response]);
          }}
        >
          {flight.name}
        </div>
      ))}
    </div>
  );
}
```

#### After: Use another chat hook with same ID from the component

After switching to AI SDK UI, these messages are synced by initializing the `useChat` hook in the component with the same `id` as the parent component.

```tsx title="@/app/components/list-flights.tsx"
'use client';

import { useChat } from '@ai-sdk/react';

export function ListFlights({ chatId, flights }) {
  const { append } = useChat({
    id: chatId,
    body: { id: chatId },
  });

  return (
    <div>
      {flights.map(flight => (
        <div
          key={flight.id}
          onClick={async () => {
            await append({
              role: 'user',
              content: `I would like to choose flight ${flight.id}!`,
            });
          }}
        >
          {flight.name}
        </div>
      ))}
    </div>
  );
}
```

### Loading Indicators

In AI SDK RSC, you can use the `initial` parameter of `streamUI` to define the component to display while the generation is in progress.

#### Before: Use `loading` to show loading indicator

```tsx title="@/app/actions.tsx"
import { openai } from '@ai-sdk/openai';
import { streamUI } from '@ai-sdk/rsc';

const { value: stream } = await streamUI({
  model: openai('gpt-6-astra'),
  instructions: 'you are a friendly assistant!',
  messages,
  initial: <div>Loading...</div>,
  text: async function* ({ content, done }) {
    // process text
  },
  tools: {
    // tool definitions
  },
});

return stream;
```

With AI SDK UI, you can use the tool invocation state to show a loading indicator while the tool is executing.

#### After: Use tool invocation state to show loading indicator

```tsx title="@/app/components/message.tsx"
'use client';

export function Message({ role, content, toolInvocations }) {
  return (
    <div>
      <div>{role}</div>
      <div>{content}</div>

      {toolInvocations && (
        <div>
          {toolInvocations.map(toolInvocation => {
            const { toolName, toolCallId, state } = toolInvocation;

            if (state === 'result') {
              const { result } = toolInvocation;

              return (
                <div key={toolCallId}>
                  {toolName === 'getWeather' ? (
                    <Weather weatherAtLocation={result} />
                  ) : null}
                </div>
              );
            } else {
              return (
                <div key={toolCallId}>
                  {toolName === 'getWeather' ? (
                    <Weather isLoading={true} />
                  ) : (
                    <div>Loading...</div>
                  )}
                </div>
              );
            }
          })}
        </div>
      )}
    </div>
  );
}
```

### Saving Chats

Before implementing `streamUI` as a server action, you should create an `<AI/>` provider and wrap your application at the root layout to sync the AI and UI states. During initialization, you typically use the `onSetAIState` callback function to track updates to the AI state and save it to the database when `done(...)` is called.

#### Before: Save chats using callback function of context provider

```ts title="@/app/actions.ts"
import { createAI } from '@ai-sdk/rsc';
import { saveChat } from '@/utils/queries';

export const AI = createAI({
  initialAIState: {},
  initialUIState: {},
  actions: {
    // server actions
  },
  onSetAIState: async ({ state, done }) => {
    'use server';

    if (done) {
      await saveChat(state);
    }
  },
});
```

#### After: Save chats using callback function of `streamText`

With AI SDK UI, you will save chats using the `onEnd` callback function of `streamText` in your route handler.

```ts title="@/app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { saveChat } from '@/utils/queries';
import {
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  convertToModelMessages,
} from 'ai';

export async function POST(request) {
  const { id, messages } = await request.json();

  const coreMessages = await convertToModelMessages(messages);

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    instructions: 'you are a friendly assistant!',
    messages: coreMessages,
    onEnd: async ({ responseMessages }) => {
      try {
        await saveChat({
          id,
          messages: [...coreMessages, ...responseMessages],
        });
      } catch (error) {
        console.error('Failed to save chat');
      }
    },
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

### Restoring Chats

When using AI SDK RSC, the `useUIState` hook contains the UI state of the chat. When restoring a previously saved chat, the UI state needs to be loaded with messages.

Similar to how you typically save chats in AI SDK RSC, you should use the `onGetUIState` callback function to retrieve the chat from the database, convert it into UI state, and return it to be accessible through `useUIState`.

#### Before: Load chat from database using callback function of context provider

```ts title="@/app/actions.ts"
import { createAI } from '@ai-sdk/rsc';
import { loadChatFromDB, convertToUIState } from '@/utils/queries';

export const AI = createAI({
  actions: {
    // server actions
  },
  onGetUIState: async () => {
    'use server';

    const chat = await loadChatFromDB();
    const uiState = convertToUIState(chat);

    return uiState;
  },
});
```

AI SDK UI uses the `messages` field of `useChat` to store messages. To load messages when `useChat` is mounted, you should use `initialMessages`.

As messages are typically loaded from the database, we can use a server actions inside a Page component to fetch an older chat from the database during static generation and pass the messages as props to the `<Chat/>` component.

#### After: Load chat from database during static generation of page

```tsx title="@/app/chat/[id]/page.tsx"
import { Chat } from '@/app/components/chat';
import { getChatById } from '@/utils/queries';

// link to example implementation: https://github.com/vercel/ai-chatbot/blob/00b125378c998d19ef60b73fe576df0fe5a0e9d4/lib/utils.ts#L87-L127
import { convertToUIMessages } from '@/utils/functions';

export default async function Page({ params }: { params: any }) {
  const { id } = params;
  const chatFromDb = await getChatById({ id });

  const chat: Chat = {
    ...chatFromDb,
    messages: convertToUIMessages(chatFromDb.messages),
  };

  return <Chat key={id} id={chat.id} initialMessages={chat.messages} />;
}
```

#### After: Pass chat messages as props and load into chat hook

```tsx title="@/app/components/chat.tsx"
'use client';

import { Message } from 'ai';
import { useChat } from '@ai-sdk/react';

export function Chat({
  id,
  initialMessages,
}: {
  id;
  initialMessages: Array<Message>;
}) {
  const { messages } = useChat({
    id,
    initialMessages,
  });

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role}</div>
          <div>{message.content}</div>
        </div>
      ))}
    </div>
  );
}
```

## Streaming Object Generation

The `createStreamableValue` function streams any serializable data from the server to the client. As a result, this function allows you to stream object generations from the server to the client when paired with `streamText` and `Output`.

#### Before: Use streamable value to stream object generations

```ts title="@/app/actions.ts"
import { Output, streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableValue } from '@ai-sdk/rsc';
import { notificationsSchema } from '@/utils/schemas';

export async function generateSampleNotifications() {
  'use server';

  const stream = createStreamableValue();

  (async () => {
    const { partialOutputStream } = streamText({
      model: "anthropic/claude-sonnet-5.5",
      instructions: 'generate sample ios messages for testing',
      prompt: 'messages from a family group chat during diwali, max 4',
      output: Output.object({ schema: notificationsSchema }),
    });

    for await (const partialObject of partialOutputStream) {
      stream.update(partialObject);
    }

    stream.done();
  })();

  return { partialNotificationsStream: stream.value };
}
```

#### Before: Read streamable value and update object

```tsx title="@/app/page.tsx"
'use client';

import { useState } from 'react';
import { readStreamableValue } from '@ai-sdk/rsc';
import { generateSampleNotifications } from '@/app/actions';

export default function Page() {
  const [notifications, setNotifications] = useState(null);

  return (
    <div>
      <button
        onClick={async () => {
          const { partialNotificationsStream } =
            await generateSampleNotifications();

          for await (const partialNotifications of readStreamableValue(
            partialNotificationsStream,
          )) {
            if (partialNotifications) {
              setNotifications(partialNotifications.notifications);
            }
          }
        }}
      >
        Generate
      </button>
    </div>
  );
}
```

To migrate to AI SDK UI, you should use the `useObject` hook and implement `streamText` with `Output` within your route handler.

#### After: Replace with route handler and stream text response

```ts title="@/app/api/object/route.ts"
import { Output, createTextStreamResponse, streamText, toTextStream } from 'ai';
import { openai } from '@ai-sdk/openai';
import { notificationSchema } from '@/utils/schemas';

export async function POST(req: Request) {
  const context = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    output: Output.object({ schema: notificationSchema }),
    prompt:
      `Generate 3 notifications for a messages app in this context:` + context,
  });

  return createTextStreamResponse({
    stream: toTextStream({ stream: result.stream }),
  });
}
```

#### After: Use object hook to decode stream and update object

```tsx title="@/app/page.tsx"
'use client';

import { useObject } from '@ai-sdk/react';
import { notificationSchema } from '@/utils/schemas';

export default function Page() {
  const { object, submit } = useObject({
    api: '/api/object',
    schema: notificationSchema,
  });

  return (
    <div>
      <button onClick={() => submit('Messages during finals week.')}>
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </div>
  );
}
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Multistep Interfaces
description: Overview of Building Multistep Interfaces with AI SDK RSC
url: "https://ai-sdk.dev/docs/ai-sdk-rsc/multistep-interfaces"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK RSC is currently experimental. We recommend using [AI SDK
UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).

Multistep interfaces refer to user interfaces that require multiple independent steps to be executed in order to complete a specific task.

For example, if you wanted to build a Generative UI chatbot capable of booking flights, it could have three steps:

- Search all flights
- Pick flight
- Check availability

To build this kind of application you will leverage two concepts, **tool composition** and **application context**.

**Tool composition** is the process of combining multiple [tools](/docs/ai-sdk-core/tools-and-tool-calling) to create a new tool. This is a powerful concept that allows you to break down complex tasks into smaller, more manageable steps. In the example above, *"search all flights"*, *"pick flight"*, and *"check availability"* come together to create a holistic *"book flight"* tool.

**Application context** refers to the state of the application at any given point in time. This includes the user's input, the output of the language model, and any other relevant information. In the example above, the flight selected in *"pick flight"* would be used as context necessary to complete the *"check availability"* task.

## Overview

In order to build a multistep interface with `@ai-sdk/rsc`, you will need a few things:

- A Server Action that calls and returns the result from the `streamUI` function
- Tool(s) (sub-tasks necessary to complete your overall task)
- React component(s) that should be rendered when the tool is called
- A page to render your chatbot

The general flow that you will follow is:

- User sends a message (calls your Server Action with `useActions`, passing the message as an input)
- Message is appended to the AI State and then passed to the model alongside a number of tools
- Model can decide to call a tool, which will render the `<SomeTool />` component
- Within that component, you can add interactivity by using `useActions` to call the model with your Server Action and `useUIState` to append the model's response (`<SomeOtherTool />`) to the UI State
- And so on...

## Implementation

The turn-by-turn implementation is the simplest form of multistep interfaces. In this implementation, the user and the model take turns during the conversation. For every user input, the model generates a response, and the conversation continues in this turn-by-turn fashion.

In the following example, you specify two tools (`searchFlights` and `lookupFlight`) that the model can use to search for flights and lookup details for a specific flight.

```tsx title="app/actions.tsx"
import { streamUI } from '@ai-sdk/rsc';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const searchFlights = async (
  source: string,
  destination: string,
  date: string,
) => {
  return [
    {
      id: '1',
      flightNumber: 'AA123',
    },
    {
      id: '2',
      flightNumber: 'AA456',
    },
  ];
};

const lookupFlight = async (flightNumber: string) => {
  return {
    flightNumber: flightNumber,
    departureTime: '10:00 AM',
    arrivalTime: '12:00 PM',
  };
};

export async function submitUserMessage(input: string) {
  'use server';

  const ui = await streamUI({
    model: openai('gpt-6-astra'),
    instructions: 'you are a flight booking assistant',
    prompt: input,
    text: async ({ content }) => <div>{content}</div>,
    tools: {
      searchFlights: {
        description: 'search for flights',
        inputSchema: z.object({
          source: z.string().describe('The origin of the flight'),
          destination: z.string().describe('The destination of the flight'),
          date: z.string().describe('The date of the flight'),
        }),
        generate: async function* ({ source, destination, date }) {
          yield `Searching for flights from ${source} to ${destination} on ${date}...`;
          const results = await searchFlights(source, destination, date);

          return (
            <div>
              {results.map(result => (
                <div key={result.id}>
                  <div>{result.flightNumber}</div>
                </div>
              ))}
            </div>
          );
        },
      },
      lookupFlight: {
        description: 'lookup details for a flight',
        inputSchema: z.object({
          flightNumber: z.string().describe('The flight number'),
        }),
        generate: async function* ({ flightNumber }) {
          yield `Looking up details for flight ${flightNumber}...`;
          const details = await lookupFlight(flightNumber);

          return (
            <div>
              <div>Flight Number: {details.flightNumber}</div>
              <div>Departure Time: {details.departureTime}</div>
              <div>Arrival Time: {details.arrivalTime}</div>
            </div>
          );
        },
      },
    },
  });

  return ui.value;
}
```

Next, create an AI context that will hold the UI State and AI State.

```ts title='app/ai.ts'
import { createAI } from '@ai-sdk/rsc';
import { submitUserMessage } from './actions';

export const AI = createAI<any[], React.ReactNode[]>({
  initialUIState: [],
  initialAIState: [],
  actions: {
    submitUserMessage,
  },
});
```

Next, wrap your application with your newly created context.

```tsx title='app/layout.tsx'
import { type ReactNode } from 'react';
import { AI } from './ai';

export default function RootLayout({
  children,
}: Readonly<{ children: ReactNode }>) {
  return (
    <AI>
      <html lang="en">
        <body>{children}</body>
      </html>
    </AI>
  );
}
```

To call your Server Action, update your root page with the following:

```tsx title="app/page.tsx"
'use client';

import { useState } from 'react';
import { AI } from './ai';
import { useActions, useUIState } from '@ai-sdk/rsc';

export default function Page() {
  const [input, setInput] = useState<string>('');
  const [conversation, setConversation] = useUIState<typeof AI>();
  const { submitUserMessage } = useActions();

  const handleSubmit = async (e: React.FormEvent<HTMLFormElement>) => {
    e.preventDefault();
    setInput('');
    setConversation(currentConversation => [
      ...currentConversation,
      <div>{input}</div>,
    ]);
    const message = await submitUserMessage(input);
    setConversation(currentConversation => [...currentConversation, message]);
  };

  return (
    <div>
      <div>
        {conversation.map((message, i) => (
          <div key={i}>{message}</div>
        ))}
      </div>
      <div>
        <form onSubmit={handleSubmit}>
          <input
            type="text"
            value={input}
            onChange={e => setInput(e.target.value)}
          />
          <button>Send Message</button>
        </form>
      </div>
    </div>
  );
}
```

This page pulls in the current UI State using the `useUIState` hook, which is then mapped over and rendered in the UI. To access the Server Action, you use the `useActions` hook which will return all actions that were passed to the `actions` key of the `createAI` function in your `actions.tsx` file. Finally, you call the `submitUserMessage` function like any other TypeScript function. This function returns a React component (`message`) that is then rendered in the UI by updating the UI State with `setConversation`.

In this example, to call the next tool, the user must respond with plain text. **Given you are streaming a React component, you can add a button to trigger the next step in the conversation**.

To add user interaction, you will have to convert the component into a client component and use the `useActions` hook to trigger the next step in the conversation.

```tsx title="components/flights.tsx"
'use client';

import { useActions, useUIState } from '@ai-sdk/rsc';
import { ReactNode } from 'react';

interface FlightsProps {
  flights: { id: string; flightNumber: string }[];
}

export const Flights = ({ flights }: FlightsProps) => {
  const { submitUserMessage } = useActions();
  const [_, setMessages] = useUIState();

  return (
    <div>
      {flights.map(result => (
        <div key={result.id}>
          <div
            onClick={async () => {
              const display = await submitUserMessage(
                `lookupFlight ${result.flightNumber}`,
              );

              setMessages((messages: ReactNode[]) => [...messages, display]);
            }}
          >
            {result.flightNumber}
          </div>
        </div>
      ))}
    </div>
  );
};
```

Now, update your `searchFlights` tool to render the new `<Flights />` component.

```tsx title="actions.tsx"
...
searchFlights: {
  description: 'search for flights',
  inputSchema: z.object({
    source: z.string().describe('The origin of the flight'),
    destination: z.string().describe('The destination of the flight'),
    date: z.string().describe('The date of the flight'),
  }),
  generate: async function* ({ source, destination, date }) {
    yield `Searching for flights from ${source} to ${destination} on ${date}...`;
    const results = await searchFlights(source, destination, date);
    return (<Flights flights={results} />);
  },
}
...
```

In the above example, the `Flights` component is used to display the search results. When the user clicks on a flight number, the `lookupFlight` tool is called with the flight number as a parameter. The `submitUserMessage` action is then called to trigger the next step in the conversation.

Learn more about tool calling in Next.js App Router by checking out examples [here](/examples/next-app/tools).

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Overview
description: An overview of AI SDK RSC.
url: "https://ai-sdk.dev/docs/ai-sdk-rsc/overview"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK RSC is currently experimental. We recommend using [AI SDK
UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).

The `@ai-sdk/rsc` package is compatible with frameworks that support React
Server Components.

[React Server Components](https://nextjs.org/docs/app/building-your-application/rendering/server-components) (RSC) allow you to write UI that can be rendered on the server and streamed to the client. RSCs enable [Server Actions](https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations#with-client-components), a new way to call server-side code directly from the client just like any other function with end-to-end type-safety. This combination opens the door to a new way of building AI applications, allowing the large language model (LLM) to generate and stream UI directly from the server to the client.

## AI SDK RSC Functions

AI SDK RSC has various functions designed to help you build AI-native applications with React Server Components. These functions:

1. Provide abstractions for building Generative UI applications.
   - [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui): calls a model and allows it to respond with React Server Components.
   - [`useUIState`](/docs/reference/ai-sdk-rsc/use-ui-state): returns the current UI state and a function to update the UI State (like React's `useState`). UI State is the visual representation of the AI state.
   - [`useAIState`](/docs/reference/ai-sdk-rsc/use-ai-state): returns the current AI state and a function to update the AI State (like React's `useState`). The AI state is intended to contain context and information shared with the AI model, such as system messages, function responses, and other relevant data.
   - [`useActions`](/docs/reference/ai-sdk-rsc/use-actions): provides access to your Server Actions from the client. This is particularly useful for building interfaces that require user interactions with the server.
   - [`createAI`](/docs/reference/ai-sdk-rsc/create-ai): creates a client-server context provider that can be used to wrap parts of your application tree to easily manage both UI and AI states of your application.
2. Make it simple to work with streamable values between the server and client.
   - [`createStreamableValue`](/docs/reference/ai-sdk-rsc/create-streamable-value): creates a stream that sends values from the server to the client. The value can be any serializable data.
   - [`readStreamableValue`](/docs/reference/ai-sdk-rsc/read-streamable-value): reads a streamable value from the client that was originally created using `createStreamableValue`.
   - [`createStreamableUI`](/docs/reference/ai-sdk-rsc/create-streamable-ui): creates a stream that sends UI from the server to the client.
   - [`useStreamableValue`](/docs/reference/ai-sdk-rsc/use-streamable-value): accepts a streamable value created using `createStreamableValue` and returns the current value, error, and pending state.

## Templates

Check out the following templates to see AI SDK RSC in action.

## API Reference

Please check out the [AI SDK RSC API Reference](/docs/reference/ai-sdk-rsc) for more details on each function.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Saving and Restoring States
description: Saving and restoring AI and UI states with onGetUIState and onSetAIState
url: "https://ai-sdk.dev/docs/ai-sdk-rsc/saving-and-restoring-states"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK RSC is currently experimental. We recommend using [AI SDK
UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).

AI SDK RSC provides convenient methods for saving and restoring AI and UI state. This is useful for saving the state of your application after every model generation, and restoring it when the user revisits the generations.

## AI State

### Saving AI state

The AI state can be saved using the [`onSetAIState`](/docs/reference/ai-sdk-rsc/create-ai#on-set-ai-state) callback, which gets called whenever the AI state is updated. In the following example, you save the chat history to a database whenever the generation is marked as done.

```tsx title='app/ai.ts'
export const AI = createAI<ServerMessage[], ClientMessage[]>({
  actions: {
    continueConversation,
  },
  onSetAIState: async ({ state, done }) => {
    'use server';

    if (done) {
      saveChatToDB(state);
    }
  },
});
```

### Restoring AI state

The AI state can be restored using the [`initialAIState`](/docs/reference/ai-sdk-rsc/create-ai#initial-ai-state) prop passed to the context provider created by the [`createAI`](/docs/reference/ai-sdk-rsc/create-ai) function. In the following example, you restore the chat history from a database when the component is mounted.

```tsx title='app/layout.tsx'
import { ReactNode } from 'react';
import { AI } from './ai';

export default async function RootLayout({
  children,
}: Readonly<{ children: ReactNode }>) {
  const chat = await loadChatFromDB();

  return (
    <html lang="en">
      <body>
        <AI initialAIState={chat}>{children}</AI>
      </body>
    </html>
  );
}
```

## UI State

### Saving UI state

The UI state cannot be saved directly, since the contents aren't yet serializable. Instead, you can use the AI state as proxy to store details about the UI state and use it to restore the UI state when needed.

### Restoring UI state

The UI state can be restored using the AI state as a proxy. In the following example, you restore the chat history from the AI state when the component is mounted. You use the [`onGetUIState`](/docs/reference/ai-sdk-rsc/create-ai#on-get-ui-state) callback to listen for SSR events and restore the UI state.

```tsx title='app/ai.ts'
export const AI = createAI<ServerMessage[], ClientMessage[]>({
  actions: {
    continueConversation,
  },
  onGetUIState: async () => {
    'use server';

    const historyFromDB: ServerMessage[] = await loadChatFromDB();
    const historyFromApp: ServerMessage[] = getAIState();

    // If the history from the database is different from the
    // history in the app, they're not in sync so return the UIState
    // based on the history from the database

    if (historyFromDB.length !== historyFromApp.length) {
      return historyFromDB.map(({ role, content }) => ({
        id: generateId(),
        role,
        display:
          role === 'function' ? (
            <Component {...JSON.parse(content)} />
          ) : (
            content
          ),
      }));
    }
  },
});
```

To learn more, check out this [example](/examples/next-app/state-management/save-and-restore-states) that persists and restores states in your Next.js application.

***

Next, you will learn how you can use `@ai-sdk/rsc` functions like `useActions` and `useUIState` to create interactive, multistep interfaces.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Streaming React Components
description: Overview of streaming RSCs
url: "https://ai-sdk.dev/docs/ai-sdk-rsc/streaming-react-components"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK RSC is currently experimental. We recommend using [AI SDK
UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).

The RSC API allows you to stream React components from the server to the client with the [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui) function. This is useful when you want to go beyond raw text and stream components to the client in real-time.

Similar to [ AI SDK Core ](/docs/ai-sdk-core/overview) APIs (like [ `streamText` ](/docs/reference/ai-sdk-core/stream-text)), `streamUI` provides a single function to call a model and allow it to respond with React Server Components.
It supports the same model interfaces as AI SDK Core APIs.

### Concepts

To give the model the ability to respond to a user's prompt with a React component, you can leverage [tools](/docs/ai-sdk-core/tools-and-tool-calling).

Remember, tools are like programs you can give to the model, and the model can
decide as and when to use based on the context of the conversation.

With the `streamUI` function, **you provide tools that return React components**. With the ability to stream components, the model is akin to a dynamic router that is able to understand the user's intention and display relevant UI.

At a high level, the `streamUI` works like other AI SDK Core functions: you can provide the model with a prompt or some conversation history and, optionally, some tools. If the model decides, based on the context of the conversation, to call a tool, it will generate a tool call. The `streamUI` function will then run the respective tool, returning a React component. If the model doesn't have a relevant tool to use, it will return a text generation, which will be passed to the `text` function, for you to handle (render and return as a React component).

Remember, the

`streamUI`

function must return a React component.

```tsx
const result = await streamUI({
  model: openai('gpt-6-astra'),
  prompt: 'Get the weather for San Francisco',
  text: ({ content }) => <div>{content}</div>,
  tools: {},
});
```

This example calls the `streamUI` function using OpenAI's `gpt-6-astra` model, passes a prompt, specifies how the model's plain text response (`content`) should be rendered, and then provides an empty object for tools. Even though this example does not define any tools, it will stream the model's response as a `div` rather than plain text.

### Adding A Tool

Using tools with `streamUI` is similar to how you use tools with `generateText` and `streamText`.
A tool is an object that has:

- `description`: a string telling the model what the tool does and when to use it
- `inputSchema`: a Zod schema describing what the tool needs in order to run
- `generate`: an asynchronous function that will be run if the model calls the tool. This must return a React component

Let's expand the previous example to add a tool.

```tsx {6-14}
const result = await streamUI({
  model: openai('gpt-6-astra'),
  prompt: 'Get the weather for San Francisco',
  text: ({ content }) => <div>{content}</div>,
  tools: {
    getWeather: {
      description: 'Get the weather for a location',
      inputSchema: z.object({ location: z.string() }),
      generate: async function* ({ location }) {
        yield <LoadingComponent />;
        const weather = await getWeather(location);
        return <WeatherComponent weather={weather} location={location} />;
      },
    },
  },
});
```

This tool would be run if the user asks for the weather for their location. If the user hasn't specified a location, the model will ask for it before calling the tool. When the model calls the tool, the generate function will initially return a loading component. This component will show until the awaited call to `getWeather` is resolved, at which point, the model will stream the `<WeatherComponent />` to the user.

Note: This example uses a [ generator function
](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/function*)
(`function*`), which allows you to pause its execution and return a value,
then resume from where it left off on the next call. This is useful for
handling data streams, as you can fetch and return data from an asynchronous
source like an API, then resume the function to fetch the next chunk when
needed. By yielding values one at a time, generator functions enable efficient
processing of streaming data without blocking the main thread.

## Using `streamUI` with Next.js

Let's see how you can use the example above in a Next.js application.

To use `streamUI` in a Next.js application, you will need two things:

1. A Server Action (where you will call `streamUI`)
2. A page to call the Server Action and render the resulting components

### Step 1: Create a Server Action

Server Actions are server-side functions that you can call directly from the
frontend. For more info, see [the
documentation](https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations#with-client-components).

Create a Server Action at `app/actions.tsx` and add the following code:

```tsx title="app/actions.tsx"
'use server';

import { streamUI } from '@ai-sdk/rsc';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

const LoadingComponent = () => (
  <div className="animate-pulse p-4">getting weather...</div>
);

const getWeather = async (location: string) => {
  await new Promise(resolve => setTimeout(resolve, 2000));
  return '82°F️ ☀️';
};

interface WeatherProps {
  location: string;
  weather: string;
}

const WeatherComponent = (props: WeatherProps) => (
  <div className="border border-neutral-200 p-4 rounded-lg max-w-fit">
    The weather in {props.location} is {props.weather}
  </div>
);

export async function streamComponent() {
  const result = await streamUI({
    model: openai('gpt-6-astra'),
    prompt: 'Get the weather for San Francisco',
    text: ({ content }) => <div>{content}</div>,
    tools: {
      getWeather: {
        description: 'Get the weather for a location',
        inputSchema: z.object({
          location: z.string(),
        }),
        generate: async function* ({ location }) {
          yield <LoadingComponent />;
          const weather = await getWeather(location);
          return <WeatherComponent weather={weather} location={location} />;
        },
      },
    },
  });

  return result.value;
}
```

The `getWeather` tool should look familiar as it is identical to the example in the previous section. In order for this tool to work:

1. First define a `LoadingComponent`, which renders a pulsing `div` that will show some loading text.
2. Next, define a `getWeather` function that will timeout for 2 seconds (to simulate fetching the weather externally) before returning the "weather" for a `location`. Note: you could run any asynchronous TypeScript code here.
3. Finally, define a `WeatherComponent` which takes in `location` and `weather` as props, which are then rendered within a `div`.

Your Server Action is an asynchronous function called `streamComponent` that takes no inputs, and returns a `ReactNode`. Within the action, you call the `streamUI` function, specifying the model (`gpt-6-astra`), the prompt, the component that should be rendered if the model chooses to return text, and finally, your `getWeather` tool. Last but not least, you return the resulting component generated by the model with `result.value`.

To call this Server Action and display the resulting React Component, you will need a page.

### Step 2: Create a Page

Create or update your root page (`app/page.tsx`) with the following code:

```tsx title="app/page.tsx"
'use client';

import { useState } from 'react';
import { Button } from '@/components/ui/button';
import { streamComponent } from './actions';

export default function Page() {
  const [component, setComponent] = useState<React.ReactNode>();

  return (
    <div>
      <form
        onSubmit={async e => {
          e.preventDefault();
          setComponent(await streamComponent());
        }}
      >
        <Button>Stream Component</Button>
      </form>
      <div>{component}</div>
    </div>
  );
}
```

This page is first marked as a client component with the `"use client";` directive given it will be using hooks and interactivity. On the page, you render a form. When that form is submitted, you call the `streamComponent` action created in the previous step (just like any other function). The `streamComponent` action returns a `ReactNode` that you can then render on the page using React state (`setComponent`).

## Going beyond a single prompt

You can now allow the model to respond to your prompt with a React component. However, this example is limited to a static prompt that is set within your Server Action. You could make this example interactive by turning it into a chatbot.

Learn how to stream React components with the Next.js App Router using `streamUI` with this [example](/examples/next-app/interface/route-components).

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Streaming Values
description: Overview of streaming RSCs
url: "https://ai-sdk.dev/docs/ai-sdk-rsc/streaming-values"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK RSC is currently experimental. We recommend using [AI SDK
UI](/docs/ai-sdk-ui/overview) for production. For guidance on migrating from
RSC to UI, see our [migration guide](/docs/ai-sdk-rsc/migrating-to-ui).

The RSC API provides several utility functions to allow you to stream values from the server to the client. This is useful when you need more granular control over what you are streaming and how you are streaming it.

These utilities can also be paired with [AI SDK Core](/docs/ai-sdk-core)
functions like [`streamText`](/docs/reference/ai-sdk-core/stream-text) to
easily stream LLM generations from the server to the client.

There are two functions provided by the RSC API that allow you to create streamable values:

- [`createStreamableValue`](/docs/reference/ai-sdk-rsc/create-streamable-value) - creates a streamable (serializable) value, with full control over how you create, update, and close the stream.
- [`createStreamableUI`](/docs/reference/ai-sdk-rsc/create-streamable-ui) - creates a streamable React component, with full control over how you create, update, and close the stream.

## `createStreamableValue`

The RSC API allows you to stream serializable JavaScript values from the server to the client using [`createStreamableValue`](/docs/reference/ai-sdk-rsc/create-streamable-value), such as strings, numbers, objects, and arrays.

This is useful when you want to stream:

- Text generations from the language model in real-time.
- Buffer values of image and audio generations from multi-modal models.
- Progress updates from multi-step agent runs.

## Creating a Streamable Value

You can import `createStreamableValue` from `@ai-sdk/rsc` and use it to create a streamable value.

```tsx title='app/actions.ts'
'use server';

import { createStreamableValue } from '@ai-sdk/rsc';

export const runThread = async () => {
  const streamableStatus = createStreamableValue('thread.init');

  setTimeout(() => {
    streamableStatus.update('thread.run.create');
    streamableStatus.update('thread.run.update');
    streamableStatus.update('thread.run.end');
    streamableStatus.done('thread.end');
  }, 1000);

  return {
    status: streamableStatus.value,
  };
};
```

## Reading a Streamable Value

You can read streamable values on the client using `readStreamableValue`. It returns an async iterator that yields the value of the streamable as it is updated:

```tsx title='app/page.tsx'
import { readStreamableValue } from '@ai-sdk/rsc';
import { runThread } from '@/actions';

export default function Page() {
  return (
    <button
      onClick={async () => {
        const { status } = await runThread();

        for await (const value of readStreamableValue(status)) {
          console.log(value);
        }
      }}
    >
      Ask
    </button>
  );
}
```

Learn how to stream a text generation (with `streamText`) using the Next.js App Router and `createStreamableValue` in this [example](/examples/next-app/basics/streaming-text-generation).

## `createStreamableUI`

`createStreamableUI` creates a stream that holds a React component. Unlike AI SDK Core APIs, this function does not call a large language model. Instead, it provides a primitive that can be used to have granular control over streaming a React component.

## Using `createStreamableUI`

Let's look at how you can use the `createStreamableUI` function with a Server Action.

```tsx title='app/actions.tsx'
'use server';

import { createStreamableUI } from '@ai-sdk/rsc';

export async function getWeather() {
  const weatherUI = createStreamableUI();

  weatherUI.update(<div style={{ color: 'gray' }}>Loading...</div>);

  setTimeout(() => {
    weatherUI.done(<div>It&apos;s a sunny day!</div>);
  }, 1000);

  return weatherUI.value;
}
```

First, you create a streamable UI with an empty state and then update it with a loading message. After 1 second, you mark the stream as done passing in the actual weather information as its final value. The `.value` property contains the actual UI that can be sent to the client.

## Reading a Streamable UI

On the client side, you can call the `getWeather` Server Action and render the returned UI like any other React component.

```tsx title='app/page.tsx'
'use client';

import { useState } from 'react';
import { readStreamableValue } from '@ai-sdk/rsc';
import { getWeather } from '@/actions';

export default function Page() {
  const [weather, setWeather] = useState<React.ReactNode | null>(null);

  return (
    <div>
      <button
        onClick={async () => {
          const weatherUI = await getWeather();
          setWeather(weatherUI);
        }}
      >
        What&apos;s the weather?
      </button>

      {weather}
    </div>
  );
}
```

When the button is clicked, the `getWeather` function is called, and the returned UI is set to the `weather` state and rendered on the page. Users will see the loading message first and then the actual weather information after 1 second.

Learn more about handling multiple streams in a single request in the [Multiple Streamables](/docs/advanced/multiple-streamables) guide.

Learn more about handling state for more complex use cases with [ AI/UI State ](/docs/ai-sdk-rsc/generative-ui-state).

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Chatbot Message Persistence
description: Learn how to store and load chat messages in a chatbot.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/chatbot-message-persistence"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Being able to store and load chat messages is crucial for most AI chatbots.
In this guide, we'll show how to implement message persistence with `useChat` and `streamText`.

This guide does not cover authorization, error handling, or other real-world
considerations. It is intended to be a simple example of how to implement
message persistence.

## Starting a new chat

When the user navigates to the chat page without providing a chat ID,
we need to create a new chat and redirect to the chat page with the new chat ID.

```tsx title="app/chat/page.tsx"
import { redirect } from 'next/navigation';
import { createChat } from '@util/chat-store';

export default async function Page() {
  const id = await createChat(); // create a new chat
  redirect(`/chat/${id}`); // redirect to chat page, see below
}
```

Our example chat store implementation uses files to store the chat messages.
In a real-world application, you would use a database or a cloud storage service,
and get the chat ID from the database.
That being said, the function interfaces are designed to be easily replaced with other implementations.

```tsx title="util/chat-store.ts"
import { generateId } from 'ai';
import { existsSync, mkdirSync } from 'fs';
import { writeFile } from 'fs/promises';
import path from 'path';

// Treat chat IDs as opaque tokens before using them in file paths.
const chatIdRegex = /^[A-Za-z0-9_-]+$/;

export async function createChat(): Promise<string> {
  const id = generateId(); // generate a unique chat ID
  await writeFile(getChatFile(id), '[]'); // create an empty chat file
  return id;
}

function getChatFile(id: string): string {
  if (!chatIdRegex.test(id)) {
    throw new Error('Invalid chat ID');
  }

  const chatDir = path.resolve(process.cwd(), '.chats');
  const chatFile = path.resolve(chatDir, `${id}.json`);

  // Defense in depth: keep the resolved file inside the chat directory.
  if (!chatFile.startsWith(`${chatDir}${path.sep}`)) {
    throw new Error('Invalid chat ID');
  }

  if (!existsSync(chatDir)) mkdirSync(chatDir, { recursive: true });
  return chatFile;
}
```

The chat ID can come from the URL or the request body, so validate it as an
opaque token before using it in a file path. The resolved path check ensures the
file stays inside the intended `.chats` directory.

## Loading an existing chat

When the user navigates to the chat page with a chat ID, we need to load the chat messages from storage.

The `loadChat` function in our file-based chat store is implemented as follows:

```tsx title="util/chat-store.ts"
import { UIMessage } from 'ai';
import { readFile } from 'fs/promises';

export async function loadChat(id: string): Promise<UIMessage[]> {
  return JSON.parse(await readFile(getChatFile(id), 'utf8'));
}

// ... rest of the file
```

## Validating messages on the server

When processing messages on the server that contain tool calls, custom metadata, or data parts, you should validate them using `validateUIMessages` before sending them to the model.

### Persisting streaming tool input

An `input-streaming` tool part includes `rawInput` while tool arguments are
still arriving. Persist the complete UI message, including this field, so a
resumed stream can append later tool-input deltas to the exact text received
before the interruption. AI SDK updates `rawInput` as each delta arrives and
removes it when the tool input becomes available.

### Migrating deprecated tool `rawInput`

Older persisted tool parts in the `output-error` state may contain `rawInput`.
This field is deprecated and will be removed in the next major version. Store
tool arguments in `input` instead. AI SDK emits a deprecation warning when
`validateUIMessages`, `safeValidateUIMessages`, or `convertToModelMessages`
encounters a defined `rawInput` value. UI stream processing also warns when it
reconstructs a static `output-error` part with `rawInput`.

Migrate each legacy part by copying `rawInput` to `input` when `input` is
`null` or `undefined`, then remove `rawInput`. This preserves the current
backward-compatible conversion behavior:

```ts
import { isToolUIPart } from 'ai';

const migratedParts = message.parts.map(part => {
  if (
    !isToolUIPart(part) ||
    part.state !== 'output-error' ||
    !('rawInput' in part) ||
    part.rawInput === undefined
  ) {
    return part;
  }

  const { rawInput, ...partWithoutRawInput } = part;

  return {
    ...partWithoutRawInput,
    input: part.input ?? rawInput,
  };
});
```

### Validation with tools

When your messages include tool calls, validate them against your tool definitions:

```tsx title="app/api/chat/route.ts" {7-8,14-27,35-45}
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
  validateUIMessages,
  tool,
} from 'ai';
import { z } from 'zod';
import { loadChat, saveChat } from '@util/chat-store';
import { dataSchemas, metadataSchema } from '@util/schemas';

// Define your tools
const tools = {
  weather: tool({
    description: 'Get weather information',
    inputSchema: z.object({
      location: z.string(),
      units: z.enum(['celsius', 'fahrenheit']),
    }),
    execute: async ({ location, units }) => {
      /* tool implementation */
    },
  }),
  // other tools
};

export async function POST(req: Request) {
  const { message, id } = await req.json();

  // Load previous messages from database
  const previousMessages = await loadChat(id);

  // Append new message to previousMessages messages
  const messages = [...previousMessages, message];

  // Validate loaded messages against
  // tools, data parts schema, and metadata schema
  const validatedMessages = await validateUIMessages({
    messages,
    tools, // Ensures tool calls in messages match current schemas
    dataSchemas,
    metadataSchema,
  });

  const result = streamText({
    model: 'openai/gpt-6-luna',
    messages: await convertToModelMessages(validatedMessages),
    tools,
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      originalMessages: messages,
      onEnd: ({ messages }) => {
        saveChat({ chatId: id, messages });
      },
    }),
  });
}
```

### Handling validation errors

Handle validation errors gracefully when messages from the database don't match current schemas:

```tsx title="app/api/chat/route.ts" {3,10-24}
import {
  convertToModelMessages,
  streamText,
  validateUIMessages,
  TypeValidationError,
} from 'ai';
import { type MyUIMessage } from '@/types';

export async function POST(req: Request) {
  const { message, id } = await req.json();

  // Load and validate messages from database
  let validatedMessages: MyUIMessage[];

  try {
    const previousMessages = await loadMessagesFromDB(id);
    validatedMessages = await validateUIMessages({
      // append the new message to the previous messages:
      messages: [...previousMessages, message],
      tools,
      metadataSchema,
    });
  } catch (error) {
    if (error instanceof TypeValidationError) {
      // Log validation error for monitoring
      console.error('Database messages validation failed:', error);
      // Could implement message migration or filtering here
      // For now, start with empty history
      validatedMessages = [];
    } else {
      throw error;
    }
  }

  // Continue with validated messages...
}
```

## Displaying the chat

Once messages are loaded from storage, you can display them in your chat UI. Here's how to set up the page component and the chat display:

```tsx title="app/chat/[id]/page.tsx"
import { loadChat } from '@util/chat-store';
import Chat from '@ui/chat';

export default async function Page(props: { params: Promise<{ id: string }> }) {
  const { id } = await props.params;
  const messages = await loadChat(id);
  return <Chat id={id} initialMessages={messages} />;
}
```

The chat component uses the `useChat` hook to manage the conversation:

```tsx title="ui/chat.tsx" {10-16}
'use client';

import { UIMessage, useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';
import { useState } from 'react';

export default function Chat({
  id,
  initialMessages,
}: { id?: string | undefined; initialMessages?: UIMessage[] } = {}) {
  const [input, setInput] = useState('');
  const { sendMessage, messages } = useChat({
    id, // use the provided chat ID
    messages: initialMessages, // load initial messages
    transport: new DefaultChatTransport({
      api: '/api/chat',
    }),
  });

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    if (input.trim()) {
      sendMessage({ text: input });
      setInput('');
    }
  };

  // simplified rendering code, extend as needed:
  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role === 'user' ? 'User: ' : 'AI: '}
          {m.parts
            .map(part => (part.type === 'text' ? part.text : ''))
            .join('')}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          placeholder="Type a message..."
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

## Storing messages

`useChat` sends the chat id and the messages to the backend.

The `useChat` message format is different from the `ModelMessage` format. The
`useChat` message format is designed for frontend display, and contains
additional fields such as `id` and `createdAt`. We recommend storing the
messages in the `useChat` message format.

When loading messages from storage that contain tools, metadata, or custom data
parts, validate them using `validateUIMessages` before processing (see the
[validation section](#validating-messages-on-the-server) above).

Storing messages is done in the `onEnd` callback of the `toUIMessageStream` function.
`onEnd` receives the complete messages including the new AI response as `UIMessage[]`.

```tsx title="app/api/chat/route.ts" {12,17-25}
import { openai } from '@ai-sdk/openai';
import { saveChat } from '@util/chat-store';
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';

export async function POST(req: Request) {
  const { messages, chatId }: { messages: UIMessage[]; chatId: string } =
    await req.json();

  const result = streamText({
    model: 'openai/gpt-6-luna',
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      originalMessages: messages,
      onEnd: ({ messages }) => {
        saveChat({ chatId, messages });
      },
    }),
  });
}
```

The actual storage of the messages is done in the `saveChat` function, which in
our file-based chat store is implemented as follows:

```tsx title="util/chat-store.ts"
import { UIMessage } from 'ai';
import { writeFile } from 'fs/promises';

export async function saveChat({
  chatId,
  messages,
}: {
  chatId: string;
  messages: UIMessage[];
}): Promise<void> {
  const content = JSON.stringify(messages, null, 2);
  await writeFile(getChatFile(chatId), content);
}

// ... rest of the file
```

## Message IDs

In addition to a chat ID, each message has an ID.
You can use this message ID to e.g. manipulate individual messages.

### Client-side vs Server-side ID Generation

By default, message IDs are generated client-side:

- User message IDs are generated by the `useChat` hook on the client
- AI response message IDs are also generated by `useChat` on the client, unless
  the server sends one in the stream (see below)

For applications without persistence, client-side ID generation works perfectly.
However, **for persistence, you should use IDs that are stable before messages
are stored** to ensure consistency across sessions and prevent ID conflicts when
messages are restored.

The server-side options below control IDs for generated assistant response
messages. User message IDs are created by `useChat` before the request is sent
to your API route, so keep those client-generated IDs when saving incoming
request messages, or generate and persist your own user message IDs before
sending/storing them.

### Setting Up Server-side ID Generation

When implementing persistence, you have two options for generating server-side
IDs for assistant response messages:

1. **Using `generateMessageId` in `toUIMessageStream`**
2. **Setting IDs in your start message part with `createUIMessageStream`**

#### Option 1: Using `generateMessageId` in `toUIMessageStream`

You can control the ID format by providing ID generators using [`createIdGenerator()`](/docs/reference/ai-sdk-core/create-id-generator):

```tsx title="app/api/chat/route.ts" {18-22}
import {
  createIdGenerator,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
} from 'ai';

export async function POST(req: Request) {
  // ...
  const result = streamText({
    // ...
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      originalMessages: messages,
      // Generate consistent server-side IDs for persistence:
      generateMessageId: createIdGenerator({
        prefix: 'msg',
        size: 16,
      }),
      onEnd: ({ messages }) => {
        saveChat({ chatId, messages });
      },
    }),
  });
}
```

#### Option 2: Setting IDs with `createUIMessageStream`

Alternatively, you can use `createUIMessageStream` to control the message ID by writing a start message part:

```tsx title="app/api/chat/route.ts" {15-19,26-28}
import {
  convertToModelMessages,
  generateId,
  streamText,
  createUIMessageStream,
  createUIMessageStreamResponse,
  toUIMessageStream,
} from 'ai';

export async function POST(req: Request) {
  const { messages, chatId } = await req.json();

  const stream = createUIMessageStream({
    execute: async ({ writer }) => {
      // Write start message part with custom ID
      writer.write({
        type: 'start',
        messageId: generateId(), // Generate server-side ID for persistence
      });

      const result = streamText({
        model: 'openai/gpt-6-luna',
        messages: await convertToModelMessages(messages),
      });

      writer.merge(
        toUIMessageStream({ stream: result.stream, sendStart: false }),
      ); // omit start message part
    },
    originalMessages: messages,
    onEnd: ({ responseMessage }) => {
      // save your chat here
    },
  });

  return createUIMessageStreamResponse({ stream });
}
```

For client-side applications that don't require persistence, you can still customize client-side ID generation:

```tsx title="ui/chat.tsx"
import { createIdGenerator } from 'ai';
import { useChat } from '@ai-sdk/react';

const { ... } = useChat({
  generateId: createIdGenerator({
    prefix: 'msgc',
    size: 16,
  }),
  // ...
});
```

## Sending only the last message

Once you have implemented message persistence, you might want to send only the last message to the server.
This reduces the amount of data sent to the server on each request and can improve performance.

To achieve this, you can provide a `prepareSendMessagesRequest` function to the transport.
This function receives the messages and the chat ID, and returns the request body to be sent to the server.

```tsx title="ui/chat.tsx" {7-12}
import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';

const {
  // ...
} = useChat({
  // ...
  transport: new DefaultChatTransport({
    api: '/api/chat',
    // only send the last message to the server:
    prepareSendMessagesRequest({ messages, id }) {
      return { body: { message: messages[messages.length - 1], id } };
    },
  }),
});
```

On the server, you can then load the previous messages and append the new message to the previous messages. If your messages contain tools, metadata, or custom data parts, you should validate them:

```tsx title="app/api/chat/route.ts" {18-24}
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  toUIMessageStream,
  UIMessage,
  validateUIMessages,
} from 'ai';
// import your tools and schemas

export async function POST(req: Request) {
  // get the last message from the client:
  const { message, id } = await req.json();

  // load the previous messages from the server:
  const previousMessages = await loadChat(id);

  // validate messages if they contain tools, metadata, or data parts:
  const validatedMessages = await validateUIMessages({
    // append the new message to the previous messages:
    messages: [...previousMessages, message],
    tools, // if using tools
    metadataSchema, // if using custom metadata
    dataSchemas, // if using custom data parts
  });

  const result = streamText({
    // ...
    messages: await convertToModelMessages(validatedMessages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      originalMessages: validatedMessages,
      onEnd: ({ messages }) => {
        saveChat({ chatId: id, messages });
      },
    }),
  });
}
```

## Handling client disconnects

By default, the AI SDK `streamText` function uses backpressure to the language model provider to prevent
the consumption of tokens that are not yet requested.

However, this means that when the client disconnects, e.g. by closing the browser tab or because of a network issue,
the stream from the LLM will be aborted and the conversation may end up in a broken state.

Assuming that you have a [storage solution](#storing-messages) in place, you can use the `consumeStream` method to consume the stream on the backend,
and then save the result as usual.
`consumeStream` effectively removes the backpressure,
meaning that the result is stored even when the client has already disconnected.

```tsx title="app/api/chat/route.ts" {19-21}
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';
import { saveChat } from '@util/chat-store';

export async function POST(req: Request) {
  const { messages, chatId }: { messages: UIMessage[]; chatId: string } =
    await req.json();

  const result = streamText({
    model,
    messages: await convertToModelMessages(messages),
  });

  // consume the stream to ensure it runs to completion & triggers onEnd
  // even when the client response is aborted:
  result.consumeStream(); // no await

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      originalMessages: messages,
      onEnd: ({ messages }) => {
        saveChat({ chatId, messages });
      },
    }),
  });
}
```

When the client reloads the page after a disconnect, the chat will be restored from the storage solution.

In production applications, you would also track the state of the request (in
progress, complete) in your stored messages and use it on the client to cover
the case where the client reloads the page after a disconnection, but the
streaming is not yet complete.

For more robust handling of disconnects, you may want to add resumability on disconnects. Check out the [Chatbot Resume Streams](/docs/ai-sdk-ui/chatbot-resume-streams) documentation to learn more.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Chatbot Resume Streams
description: Learn how to resume chatbot streams after client disconnects.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/chatbot-resume-streams"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

`useChat` supports resuming ongoing streams after page reloads. Use this feature to build applications with long-running generations.

In a resumable stream setup, client-side aborts are treated as disconnects.
Closing a tab, refreshing the page, or calling `stop()` only closes the
current HTTP connection and should not cancel the underlying generation. To
let users stop generation, add a dedicated stop endpoint that persists the
partial response, cancels the active work, and clears the active stream. See
[Stop an Active Resumable Stream](#stop-an-active-resumable-stream) and
[troubleshooting](/docs/troubleshooting/abort-breaks-resumable-streams) for
more details.

## How stream resumption works

Stream resumption requires persistence for messages and active streams in your application. The AI SDK provides tools to connect to storage, but you need to set up the storage yourself.

**The AI SDK provides:**

- A `resume` option in `useChat` that automatically reconnects to active streams
- Access to the outgoing stream through the `consumeSseStream` callback
- Automatic HTTP requests to your resume endpoints

**You build:**

- Storage to track which stream belongs to each chat
- Redis to store the UIMessage stream
- Two API endpoints: POST to create streams, GET to resume them
- Integration with [`resumable-stream`](https://www.npmjs.com/package/resumable-stream) to manage Redis storage

## Prerequisites

To implement resumable streams in your chat application, you need:

1. **The `resumable-stream` package** - Handles the publisher/subscriber mechanism for streams
2. **A Redis instance** - Stores stream data (e.g. [Redis through Vercel](https://vercel.com/marketplace/redis))
3. **A persistence layer** - Tracks which stream ID is active for each chat (e.g. database)

## Implementation

### 1. Client-side: Enable stream resumption

Use the `resume` option in the `useChat` hook to enable stream resumption. When `resume` is true, the hook automatically attempts to reconnect to any active stream for the chat on mount:

```tsx title="app/chat/[chatId]/chat.tsx"
'use client';

import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport, type UIMessage } from 'ai';

export function Chat({
  chatData,
  resume = false,
}: {
  chatData: { id: string; messages: UIMessage[] };
  resume?: boolean;
}) {
  const { messages, sendMessage, status } = useChat({
    id: chatData.id,
    messages: chatData.messages,
    resume, // Enable automatic stream resumption
    transport: new DefaultChatTransport({
      // You must send the id of the chat
      prepareSendMessagesRequest: ({ id, messages }) => {
        return {
          body: {
            id,
            message: messages[messages.length - 1],
          },
        };
      },
    }),
  });

  return <div>{/* Your chat UI */}</div>;
}
```

You must send the chat ID with each request (see
`prepareSendMessagesRequest`).

When you enable `resume`, the `useChat` hook makes a `GET` request to `/api/chat/[id]/stream` on mount to check for and resume any active streams.

The hook also reconnects when the page becomes visible again after being in the background (for example, after switching browser tabs or returning to a mobile app). This only happens when the stream was interrupted by a network disconnect. Healthy streams and other errors are left untouched.

`DefaultChatTransport` treats resumed streams as full replays by default, as
returned by the `resumable-stream` setup below. On the first `start` chunk, the
client resets the interrupted response and rebuilds it from the replay. This
prevents duplicate text, reasoning, and step parts after a network disconnect.

If your resume endpoint sends only the remaining chunks, set
`resumeStreamIsReplay: false`. This preserves the interrupted response even if
the continuation includes a `start` chunk with the same message ID:

```ts
const transport = new DefaultChatTransport({
  resumeStreamIsReplay: false,
});
```

When a continuation omits the `start` chunk, the client preserves its interrupted
streaming state with either setting. Continuing active text or reasoning parts
requires the same chat instance; a page reload does not retain that streaming
state, so use a full replay for reloads.

The default transport URL-encodes the chat ID as a single path segment. IDs equal
to `.` or `..` are rejected with an `InvalidArgumentError` because URL parsers
interpret them as path traversal segments.

Let's start by creating the POST handler to create the resumable stream.

### 2. Create the POST handler

The POST handler creates resumable streams using the `consumeSseStream` callback:

```ts title="app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { readChat, saveChat } from '@util/chat-store';
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  generateId,
  streamText,
  toUIMessageStream,
  type UIMessage,
} from 'ai';
import { after } from 'next/server';
import { createResumableStreamContext } from 'resumable-stream';

export async function POST(req: Request) {
  const {
    message,
    id,
  }: {
    message: UIMessage | undefined;
    id: string;
  } = await req.json();

  const chat = await readChat(id);
  let messages = chat.messages;

  messages = [...messages, message!];

  // Clear any previous active stream and save the user message
  saveChat({ id, messages, activeStreamId: null });

  const result = streamText({
    model: 'openai/gpt-6-luna',
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      originalMessages: messages,
      generateMessageId: generateId,
      onEnd: ({ messages }) => {
        // Clear the active stream when finished
        saveChat({ id, messages, activeStreamId: null });
      },
    }),
    async consumeSseStream({ stream }) {
      const streamId = generateId();

      // Create a resumable stream from the SSE stream
      const streamContext = createResumableStreamContext({ waitUntil: after });
      await streamContext.createNewResumableStream(streamId, () => stream);

      // Update the chat with the active stream ID
      saveChat({ id, activeStreamId: streamId });
    },
  });
}
```

### 3. Implement the GET handler

Create a GET handler at `/api/chat/[id]/stream` that:

1. Reads the chat ID from the route params
2. Loads the chat data to check for an active stream
3. Returns 204 (No Content) if no stream is active
4. Resumes the existing stream if one is found

```ts title="app/api/chat/[id]/stream/route.ts"
import { readChat } from '@util/chat-store';
import { UI_MESSAGE_STREAM_HEADERS } from 'ai';
import { after } from 'next/server';
import { createResumableStreamContext } from 'resumable-stream';

export async function GET(
  _: Request,
  { params }: { params: Promise<{ id: string }> },
) {
  const { id } = await params;

  const chat = await readChat(id);

  if (chat.activeStreamId == null) {
    // no content response when there is no active stream
    return new Response(null, { status: 204 });
  }

  const streamContext = createResumableStreamContext({
    waitUntil: after,
  });

  return new Response(
    await streamContext.resumeExistingStream(chat.activeStreamId),
    { headers: UI_MESSAGE_STREAM_HEADERS },
  );
}
```

The `after` function from Next.js allows work to continue after the response
has been sent. This ensures that the resumable stream persists in Redis even
after the initial response is returned to the client, enabling reconnection
later.

## How it works

### Request lifecycle

<img alt="Diagram showing the architecture and lifecycle of resumable stream requests" src="https://e742qlubrjnjqpp0.public.blob.vercel-storage.com/resume-stream-diagram.png" width="1608" height="1940" />

The diagram above shows the complete lifecycle of a resumable stream:

1. **Stream creation**: When you send a new message, the POST handler uses `streamText` to generate the response. The `consumeSseStream` callback creates a resumable stream with a unique ID and stores it in Redis through the `resumable-stream` package
2. **Stream tracking**: Your persistence layer saves the `activeStreamId` in the chat data
3. **Client reconnection**: When the client reconnects (page reload, or returning to a backgrounded page after a network disconnect), the `resume` option triggers a GET request to `/api/chat/[id]/stream`
4. **Stream recovery**: The GET handler checks for an `activeStreamId` and uses `resumeExistingStream` to reconnect. If no active stream exists, it returns a 204 (No Content) response
5. **Completion cleanup**: When the stream finishes, the `onEnd` callback clears the `activeStreamId` by setting it to `null`

## Customize the resume endpoint

By default, the `useChat` hook makes a GET request to `/api/chat/[id]/stream` when resuming. Customize this endpoint, credentials, and headers, using the `prepareReconnectToStreamRequest` option in `DefaultChatTransport`:

The callback receives the original, unencoded ID. Its returned `api` URL is used
verbatim, so encode and validate any ID you include in a custom URL yourself.

```tsx title="app/chat/[chatId]/chat.tsx"
import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';

export function Chat({ chatData, resume }) {
  const { messages, sendMessage } = useChat({
    id: chatData.id,
    messages: chatData.messages,
    resume,
    transport: new DefaultChatTransport({
      // Customize reconnect settings (optional)
      prepareReconnectToStreamRequest: ({ id }) => {
        return {
          api: `/api/chat/${encodeURIComponent(id)}/stream`,
          // Or use a different pattern:
          // api: `/api/streams/${encodeURIComponent(id)}/resume`,
          // api: `/api/resume-chat?id=${encodeURIComponent(id)}`,
          credentials: 'include', // Include cookies/auth
          headers: {
            Authorization: 'Bearer token',
            'X-Custom-Header': 'value',
          },
        };
      },
    }),
  });

  return <div>{/* Your chat UI */}</div>;
}
```

This lets you:

- Match your existing API route structure
- Add query parameters or custom paths
- Integrate with different backend architectures

## Stop an Active Resumable Stream

`useChat` includes a `stop()` function that aborts the current client request. In a resumable stream setup, that abort is a disconnect signal, not a request to stop generation.

Stream resumption lets a client reconnect to an active stream after the original connection closes. To make that possible, the server keeps the stream running even when no client is actively consuming it. If the user refreshes the page, closes the tab, loses their connection, or navigates away, the client can reconnect later with `resumeStream()`.

Because of this, a client-side abort (e.g. closing the page or refreshing) only closes the current HTTP connection. It is not a request to cancel the underlying work. If your stop button only calls `stop()`, the model request, background job, workflow, or stream writer can continue running, and the client can reconnect to the same active stream.

To support an explicit stop button, create a dedicated stop endpoint. The endpoint should accept the current assistant message from the client, persist that partial response, cancel the work that is producing the stream, and clear the active stream record for the chat.

Stream resumption also needs your application to store a reference from the chat to the stream that can be resumed. This guide calls that reference `activeStreamId`. The resume endpoint uses it to find the stream to reconnect to. The stop endpoint uses the same value to find the work to cancel, and to avoid clearing a newer stream that may have started while the stop request was in flight.

### Client-side: send the current assistant message

The chat setup is the same as the resumable stream setup above. To add stop behavior, send the latest partial assistant message to your stop endpoint before stopping the local chat stream.

When the client knows the active stream ID, include it in the request. This lets the server ignore stale stop requests that arrive after a newer stream has already started.

```tsx title="app/chat/[chatId]/chat.tsx"
'use client';

import { useChat } from '@ai-sdk/react';
import { type UIMessage } from 'ai';

export function Chat({
  chatData,
  resume = false,
}: {
  chatData: {
    id: string;
    messages: UIMessage[];
    activeStreamId?: string | null;
  };
  resume?: boolean;
}) {
  const chat = useChat({
    id: chatData.id,
    messages: chatData.messages,
    resume,
  });

  const stop = () => {
    const lastMessage = chat.messages[chat.messages.length - 1];
    const assistantMessage =
      lastMessage?.role === 'assistant' ? lastMessage : undefined;

    void fetch(`/api/chat/${encodeURIComponent(chatData.id)}/stop`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(
        assistantMessage || chatData.activeStreamId
          ? {
              assistantMessage,
              activeStreamId: chatData.activeStreamId,
            }
          : {},
      ),
    });

    void chat.stop();
  };

  return <button onClick={stop}>Stop</button>;
}
```

The stop request tells your server to cancel the active work. `chat.stop()` stops the local client from reading more chunks.

### Server-side: stop the active work and clear the stream

The stop endpoint should:

1. Load the chat and read its `activeStreamId`
2. Persist the assistant snapshot if one was sent
3. Cancel the work that is producing the stream
4. Clear `activeStreamId` only if it still points to the same stream

```ts title="app/api/chat/[id]/stop/route.ts"
import { readChat, saveChat } from '@util/chat-store';
import { type UIMessage } from 'ai';

type StopRequest = {
  activeStreamId?: string | null;
  assistantMessage?: UIMessage;
};

export async function POST(
  req: Request,
  { params }: { params: Promise<{ id: string }> },
) {
  const { id } = await params;
  const chat = await readChat(id);

  if (chat.activeStreamId == null) {
    return Response.json({ success: true });
  }

  const activeStreamId = chat.activeStreamId;
  const body = (await req.json().catch(() => ({}))) as StopRequest;

  if (body.activeStreamId != null && body.activeStreamId !== activeStreamId) {
    return Response.json({ success: true });
  }

  if (body.assistantMessage) {
    await saveAssistantSnapshot({
      chatId: id,
      message: body.assistantMessage,
    });
  }

  await markStreamAsStopped(activeStreamId);
  await cancelActiveWork(activeStreamId);

  const latestChat = await readChat(id);
  if (latestChat.activeStreamId === activeStreamId) {
    await saveChat({ id, activeStreamId: null });
  }

  return Response.json({ success: true });
}
```

`markStreamAsStopped` and `cancelActiveWork` depend on your backend. In a Redis-backed resumable stream setup, you might close the stored stream and abort the model request that is writing to it. In a workflow setup, you might cancel the workflow run that owns the stream. In a job queue setup, you might cancel the job or write a cancellation flag that the job checks.

The `activeStreamId` can identify replay state, producer state, or both. If those are separate in your system, store enough information with the chat to cancel the producer that writes to the stream.

Persist the assistant snapshot as an insert or merge. Avoid overwriting a newer server-written message with an older client snapshot.

### Keep navigation separate from stop

Do not call the stop endpoint from route cleanup code. Route cleanup is a disconnect, not an explicit stop. The active stream should remain resumable when the user refreshes the page or navigates away.

Only call the stop endpoint for an explicit user action, such as pressing a stop button.

After a user stops a stream, avoid automatic reconnect attempts for that chat until the user sends another message or explicitly retries. Otherwise the client can reconnect before cancellation has finished.

## Important considerations

- **Stream expiration**: Streams in Redis expire after a set time (configurable in the `resumable-stream` package)
- **Multiple clients**: Multiple clients can connect to the same stream simultaneously
- **Error handling**: When no active stream exists, the GET handler returns a 204 (No Content) status code
- **Security**: Ensure proper authentication and authorization for both creating and resuming streams
- **Race conditions**: Clear the `activeStreamId` when starting a new stream to prevent resuming outdated streams

<br />

[View Example on GitHub](https://github.com/vercel/ai/blob/main/examples/next)

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Chatbot Tool Usage
description: Learn how to use tools with the useChat hook.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/chatbot-tool-usage"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

With [`useChat`](/docs/reference/ai-sdk-ui/use-chat) and [`streamText`](/docs/reference/ai-sdk-core/stream-text), you can use tools in your chatbot application.
The AI SDK supports three tool execution patterns in this context:

1. Automatically executed server-side tools
2. Automatically executed client-side tools
3. Tools that require user interaction, such as confirmation dialogs

The flow is as follows:

1. The user enters a message in the chat UI.
2. The message is sent to the API route.
3. In your server-side route, the language model generates tool calls during the `streamText` call.
4. All tool calls are forwarded to the client.
5. Server-side tools are executed using their `execute` method and their results are forwarded to the client.
6. Client-side tools that should be automatically executed are handled with the `onToolCall` callback.
   You must call `addToolOutput` to provide the tool result.
7. Client-side tools that require user interactions can be displayed in the UI.
   The tool calls and results are available as tool invocation parts in the `parts` property of the last assistant message.
8. When the user interaction is done, `addToolOutput` can be used to add the tool result to the chat.
9. The chat can be configured to automatically submit when all tool results are available using `sendAutomaticallyWhen`.
   This triggers another iteration of this flow.

The tool calls and tool executions are integrated into the assistant message as typed tool parts.
A tool part is at first a tool call, and then it becomes a tool result when the tool is executed.
The tool result contains all information about the tool call as well as the result of the tool execution.

Tool result submission can be configured using the `sendAutomaticallyWhen`
option. You can use the `lastAssistantMessageIsCompleteWithToolCalls` helper
to automatically submit when all tool results are available. This simplifies
the client-side code while still allowing full control when needed.

## Example

In this example, we'll use three tools:

- `getWeatherInformation`: An automatically executed server-side tool that returns the weather in a given city.
- `askForConfirmation`: A user-interaction client-side tool that asks the user for confirmation.
- `getLocation`: An automatically executed client-side tool that returns a random city.

### API route

```tsx title='app/api/chat/route.ts'
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';
import { z } from 'zod';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
    tools: {
      // server-side tool with execute function:
      getWeatherInformation: {
        description: 'show the weather in a given city to the user',
        inputSchema: z.object({ city: z.string() }),
        execute: async ({}: { city: string }) => {
          const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
          return weatherOptions[
            Math.floor(Math.random() * weatherOptions.length)
          ];
        },
      },
      // client-side tool that starts user interaction:
      askForConfirmation: {
        description: 'Ask the user for confirmation.',
        inputSchema: z.object({
          message: z.string().describe('The message to ask for confirmation.'),
        }),
      },
      // client-side tool that is automatically executed on the client:
      getLocation: {
        description:
          'Get the user location. Always ask for confirmation before using this tool.',
        inputSchema: z.object({}),
      },
    },
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

### Client-side page

The client-side page uses the `useChat` hook to create a chatbot application with real-time message streaming.
Tool calls are displayed in the chat UI as typed tool parts.
Please make sure to render the messages using the `parts` property of the message.

There are three things worth mentioning:

1. The [`onToolCall`](/docs/reference/ai-sdk-ui/use-chat#on-tool-call) callback is used to handle client-side tools that should be automatically executed.
   In this example, the `getLocation` tool is a client-side tool that returns a random city.
   You call `addToolOutput` to provide the result (without `await` to avoid potential deadlocks).

   Always check `if (toolCall.dynamic)` first in your `onToolCall` handler.
   Without this check, TypeScript will throw an error like: `Type 'string' is
       not assignable to type '"toolName1" | "toolName2"'` when you try to use
   `toolCall.toolName` in `addToolOutput`.

2. The [`sendAutomaticallyWhen`](/docs/reference/ai-sdk-ui/use-chat#send-automatically-when) option with `lastAssistantMessageIsCompleteWithToolCalls` helper automatically submits when all tool results are available.

3. The `parts` array of assistant messages contains tool parts with typed names like `tool-askForConfirmation`.
   The client-side tool `askForConfirmation` is displayed in the UI.
   It asks the user for confirmation and displays the result once the user confirms or denies the execution.
   The result is added to the chat using `addToolOutput` with the `tool` parameter for type safety.

Typed tool parts also include the `approval-requested`, `approval-responded`,
and `output-denied` states. Include these states when handling `part.state`
exhaustively, even when a tool does not require approval. See
[Tool execution approval](#tool-execution-approval) for a complete approval UI.

```tsx title='app/page.tsx' {6,11,16,19-23,25-26,28-33,51,66-70,77-81}
'use client';

import { useChat } from '@ai-sdk/react';
import {
  DefaultChatTransport,
  lastAssistantMessageIsCompleteWithToolCalls,
} from 'ai';
import { useState } from 'react';

export default function Chat() {
  const { messages, sendMessage, addToolOutput } = useChat({
    transport: new DefaultChatTransport({
      api: '/api/chat',
    }),

    sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls,

    // run client-side tools that are automatically executed:
    async onToolCall({ toolCall }) {
      // Check if it's a dynamic tool first for proper type narrowing
      if (toolCall.dynamic) {
        return;
      }

      if (toolCall.toolName === 'getLocation') {
        const cities = ['New York', 'Los Angeles', 'Chicago', 'San Francisco'];

        // No await - avoids potential deadlocks
        addToolOutput({
          tool: 'getLocation',
          toolCallId: toolCall.toolCallId,
          output: cities[Math.floor(Math.random() * cities.length)],
        });
      }
    },
  });
  const [input, setInput] = useState('');

  return (
    <>
      {messages?.map(message => (
        <div key={message.id}>
          <strong>{`${message.role}: `}</strong>
          {message.parts.map(part => {
            switch (part.type) {
              // render text parts as simple text:
              case 'text':
                return part.text;

              // for tool parts, use the typed tool part names:
              case 'tool-askForConfirmation': {
                const callId = part.toolCallId;

                switch (part.state) {
                  case 'input-streaming':
                    return (
                      <div key={callId}>Loading confirmation request...</div>
                    );
                  case 'input-available':
                    return (
                      <div key={callId}>
                        {part.input.message}
                        <div>
                          <button
                            onClick={() =>
                              addToolOutput({
                                tool: 'askForConfirmation',
                                toolCallId: callId,
                                output: 'Yes, confirmed.',
                              })
                            }
                          >
                            Yes
                          </button>
                          <button
                            onClick={() =>
                              addToolOutput({
                                tool: 'askForConfirmation',
                                toolCallId: callId,
                                output: 'No, denied',
                              })
                            }
                          >
                            No
                          </button>
                        </div>
                      </div>
                    );
                  case 'approval-requested':
                    return <div key={callId}>Approval requested.</div>;
                  case 'approval-responded':
                    return <div key={callId}>Approval response received.</div>;
                  case 'output-available':
                    return (
                      <div key={callId}>Confirmation result: {part.output}</div>
                    );
                  case 'output-error':
                    return <div key={callId}>Error: {part.errorText}</div>;
                  case 'output-denied':
                    return <div key={callId}>Tool call denied.</div>;
                }
                break;
              }

              case 'tool-getLocation': {
                const callId = part.toolCallId;

                switch (part.state) {
                  case 'input-streaming':
                    return (
                      <div key={callId}>Preparing location request...</div>
                    );
                  case 'input-available':
                    return <div key={callId}>Getting location...</div>;
                  case 'approval-requested':
                    return <div key={callId}>Approval requested.</div>;
                  case 'approval-responded':
                    return <div key={callId}>Approval response received.</div>;
                  case 'output-available':
                    return <div key={callId}>Location: {part.output}</div>;
                  case 'output-error':
                    return (
                      <div key={callId}>
                        Error getting location: {part.errorText}
                      </div>
                    );
                  case 'output-denied':
                    return <div key={callId}>Location request denied.</div>;
                }
                break;
              }

              case 'tool-getWeatherInformation': {
                const callId = part.toolCallId;

                switch (part.state) {
                  // example of pre-rendering streaming tool inputs:
                  case 'input-streaming':
                    return (
                      <pre key={callId}>{JSON.stringify(part, null, 2)}</pre>
                    );
                  case 'input-available':
                    return (
                      <div key={callId}>
                        Getting weather information for {part.input.city}...
                      </div>
                    );
                  case 'approval-requested':
                    return <div key={callId}>Approval requested.</div>;
                  case 'approval-responded':
                    return <div key={callId}>Approval response received.</div>;
                  case 'output-available':
                    return (
                      <div key={callId}>
                        Weather in {part.input.city}: {part.output}
                      </div>
                    );
                  case 'output-error':
                    return (
                      <div key={callId}>
                        Error getting weather for {part.input.city}:{' '}
                        {part.errorText}
                      </div>
                    );
                  case 'output-denied':
                    return <div key={callId}>Weather request denied.</div>;
                }
                break;
              }
            }
          })}
          <br />
        </div>
      ))}

      <form
        onSubmit={e => {
          e.preventDefault();
          if (input.trim()) {
            sendMessage({ text: input });
            setInput('');
          }
        }}
      >
        <input value={input} onChange={e => setInput(e.target.value)} />
      </form>
    </>
  );
}
```

### Error handling

Sometimes an error may occur during client-side tool execution. Use the `addToolOutput` method with a `state` of `output-error` and `errorText` value instead of `output` to record the error.

```tsx title='app/page.tsx' {19,36-41}
'use client';

import { useChat } from '@ai-sdk/react';
import {
  DefaultChatTransport,
  lastAssistantMessageIsCompleteWithToolCalls,
} from 'ai';
import { useState } from 'react';

export default function Chat() {
  const { messages, sendMessage, addToolOutput } = useChat({
    transport: new DefaultChatTransport({
      api: '/api/chat',
    }),

    sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls,

    // run client-side tools that are automatically executed:
    async onToolCall({ toolCall }) {
      // Check if it's a dynamic tool first for proper type narrowing
      if (toolCall.dynamic) {
        return;
      }

      if (toolCall.toolName === 'getLocation') {
        try {
          const location = await getLocation();

          // No await - avoids potential deadlocks
          addToolOutput({
            tool: 'getLocation',
            toolCallId: toolCall.toolCallId,
            output: location,
          });
        } catch (err) {
          addToolOutput({
            tool: 'getLocation',
            toolCallId: toolCall.toolCallId,
            state: 'output-error',
            errorText: 'Unable to get the user location',
          });
        }
      }
    },
  });
}
```

When rendering messages, use `isToolOutputErrorUIPart` to identify failed
static and dynamic tool parts without checking the tool state directly:

```tsx
import { isToolOutputErrorUIPart, type UIMessage } from 'ai';

function ToolError({ part }: { part: UIMessage['parts'][number] }) {
  if (!isToolOutputErrorUIPart(part)) {
    return null;
  }

  return <div role="alert">{part.errorText}</div>;
}
```

## Tool Execution Approval

Tool execution approval lets you require user confirmation before a server-side tool runs. Unlike [client-side tools](#example) that execute in the browser, tools with approval still execute on the server—but only after the user approves.

Use tool execution approval when you want to:

- Confirm sensitive operations (payments, deletions, external API calls)
- Let users review tool inputs before execution
- Add human oversight to automated workflows

For tools that need to run in the browser (updating UI state, accessing browser APIs), use client-side tools instead.

### Server Setup

Enable approval with `toolApproval` on `streamText`. The older
`needsApproval` property on tools is deprecated. See [Tool Execution Approval](/docs/ai-sdk-core/tools-and-tool-calling#tool-execution-approval) for configuration options including dynamic approval based on input.

```tsx title='app/api/chat/route.ts'
import {
  createUIMessageStreamResponse,
  streamText,
  tool,
  toUIMessageStream,
} from 'ai';
import { z } from 'zod';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages,
    tools: {
      getWeather: tool({
        description: 'Get the weather in a location',
        inputSchema: z.object({
          city: z.string(),
        }),
        execute: async ({ city }) => {
          const weather = await fetchWeather(city);
          return weather;
        },
      }),
    },
    toolApproval: {
      getWeather: 'user-approval',
    },
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

### Client-Side Approval UI

When a tool requires manual approval, the tool part state is
`approval-requested`. Automatic approvals and denials also flow through the
same approval states, but they set `part.approval.isAutomatic === true`, so you
can render the status without calling `addToolApprovalResponse`. Automatic
approval decisions can also include `part.approval.reason`.

```tsx title='app/page.tsx'
'use client';

import { useChat } from '@ai-sdk/react';

export default function Chat() {
  const { messages, addToolApprovalResponse } = useChat();

  return (
    <>
      {messages.map(message => (
        <div key={message.id}>
          {message.parts.map(part => {
            if (part.type === 'tool-getWeather') {
              switch (part.state) {
                case 'approval-requested': {
                  if (part.approval.isAutomatic) {
                    return (
                      <div key={part.toolCallId}>
                        Checking approval for {part.input.city}...
                      </div>
                    );
                  }

                  return (
                    <div key={part.toolCallId}>
                      <p>Get weather for {part.input.city}?</p>
                      {part.approval.requestReason && (
                        <p>{part.approval.requestReason}</p>
                      )}
                      <button
                        onClick={() =>
                          addToolApprovalResponse({
                            id: part.approval.id,
                            approved: true,
                          })
                        }
                      >
                        Approve
                      </button>
                      <button
                        onClick={() =>
                          addToolApprovalResponse({
                            id: part.approval.id,
                            approved: false,
                          })
                        }
                      >
                        Deny
                      </button>
                    </div>
                  );
                }
                case 'approval-responded':
                  return (
                    <div key={part.toolCallId}>
                      Weather request for {part.input.city} was
                      {part.approval.isAutomatic ? ' automatically' : ''}{' '}
                      {part.approval.approved ? 'approved' : 'denied'}.
                      {part.approval.reason
                        ? ` Reason: ${part.approval.reason}`
                        : ''}
                    </div>
                  );
                case 'output-available':
                  return (
                    <div key={part.toolCallId}>
                      Weather in {part.input.city}: {part.output}
                    </div>
                  );
                case 'output-denied':
                  return (
                    <div key={part.toolCallId}>
                      Weather request for {part.input.city} was denied.
                      {part.approval.reason
                        ? ` Reason: ${part.approval.reason}`
                        : ''}
                    </div>
                  );
              }
            }
            // Handle other part types...
          })}
        </div>
      ))}
    </>
  );
}
```

Call `addToolApprovalResponse` only for manual approvals. Automatic approval
decisions already arrive in the UI stream as `approval-requested` and
`approval-responded` states, and denied executions continue to `output-denied`.
If you return a `reason` from an automatic approval or denial, it is available
as `part.approval.reason`.
For manual approval requests, the reason for requiring approval is available as
`part.approval.requestReason`. It remains separate from an optional response
reason supplied to `addToolApprovalResponse`.
When a response reason is supplied with a denial, it is included in the
`execution-denied` tool result that the model sees. When it is supplied with an
approval for a tool executed by the AI SDK, it is recorded on the message part,
but the SDK does not include it in the model prompt, modify the tool's input, or
expose it through a dedicated execution option. To collect input from the user
for a tool, use a client-side tool and `addToolOutput` instead.

Approval request chunks can also include an `approvalDescriptor` with opaque
application-specific metadata. UI message processing exposes it as
`part.approval.descriptor` in the `approval-requested` state and preserves it in
subsequent approval-bearing states, including `approval-responded`. This lets
clients render or persist server-computed approval metadata without using it to
determine whether the tool was approved.

### Securing Approvals for Sensitive Tools

In the `useChat` pattern, the client sends the full message history to the server each turn. Without additional protection, a modified client could fabricate an approval response. For tools that perform sensitive operations, add `experimental_toolApprovalSecret` to your `streamText` call so the server cryptographically verifies that it issued the approval:

```tsx title='app/api/chat/route.ts' {6}
const result = streamText({
  model: "anthropic/claude-sonnet-5.5",
  messages,
  tools: { deleteFile },
  toolApproval: { deleteFile: 'user-approval' },
  experimental_toolApprovalSecret: process.env.TOOL_APPROVAL_SECRET,
});
```

See [Security Considerations](/docs/agents/tool-approvals#security-considerations) for setup details.

### Auto-Submit After Approval

If nothing happens after you approve a tool execution, make sure you either
call `sendMessage` manually or configure `sendAutomaticallyWhen` on the
`useChat` hook.

Use `lastAssistantMessageIsCompleteWithApprovalResponses` to automatically continue the conversation after approvals:

```tsx
import { useChat } from '@ai-sdk/react';
import { lastAssistantMessageIsCompleteWithApprovalResponses } from 'ai';

const { messages, addToolApprovalResponse } = useChat({
  sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithApprovalResponses,
});
```

## Dynamic Tools

When using dynamic tools (tools with unknown types at compile time), the UI parts use a generic `dynamic-tool` type instead of specific tool types:

```tsx title='app/page.tsx'
{
  message.parts.map((part, index) => {
    switch (part.type) {
      // Static tools with specific (`tool-${toolName}`) types
      case 'tool-getWeatherInformation':
        return <WeatherDisplay part={part} />;

      // Dynamic tools use generic `dynamic-tool` type
      case 'dynamic-tool':
        return (
          <div key={index}>
            <h4>Tool: {part.toolName}</h4>
            {part.state === 'input-streaming' && (
              <pre>{JSON.stringify(part.input, null, 2)}</pre>
            )}
            {part.state === 'output-available' && (
              <pre>{JSON.stringify(part.output, null, 2)}</pre>
            )}
            {part.state === 'output-error' && (
              <div>Error: {part.errorText}</div>
            )}
          </div>
        );
    }
  });
}
```

Dynamic tools are useful when integrating with:

- MCP (Model Context Protocol) tools without schemas
- User-defined functions loaded at runtime
- External tool providers

## Tool call streaming

Tool call streaming is **enabled by default** in AI SDK 5.0, allowing you to stream tool calls while they are being generated. This provides a better user experience by showing tool inputs as they are generated in real-time.

```tsx title='app/api/chat/route.ts'
export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
    // toolCallStreaming is enabled by default in v5
    // ...
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

With tool call streaming enabled, partial tool calls are streamed as part of the data stream.
They are available through the `useChat` hook.
The typed tool parts of assistant messages will also contain partial tool calls.
You can use the `state` property of the tool part to render the correct UI.

```tsx title='app/page.tsx' {9,10}
export default function Chat() {
  // ...
  return (
    <>
      {messages?.map(message => (
        <div key={message.id}>
          {message.parts.map(part => {
            switch (part.type) {
              case 'tool-askForConfirmation':
              case 'tool-getLocation':
              case 'tool-getWeatherInformation':
                switch (part.state) {
                  case 'input-streaming':
                    return <pre>{JSON.stringify(part.input, null, 2)}</pre>;
                  case 'input-available':
                    return <pre>{JSON.stringify(part.input, null, 2)}</pre>;
                  case 'approval-requested':
                    return <div>Approval requested.</div>;
                  case 'approval-responded':
                    return <div>Approval response received.</div>;
                  case 'output-available':
                    return <pre>{JSON.stringify(part.output, null, 2)}</pre>;
                  case 'output-error':
                    return <div>Error: {part.errorText}</div>;
                  case 'output-denied':
                    return <div>Tool call denied.</div>;
                }
            }
          })}
        </div>
      ))}
    </>
  );
}
```

## Step start parts

When you are using multi-step tool calls, the AI SDK will add step start parts to the assistant messages.
If you want to display boundaries between tool calls, you can use the `step-start` parts as follows:

```tsx title='app/page.tsx'
// ...
// where you render the message parts:
message.parts.map((part, index) => {
  switch (part.type) {
    case 'step-start':
      // show step boundaries as horizontal lines:
      return index > 0 ? (
        <div key={index} className="text-gray-500">
          <hr className="my-2 border-gray-300" />
        </div>
      ) : null;
    case 'text':
    // ...
    case 'tool-askForConfirmation':
    case 'tool-getLocation':
    case 'tool-getWeatherInformation':
    // ...
  }
});
// ...
```

## Server-side Multi-Step Calls

You can also use multi-step calls on the server-side with `streamText`.
This works when all invoked tools have an `execute` function on the server side.

```tsx title='app/api/chat/route.ts' {22-28,31}
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  isStepCount,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';
import { z } from 'zod';

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
    tools: {
      getWeatherInformation: {
        description: 'show the weather in a given city to the user',
        inputSchema: z.object({ city: z.string() }),
        // tool has execute function:
        execute: async ({}: { city: string }) => {
          const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
          return weatherOptions[
            Math.floor(Math.random() * weatherOptions.length)
          ];
        },
      },
    },
    stopWhen: isStepCount(5),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

## Errors

Language models can make errors when calling tools.
By default, these errors are masked for security reasons, and show up as "An error occurred" in the UI.

Tool execution errors marked `providerExecuted: true` bypass this masking and the
UI stream's `onError` callback. String errors pass through unchanged; other error
values are serialized with `JSON.stringify`. The original error data is needed
when converting persisted UI messages back into model messages, for example to
preserve Anthropic tool error codes. Harness runtime tool errors also retain
their original messages. The callback still handles invalid tool calls and
stream errors, regardless of whether a tool is provider-executed.

To customize masked errors, pass `onError` to `toUIMessageStream`. This callback
does not redact provider-executed tool execution errors sent to the client.

```tsx
export function errorHandler(error: unknown) {
  if (error == null) {
    return 'unknown error';
  }

  if (typeof error === 'string') {
    return error;
  }

  if (error instanceof Error) {
    return error.message;
  }

  return JSON.stringify(error);
}
```

```tsx
const result = streamText({
  // ...
});

return createUIMessageStreamResponse({
  stream: toUIMessageStream({
    stream: result.stream,
    onError: errorHandler,
  }),
});
```

In case you are using `createUIMessageResponse`, you can use the `onError` function when calling `toUIMessageResponse`:

```tsx
const response = createUIMessageResponse({
  // ...
  async execute(dataStream) {
    // ...
  },
  onError: error => `Custom error: ${error.message}`,
});
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Chatbot
description: Learn how to use the useChat hook.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/chatbot"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

The `useChat` hook makes it effortless to create a conversational user interface for your chatbot application. It enables the streaming of chat messages from your AI provider, manages the chat state, and updates the UI automatically as new messages arrive.

To summarize, the `useChat` hook provides the following features:

- **Message Streaming**: All the messages from the AI provider are streamed to the chat UI in real-time.
- **Managed States**: The hook manages the states for input, messages, status, error and more for you.
- **Seamless Integration**: Easily integrate your chat AI into any design or layout with minimal effort.

In this guide, you will learn how to use the `useChat` hook to create a chatbot application with real-time message streaming.
Check out our [chatbot with tools guide](/docs/ai-sdk-ui/chatbot-tool-usage) to learn how to use tools in your chatbot.
Let's start with the following example first.

## Example

```tsx title='app/page.tsx'
'use client';

import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';
import { useState } from 'react';

export default function Page() {
  const { messages, sendMessage, status } = useChat({
    transport: new DefaultChatTransport({
      api: '/api/chat',
    }),
  });
  const [input, setInput] = useState('');

  return (
    <>
      {messages.map(message => (
        <div key={message.id}>
          {message.role === 'user' ? 'User: ' : 'AI: '}
          {message.parts.map((part, index) =>
            part.type === 'text' ? <span key={index}>{part.text}</span> : null,
          )}
        </div>
      ))}

      <form
        onSubmit={e => {
          e.preventDefault();
          if (input.trim()) {
            sendMessage({ text: input });
            setInput('');
          }
        }}
      >
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          disabled={status !== 'ready'}
          placeholder="Say something..."
        />
        <button type="submit" disabled={status !== 'ready'}>
          Submit
        </button>
      </form>
    </>
  );
}
```

```ts title='app/api/chat/route.ts'
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    instructions: 'You are a helpful assistant.',
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

The UI messages have a new `parts` property that contains the message parts.
We recommend rendering the messages using the `parts` property instead of the
`content` property. The parts property supports different message types,
including text, tool invocation, and tool result, and allows for more flexible
and complex chat UIs.

In the `Page` component, the `useChat` hook will request to your AI provider endpoint whenever the user sends a message using `sendMessage`.
The messages are then streamed back in real-time and displayed in the chat UI.

This enables a seamless chat experience where the user can see the AI response as soon as it is available,
without having to wait for the entire response to be received.

## Customized UI

`useChat` also provides ways to manage the chat message states via code, show status, and update messages without being triggered by user interactions.

### Status

The `useChat` hook returns a `status`. It has the following possible values:

- `submitted`: The message has been sent to the API and we're awaiting the start of the response stream.
- `streaming`: The response is actively streaming in from the API, receiving chunks of data.
- `ready`: The full response has been received and processed; a new user message can be submitted.
- `error`: An error occurred during the API request, preventing successful completion.

You can use `status` for e.g. the following purposes:

- To show a loading spinner while the chatbot is processing the user's message.
- To show a "Stop" button to abort the current message.
- To disable the submit button.

```tsx title='app/page.tsx' {8,26-33,47,50}
'use client';

import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';
import { useState } from 'react';

export default function Page() {
  const { messages, sendMessage, status, stop } = useChat({
    transport: new DefaultChatTransport({
      api: '/api/chat',
    }),
  });
  const [input, setInput] = useState('');

  return (
    <>
      {messages.map(message => (
        <div key={message.id}>
          {message.role === 'user' ? 'User: ' : 'AI: '}
          {message.parts.map((part, index) =>
            part.type === 'text' ? <span key={index}>{part.text}</span> : null,
          )}
        </div>
      ))}

      {(status === 'submitted' || status === 'streaming') && (
        <div>
          {status === 'submitted' && <Spinner />}
          <button type="button" onClick={() => stop()}>
            Stop
          </button>
        </div>
      )}

      <form
        onSubmit={e => {
          e.preventDefault();
          if (input.trim()) {
            sendMessage({ text: input });
            setInput('');
          }
        }}
      >
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          disabled={status !== 'ready'}
          placeholder="Say something..."
        />
        <button type="submit" disabled={status !== 'ready'}>
          Submit
        </button>
      </form>
    </>
  );
}
```

### Error State

Similarly, the `error` state reflects the error object thrown during the fetch request.
It can be used to display an error message, disable the submit button, or show a retry button:

We recommend showing a generic error message to the user, such as "Something
went wrong." This is a good practice to avoid leaking information from the
server.

```tsx title="app/page.tsx" {8,26-33,47}
'use client';

import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';
import { useState } from 'react';

export default function Chat() {
  const { messages, sendMessage, error, regenerate } = useChat({
    transport: new DefaultChatTransport({
      api: '/api/chat',
    }),
  });
  const [input, setInput] = useState('');

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}:{' '}
          {m.parts.map((part, index) =>
            part.type === 'text' ? <span key={index}>{part.text}</span> : null,
          )}
        </div>
      ))}

      {error && (
        <>
          <div>An error occurred.</div>
          <button type="button" onClick={() => regenerate()}>
            Retry
          </button>
        </>
      )}

      <form
        onSubmit={e => {
          e.preventDefault();
          if (input.trim()) {
            sendMessage({ text: input });
            setInput('');
          }
        }}
      >
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          disabled={error != null}
        />
      </form>
    </div>
  );
}
```

Please also see the [error handling](/docs/ai-sdk-ui/error-handling) guide for more information.

### Modify messages

Sometimes, you may want to directly modify some existing messages. For example, a delete button can be added to each message to allow users to remove them from the chat history.

The `setMessages` function can help you achieve these tasks:

```tsx
const { messages, setMessages } = useChat()

const handleDelete = (id) => {
  setMessages(messages.filter(message => message.id !== id))
}

return <>
  {messages.map(message => (
    <div key={message.id}>
      {message.role === 'user' ? 'User: ' : 'AI: '}
      {message.parts.map((part, index) => (
        part.type === 'text' ? (
          <span key={index}>{part.text}</span>
        ) : null
      ))}
      <button onClick={() => handleDelete(message.id)}>Delete</button>
    </div>
  ))}
  ...
```

You can think of `messages` and `setMessages` as a pair of `state` and `setState` in React.

### Cancellation and regeneration

It's also a common use case to abort the response message while it's still streaming back from the AI provider. You can do this by calling the `stop` function returned by the `useChat` hook.

```tsx
const { stop, status } = useChat()

return <>
  <button onClick={stop} disabled={!(status === 'streaming' || status === 'submitted')}>Stop</button>
  ...
```

When the user clicks the "Stop" button, the fetch request will be aborted. This avoids consuming unnecessary resources and improves the UX of your chatbot application.

Similarly, you can also request the AI provider to reprocess the last message by calling the `regenerate` function returned by the `useChat` hook:

```tsx
const { regenerate, status } = useChat();

return (
  <>
    <button
      onClick={regenerate}
      disabled={!(status === 'ready' || status === 'error')}
    >
      Regenerate
    </button>
    ...
  </>
);
```

When the user clicks the "Regenerate" button, the AI provider will regenerate the last message and replace the current one correspondingly.

### Throttling UI Updates

By default, `useChat` publishes message updates as each chunk is processed.
React may batch multiple updates into a single render.
React and Vue applications can throttle reactive message updates with the `throttle` option.
Stream processing and event callbacks remain immediate, and the latest messages are published before the chat enters a terminal `ready` or `error` status.

#### React

```tsx title="page.tsx" {2-3}
const { messages, ... } = useChat({
  // Throttle reactive message updates to 50ms:
  throttle: 50,
});
```

#### Vue

```vue title="pages/index.vue" {4-6}
<script setup lang="ts">
import { useChat } from '@ai-sdk/vue';

const { messages } = useChat({
  throttle: 50,
});
</script>
```

## Event Callbacks

`useChat` provides optional event callbacks that you can use to handle different stages of the chatbot lifecycle:

- `onFinish`: Called when the assistant response is completed. The event includes the response message, all messages, and flags for abort, disconnect, and errors.
- `onError`: Called when an error occurs during the fetch request.
- `onData`: Called whenever a data part is received.

These callbacks can be used to trigger additional actions, such as logging, analytics, or custom UI updates.

```tsx
import { UIMessage } from 'ai';

const {
  /* ... */
} = useChat({
  onFinish: ({ message, messages, isAbort, isDisconnect, isError }) => {
    // use information to e.g. update other UI states
  },
  onError: error => {
    console.error('An error occurred:', error);
  },
  onData: data => {
    console.log('Received data part from server:', data);
  },
});
```

It's worth noting that you can abort the processing by throwing an error in the `onData` callback. This will trigger the `onError` callback and stop the message from being appended to the chat UI. This can be useful for handling unexpected responses from the AI provider.

## Request Configuration

### Custom headers, body, and credentials

By default, the `useChat` hook sends an HTTP POST request to the `/api/chat` endpoint with the message list as the request body. You can customize the request in two ways:

#### Hook-Level Configuration (Applied to all requests)

You can configure transport-level options that will be applied to all requests made by the hook:

```tsx
import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';

const { messages, sendMessage } = useChat({
  transport: new DefaultChatTransport({
    api: '/api/custom-chat',
    headers: {
      Authorization: 'your_token',
    },
    body: {
      user_id: '123',
    },
    credentials: 'same-origin',
  }),
});
```

#### Dynamic Hook-Level Configuration

You can also provide functions that return configuration values. This is useful for authentication tokens that need to be refreshed, or for configuration that depends on runtime conditions:

```tsx
import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';

const { messages, sendMessage } = useChat({
  transport: new DefaultChatTransport({
    api: '/api/custom-chat',
    headers: () => ({
      Authorization: `Bearer ${getAuthToken()}`,
      'X-User-ID': getCurrentUserId(),
    }),
    body: () => ({
      sessionId: getCurrentSessionId(),
      preferences: getUserPreferences(),
    }),
    credentials: () => 'include',
  }),
});
```

For component state that changes over time, use `useRef` to store the current
value and reference `ref.current` in your configuration function, or prefer
request-level options (see next section) for better reliability.

#### Request-Level Configuration (Recommended)

**Recommended**: Use request-level options for better flexibility and control.
Request-level options take precedence over hook-level options and allow you to
customize each request individually.

```tsx
// Pass options as the second parameter to sendMessage
sendMessage(
  { text: input },
  {
    headers: {
      Authorization: 'Bearer token123',
      'X-Custom-Header': 'custom-value',
    },
    body: {
      temperature: 0.7,
      max_tokens: 100,
      user_id: '123',
    },
    metadata: {
      userId: 'user123',
      sessionId: 'session456',
    },
  },
);
```

The request-level options are merged with hook-level options, with request-level options taking precedence. On your server side, you can handle the request with this additional information.

### Setting custom body fields per request

You can configure custom `body` fields on a per-request basis using the second parameter of the `sendMessage` function.
This is useful if you want to pass in additional information to your backend that is not part of the message list.

```tsx title="app/page.tsx" {25-32}
'use client';

import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

export default function Chat() {
  const { messages, sendMessage } = useChat();
  const [input, setInput] = useState('');

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}:{' '}
          {m.parts.map((part, index) =>
            part.type === 'text' ? <span key={index}>{part.text}</span> : null,
          )}
        </div>
      ))}

      <form
        onSubmit={event => {
          event.preventDefault();
          if (input.trim()) {
            sendMessage(
              { text: input },
              {
                body: {
                  customKey: 'customValue',
                },
              },
            );
            setInput('');
          }
        }}
      >
        <input value={input} onChange={e => setInput(e.target.value)} />
      </form>
    </div>
  );
}
```

You can retrieve these custom fields on your server side by destructuring the request body:

```ts title="app/api/chat/route.ts" {3,4}
export async function POST(req: Request) {
  // Extract additional information ("customKey") from the body of the request:
  const { messages, customKey }: { messages: UIMessage[]; customKey: string } =
    await req.json();
  //...
}
```

## Message Metadata

You can attach custom metadata to messages for tracking information like timestamps, model details, and token usage.

```ts
// Server: Send metadata about the message
return createUIMessageStreamResponse({
  stream: toUIMessageStream({
    stream: result.stream,
    messageMetadata: ({ part }) => {
      if (part.type === 'start') {
        return {
          createdAt: Date.now(),
          model: 'gpt-6-astra',
        };
      }

      if (part.type === 'finish') {
        return {
          totalTokens: part.totalUsage.totalTokens,
        };
      }
    },
  }),
});
```

```tsx
// Client: Access metadata via message.metadata
{
  messages.map(message => (
    <div key={message.id}>
      {message.role}:{' '}
      {message.metadata?.createdAt &&
        new Date(message.metadata.createdAt).toLocaleTimeString()}
      {/* Render message content */}
      {message.parts.map((part, index) =>
        part.type === 'text' ? <span key={index}>{part.text}</span> : null,
      )}
      {/* Show token count if available */}
      {message.metadata?.totalTokens && (
        <span>{message.metadata.totalTokens} tokens</span>
      )}
    </div>
  ));
}
```

For complete examples with type safety and advanced use cases, see the [Message Metadata documentation](/docs/ai-sdk-ui/message-metadata).

## Transport Configuration

You can configure custom transport behavior using the `transport` option to customize how messages are sent to your API:

```tsx title="app/page.tsx"
import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';

export default function Chat() {
  const { messages, sendMessage } = useChat({
    id: 'my-chat',
    transport: new DefaultChatTransport({
      prepareSendMessagesRequest: ({ id, messages }) => {
        return {
          body: {
            id,
            message: messages[messages.length - 1],
          },
        };
      },
    }),
  });

  // ... rest of your component
}
```

The corresponding API route receives the custom request format:

```ts title="app/api/chat/route.ts"
export async function POST(req: Request) {
  const { id, message } = await req.json();

  // Load existing messages and add the new one
  const messages = await loadMessages(id);
  messages.push(message);

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

### Advanced: Trigger-based routing

For more complex scenarios like message regeneration, you can use trigger-based routing:

```tsx title="app/page.tsx"
import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';

export default function Chat() {
  const { messages, sendMessage, regenerate } = useChat({
    id: 'my-chat',
    transport: new DefaultChatTransport({
      prepareSendMessagesRequest: ({ id, messages, trigger, messageId }) => {
        if (trigger === 'submit-user-message') {
          return {
            body: {
              trigger: 'submit-user-message',
              id,
              message: messages[messages.length - 1],
              messageId,
            },
          };
        } else if (trigger === 'regenerate-assistant-message') {
          return {
            body: {
              trigger: 'regenerate-assistant-message',
              id,
              messageId,
            },
          };
        }
        throw new Error(`Unsupported trigger: ${trigger}`);
      },
    }),
  });

  // ... rest of your component
}
```

The corresponding API route would handle different triggers:

```ts title="app/api/chat/route.ts"
export async function POST(req: Request) {
  const { trigger, id, message, messageId } = await req.json();

  const chat = await readChat(id);
  let messages = chat.messages;

  if (trigger === 'submit-user-message') {
    // Handle new user message
    messages = [...messages, message];
  } else if (trigger === 'regenerate-assistant-message') {
    // Handle message regeneration - remove messages after messageId
    const messageIndex = messages.findIndex(m => m.id === messageId);
    if (messageIndex !== -1) {
      messages = messages.slice(0, messageIndex);
    }
  }

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

To learn more about building custom transports, refer to the [Transport API documentation](/docs/ai-sdk-ui/transport).

### Direct Agent Transport

For scenarios where you want to communicate directly with an Agent without going through HTTP, you can use `DirectChatTransport`. This is useful for:

- Server-side rendering scenarios
- Testing without network
- Single-process applications

```tsx title="app/page.tsx"
import { useChat } from '@ai-sdk/react';
import { DirectChatTransport, ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: 'You are a helpful assistant.',
});

export default function Chat() {
  const { messages, sendMessage, status } = useChat({
    transport: new DirectChatTransport({ agent }),
  });

  return (
    <>
      {messages.map(message => (
        <div key={message.id}>
          {message.role === 'user' ? 'User: ' : 'AI: '}
          {message.parts.map((part, index) =>
            part.type === 'text' ? <span key={index}>{part.text}</span> : null,
          )}
        </div>
      ))}

      <button
        onClick={() => sendMessage({ text: 'Hello!' })}
        disabled={status !== 'ready'}
      >
        Send
      </button>
    </>
  );
}
```

The `DirectChatTransport` invokes the agent's `stream()` method directly, converting UI messages to model messages and streaming the response back as UI message chunks.

For more details, see the [DirectChatTransport reference](/docs/reference/ai-sdk-ui/direct-chat-transport).

## Controlling the response stream

With `streamText`, you can control how error messages and usage information are sent back to the client.

### Error Messages

By default, the error message is masked for security reasons.
The default error message is "An error occurred."

Tool execution errors marked `providerExecuted: true` are an exception. They
bypass the UI stream's `onError` callback: strings are forwarded unchanged, and
other error values are serialized with `JSON.stringify`. This preserves provider
error codes when chat history is converted back into model messages, as well as
error messages from harness runtimes. Stream errors and invalid tool calls still
use `onError`, even for provider-executed tools. Do not rely on this callback to
redact provider-executed tool execution errors before they reach the client.

You can forward error messages or send your own error message by providing an `onError` function:

```ts title="app/api/chat/route.ts" {21-35}
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      onError: error => {
        if (error == null) {
          return 'unknown error';
        }

        if (typeof error === 'string') {
          return error;
        }

        if (error instanceof Error) {
          return error.message;
        }

        return JSON.stringify(error);
      },
    }),
  });
}
```

### Usage Information

Track token consumption and resource usage with [message metadata](/docs/ai-sdk-ui/message-metadata):

1. Define a custom metadata type with usage fields (optional, for type safety)
2. Attach usage data using `messageMetadata` in your response
3. Display usage metrics in your UI components

Usage data is attached as metadata to messages and becomes available once the model completes its response generation.

```ts
import { openai } from '@ai-sdk/openai';
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
  type LanguageModelUsage,
} from 'ai';

// Create a new metadata type (optional for type-safety)
type MyMetadata = {
  totalUsage: LanguageModelUsage;
};

// Create a new custom message type with your own metadata
export type MyUIMessage = UIMessage<MyMetadata>;

export async function POST(req: Request) {
  const { messages }: { messages: MyUIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      originalMessages: messages,
      messageMetadata: ({ part }) => {
        // Send total usage when generation is finished
        if (part.type === 'finish') {
          return { totalUsage: part.totalUsage };
        }
      },
    }),
  });
}
```

Then, on the client, you can access the message-level metadata.

```tsx
'use client';

import { useChat } from '@ai-sdk/react';
import type { MyUIMessage } from './api/chat/route';
import { DefaultChatTransport } from 'ai';

export default function Chat() {
  // Use custom message type defined on the server (optional for type-safety)
  const { messages } = useChat<MyUIMessage>({
    transport: new DefaultChatTransport({
      api: '/api/chat',
    }),
  });

  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {messages.map(m => (
        <div key={m.id} className="whitespace-pre-wrap">
          {m.role === 'user' ? 'User: ' : 'AI: '}
          {m.parts.map(part => {
            if (part.type === 'text') {
              return part.text;
            }
          })}
          {/* Render usage via metadata */}
          {m.metadata?.totalUsage && (
            <div>Total usage: {m.metadata?.totalUsage.totalTokens} tokens</div>
          )}
        </div>
      ))}
    </div>
  );
}
```

You can also access your metadata from the `onFinish` callback of `useChat`:

```tsx
'use client';

import { useChat } from '@ai-sdk/react';
import type { MyUIMessage } from './api/chat/route';
import { DefaultChatTransport } from 'ai';

export default function Chat() {
  // Use custom message type defined on the server (optional for type-safety)
  const { messages } = useChat<MyUIMessage>({
    transport: new DefaultChatTransport({
      api: '/api/chat',
    }),
    onFinish: ({ message }) => {
      // Access message metadata via onFinish callback
      console.log(message.metadata?.totalUsage);
    },
  });
}
```

### Text Streams

`useChat` can handle plain text streams by setting the `streamProtocol` option to `text`:

```tsx title="app/page.tsx" {7}
'use client';

import { useChat } from '@ai-sdk/react';
import { TextStreamChatTransport } from 'ai';

export default function Chat() {
  const { messages } = useChat({
    transport: new TextStreamChatTransport({
      api: '/api/chat',
    }),
  });

  return <>...</>;
}
```

This configuration also works with other backend servers that stream plain text.
Check out the [stream protocol guide](/docs/ai-sdk-ui/stream-protocol) for more information.

When using `TextStreamChatTransport`, tool calls, usage information and finish
reasons are not available.

## Reasoning

Some models such as DeepSeek `deepseek-r1`
and Anthropic `claude-sonnet-5-5` support reasoning tokens.
These tokens are typically sent before the message content.
You can forward them to the client with the `sendReasoning` option:

```ts title="app/api/chat/route.ts" {20}
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: 'deepseek/deepseek-r1',
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      sendReasoning: true,
    }),
  });
}
```

On the client side, you can access the reasoning parts of the message object.

Reasoning parts have a `text` property that contains the reasoning content.

```tsx title="app/page.tsx"
messages.map(message => (
  <div key={message.id}>
    {message.role === 'user' ? 'User: ' : 'AI: '}
    {message.parts.map((part, index) => {
      // text parts:
      if (part.type === 'text') {
        return <div key={index}>{part.text}</div>;
      }

      // reasoning parts:
      if (part.type === 'reasoning') {
        return <pre key={index}>{part.text}</pre>;
      }
    })}
  </div>
));
```

Some models may also produce files as part of reasoning (e.g. images).
These are available as `reasoning-file` parts (`ReasoningFileUIPart`) with
`mediaType` and `url` properties, similar to regular file parts.

## Sources

Some providers such as [Perplexity](/providers/ai-sdk-providers/perplexity#sources) and
[Google](/providers/ai-sdk-providers/google#sources) include sources in the response.

You can forward sources to the client with the `sendSources` option. This example uses the Perplexity Agent API's `fast` preset to search the web. Install `@ai-sdk/perplexity` and set `PERPLEXITY_API_KEY` in your environment:

```bash
pnpm add @ai-sdk/perplexity
```

```ts title="app/api/chat/route.ts" {21}
import { perplexity } from '@ai-sdk/perplexity';
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: perplexity('fast'),
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      sendSources: true,
    }),
  });
}
```

On the client side, you can access source parts of the message object.
There are two types of sources: `source-url` for web pages and `source-document` for documents.
Here is an example that renders both types of sources:

```tsx title="app/page.tsx"
messages.map(message => (
  <div key={message.id}>
    {message.role === 'user' ? 'User: ' : 'AI: '}

    {/* Render URL sources */}
    {message.parts
      .filter(part => part.type === 'source-url')
      .map(part => (
        <span key={`source-${part.sourceId}`}>
          [
          <a href={part.url} target="_blank">
            {part.title ?? new URL(part.url).hostname}
          </a>
          ]
        </span>
      ))}

    {/* Render document sources */}
    {message.parts
      .filter(part => part.type === 'source-document')
      .map(part => (
        <span key={`source-${part.sourceId}`}>
          [<span>{part.title ?? `Document ${part.sourceId}`}</span>]
        </span>
      ))}
  </div>
));
```

## Image Generation

Some models such as Google `gemini-3.1-flash-image-preview` support image generation.
When images are generated, they are exposed as files to the client.
On the client side, you can access file parts of the message object
and render them as images.

```tsx title="app/page.tsx"
messages.map(message => (
  <div key={message.id}>
    {message.role === 'user' ? 'User: ' : 'AI: '}
    {message.parts.map((part, index) => {
      if (part.type === 'text') {
        return <div key={index}>{part.text}</div>;
      } else if (part.type === 'file' && part.mediaType.startsWith('image/')) {
        return <img key={index} src={part.url} alt="Generated image" />;
      }
    })}
  </div>
));
```

## Attachments

The `useChat` hook supports sending file attachments along with a message as well as rendering them on the client. This can be useful for building applications that involve sending images, files, or other media content to the AI provider.

There are two ways to send files with a message: using a `FileList` object from file inputs or using an array of file objects.

### FileList

By using `FileList`, you can send multiple files as attachments along with a message using the file input element. The `useChat` hook will automatically convert them into data URLs and send them to the AI provider.

Currently, only `image/*` and `text/*` content types get automatically
converted into [multi-modal content
parts](/docs/foundations/prompts#file-parts). You will need to handle other
content types manually.

```tsx title="app/page.tsx"
'use client';

import { useChat } from '@ai-sdk/react';
import { useRef, useState } from 'react';

export default function Page() {
  const { messages, sendMessage, status } = useChat();

  const [input, setInput] = useState('');
  const [files, setFiles] = useState<FileList | undefined>(undefined);
  const fileInputRef = useRef<HTMLInputElement>(null);

  return (
    <div>
      <div>
        {messages.map(message => (
          <div key={message.id}>
            <div>{`${message.role}: `}</div>

            <div>
              {message.parts.map((part, index) => {
                if (part.type === 'text') {
                  return <span key={index}>{part.text}</span>;
                }

                if (
                  part.type === 'file' &&
                  part.mediaType?.startsWith('image/')
                ) {
                  return <img key={index} src={part.url} alt={part.filename} />;
                }

                return null;
              })}
            </div>
          </div>
        ))}
      </div>

      <form
        onSubmit={event => {
          event.preventDefault();
          if (input.trim()) {
            sendMessage({
              text: input,
              files,
            });
            setInput('');
            setFiles(undefined);

            if (fileInputRef.current) {
              fileInputRef.current.value = '';
            }
          }
        }}
      >
        <input
          type="file"
          onChange={event => {
            if (event.target.files) {
              setFiles(event.target.files);
            }
          }}
          multiple
          ref={fileInputRef}
        />
        <input
          value={input}
          placeholder="Send message..."
          onChange={e => setInput(e.target.value)}
          disabled={status !== 'ready'}
        />
      </form>
    </div>
  );
}
```

### File Objects

You can also send files as objects along with a message. This can be useful for sending pre-uploaded files or data URLs.

```tsx title="app/page.tsx"
'use client';

import { useChat } from '@ai-sdk/react';
import { useState } from 'react';
import { FileUIPart } from 'ai';

export default function Page() {
  const { messages, sendMessage, status } = useChat();

  const [input, setInput] = useState('');
  const [files] = useState<FileUIPart[]>([
    {
      type: 'file',
      filename: 'earth.png',
      mediaType: 'image/png',
      url: 'https://example.com/earth.png',
    },
    {
      type: 'file',
      filename: 'moon.png',
      mediaType: 'image/png',
      url: 'data:image/png;base64,iVBORw0KGgo...',
    },
  ]);

  return (
    <div>
      <div>
        {messages.map(message => (
          <div key={message.id}>
            <div>{`${message.role}: `}</div>

            <div>
              {message.parts.map((part, index) => {
                if (part.type === 'text') {
                  return <span key={index}>{part.text}</span>;
                }

                if (
                  part.type === 'file' &&
                  part.mediaType?.startsWith('image/')
                ) {
                  return <img key={index} src={part.url} alt={part.filename} />;
                }

                return null;
              })}
            </div>
          </div>
        ))}
      </div>

      <form
        onSubmit={event => {
          event.preventDefault();
          if (input.trim()) {
            sendMessage({
              text: input,
              files,
            });
            setInput('');
          }
        }}
      >
        <input
          value={input}
          placeholder="Send message..."
          onChange={e => setInput(e.target.value)}
          disabled={status !== 'ready'}
        />
      </form>
    </div>
  );
}
```

Files generated as part of model reasoning are available as `reasoning-file`
parts (`ReasoningFileUIPart`) with the same `mediaType` and `url` properties.

## Type Inference for Tools

When working with tools in TypeScript, AI SDK UI provides type inference helpers to ensure type safety for your tool inputs and outputs.

### InferUITool

The `InferUITool` type helper infers the input and output types of a single tool for use in UI messages:

```tsx
import { InferUITool } from 'ai';
import { z } from 'zod';

const weatherTool = {
  description: 'Get the current weather',
  inputSchema: z.object({
    location: z.string().describe('The city and state'),
  }),
  execute: async ({ location }) => {
    return `The weather in ${location} is sunny.`;
  },
};

// Infer the types from the tool
type WeatherUITool = InferUITool<typeof weatherTool>;
// This creates a type with:
// {
//   input: { location: string };
//   output: string;
// }
```

### InferUITools

The `InferUITools` type helper infers the input and output types of a `ToolSet`:

```tsx
import { InferUITools, ToolSet } from 'ai';
import { z } from 'zod';

const tools = {
  weather: {
    description: 'Get the current weather',
    inputSchema: z.object({
      location: z.string().describe('The city and state'),
    }),
    execute: async ({ location }) => {
      return `The weather in ${location} is sunny.`;
    },
  },
  calculator: {
    description: 'Perform basic arithmetic',
    inputSchema: z.object({
      operation: z.enum(['add', 'subtract', 'multiply', 'divide']),
      a: z.number(),
      b: z.number(),
    }),
    execute: async ({ operation, a, b }) => {
      switch (operation) {
        case 'add':
          return a + b;
        case 'subtract':
          return a - b;
        case 'multiply':
          return a * b;
        case 'divide':
          return a / b;
      }
    },
  },
} satisfies ToolSet;

// Infer the types from the tool set
type MyUITools = InferUITools<typeof tools>;
// This creates a type with:
// {
//   weather: { input: { location: string }; output: string };
//   calculator: { input: { operation: 'add' | 'subtract' | 'multiply' | 'divide'; a: number; b: number }; output: number };
// }
```

### Using Inferred Types

You can use these inferred types to create a custom UIMessage type and pass it to various AI SDK UI functions:

```tsx
import { InferUITools, UIMessage, UIDataTypes } from 'ai';

type MyUITools = InferUITools<typeof tools>;
type MyUIMessage = UIMessage<never, UIDataTypes, MyUITools>;
```

Pass the custom type to `useChat` or `createUIMessageStream`:

```tsx
import { useChat } from '@ai-sdk/react';
import { createUIMessageStream } from 'ai';
import type { MyUIMessage } from './types';

// With useChat
const { messages } = useChat<MyUIMessage>();

// With createUIMessageStream
const stream = createUIMessageStream<MyUIMessage>(/* ... */);
```

This provides full type safety for tool inputs and outputs on the client and server.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Completion
description: Learn how to use the useCompletion hook.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/completion"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

The `useCompletion` hook allows you to create a user interface to handle text completions in your application. It enables the streaming of text completions from your AI provider, manages the state for chat input, and updates the UI automatically as new messages are received.

The `useCompletion` hook is now part of the `@ai-sdk/react` package.

In this guide, you will learn how to use the `useCompletion` hook in your application to generate text completions and stream them in real-time to your users.

## Example

```tsx title='app/page.tsx'
'use client';

import { useCompletion } from '@ai-sdk/react';

export default function Page() {
  const { completion, input, handleInputChange, handleSubmit } = useCompletion({
    api: '/api/completion',
  });

  return (
    <form onSubmit={handleSubmit}>
      <input
        name="prompt"
        value={input}
        onChange={handleInputChange}
        id="input"
      />
      <button type="submit">Submit</button>
      <div>{completion}</div>
    </form>
  );
}
```

```ts title='app/api/completion/route.ts'
import {
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
} from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { prompt }: { prompt: string } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    prompt,
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

In the `Page` component, the `useCompletion` hook will request to your AI provider endpoint whenever the user submits a message. The completion is then streamed back in real-time and displayed in the UI.

This enables a seamless text completion experience where the user can see the AI response as soon as it is available, without having to wait for the entire response to be received.

## Customized UI

`useCompletion` also provides ways to manage the prompt via code, show loading and error states, and update messages without being triggered by user interactions.

### Loading and error states

To show a loading spinner while the chatbot is processing the user's message, you can use the `isLoading` state returned by the `useCompletion` hook:

```tsx
const { isLoading, ... } = useCompletion()

return(
  <>
    {isLoading ? <Spinner /> : null}
  </>
)
```

Similarly, the `error` state reflects the error object thrown during the fetch request. It can be used to display an error message, or show a toast notification:

```tsx
const { error, ... } = useCompletion()

useEffect(() => {
  if (error) {
    toast.error(error.message)
  }
}, [error])

// Or display the error message in the UI:
return (
  <>
    {error ? <div>{error.message}</div> : null}
  </>
)
```

### Controlled input

In the initial example, we have `handleSubmit` and `handleInputChange` callbacks that manage the input changes and form submissions. These are handy for common use cases, but you can also use uncontrolled APIs for more advanced scenarios such as form validation or customized components.

The following example demonstrates how to use more granular APIs like `setInput` with your custom input and submit button components:

```tsx
const { input, setInput } = useCompletion();

return (
  <>
    <MyCustomInput value={input} onChange={value => setInput(value)} />
  </>
);
```

### Cancelation

It's also a common use case to abort the response message while it's still streaming back from the AI provider. You can do this by calling the `stop` function returned by the `useCompletion` hook.

```tsx
const { stop, isLoading, ... } = useCompletion()

return (
  <>
    <button onClick={stop} disabled={!isLoading}>Stop</button>
  </>
)
```

When the user clicks the "Stop" button, the fetch request will be aborted. This avoids consuming unnecessary resources and improves the UX of your application.

### Throttling UI Updates

This feature is currently only available for React.

By default, the `useCompletion` hook will trigger a render every time a new chunk is received.
You can throttle the UI updates with the `throttle` option.

```tsx title="page.tsx" {2-3}
const { completion, ... } = useCompletion({
  // Throttle the completion and data updates to 50ms:
  throttle: 50
})
```

## Event Callbacks

`useCompletion` also provides optional event callbacks that you can use to handle different stages of the chatbot lifecycle. These callbacks can be used to trigger additional actions, such as logging, analytics, or custom UI updates.

```tsx
const { ... } = useCompletion({
  onFinish: (prompt: string, completion: string) => {
    console.log('Finished streaming completion:', completion)
  },
  onError: (error: Error) => {
    console.error('An error occurred:', error)
  },
})
```

## Configure Request Options

By default, the `useCompletion` hook sends an HTTP POST request to the `/api/completion` endpoint with the prompt as part of the request body. You can customize the request by passing additional options to the `useCompletion` hook:

```tsx
const { messages, input, handleInputChange, handleSubmit } = useCompletion({
  api: '/api/custom-completion',
  headers: {
    Authorization: 'your_token',
  },
  body: {
    user_id: '123',
  },
  credentials: 'same-origin',
});
```

In this example, the `useCompletion` hook sends a POST request to the `/api/custom-completion` endpoint with the specified headers, additional body fields, and credentials for that fetch request. On your server side, you can handle the request with these additional information.

You can provide a type for the additional body fields. The type applies to
both the body configured on the hook and the body passed to `complete`:

```tsx
type CompletionBody = {
  model: 'fast' | 'smart';
};

const { complete } = useCompletion<CompletionBody>({
  body: {
    model: 'fast',
  },
});

await complete('What is a completion?', {
  body: {
    model: 'smart',
  },
});
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Error Handling
description: Learn how to handle errors in the AI SDK UI
url: "https://ai-sdk.dev/docs/ai-sdk-ui/error-handling"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

## Warnings

The AI SDK shows warnings when something might not work as expected.
These warnings help you fix problems before they cause errors.

### When Warnings Appear

Warnings use Node.js `process.emitWarning` when available and `console.warn` in
browser/edge runtimes. They appear when:

- **Unsupported features**: You use a feature or setting that is not supported by the AI model (e.g., certain options or parameters).
- **Compatibility warnings**: A feature is used in a compatibility mode, which might work differently or less optimally than intended.
- **Deprecations**: You use a deprecated API, option, or message format.
- **Other warnings**: The AI model reports another type of issue, such as general problems or advisory messages.

### Warning Messages

All warnings include "AI SDK Warning" so you can easily find them. For example:

```
AI SDK Warning: The feature "temperature" is not supported by this model
```

### Turning Off Warnings

By default, warnings are shown in the console. You can control this behavior:

#### Turn Off All Warnings

Set a global variable to turn off warnings completely:

```ts
globalThis.AI_SDK_LOG_WARNINGS = false;
```

#### Custom Warning Handler

You can also provide your own function to handle warnings.
It receives a list of warnings and optional provider and model ids. Custom
handlers receive every warning, including repeated deprecations.

```ts
globalThis.AI_SDK_LOG_WARNINGS = ({ warnings, provider, model }) => {
  // Handle warnings your own way
};
```

If your application loads multiple AI SDK versions in the same runtime, they all
share `globalThis.AI_SDK_LOG_WARNINGS`. Each version calls the logger with its own
arguments and warning types. The merged global type declarations prevent
TypeScript declaration conflicts, but do not convert these arguments at runtime.
A handler typed for only one version may therefore fail when another version
calls it.

Import `LogWarningsFunction` from every installed version that can call the logger
and accept a union of their parameter types. For example, with AI SDK 7 installed
as `ai` and AI SDK 6 installed under the package alias `ai-v6` (`npm:ai@6`):

```ts
import type { LogWarningsFunction as LogWarningsFunctionV7 } from 'ai';
import type { LogWarningsFunction as LogWarningsFunctionV6 } from 'ai-v6';

type LoggerOptions =
  | Parameters<LogWarningsFunctionV7>[0]
  | Parameters<LogWarningsFunctionV6>[0];

const logger = ({ warnings, provider, model }: LoggerOptions) => {
  for (const warning of warnings) {
    console.warn('AI SDK Warning:', warning, { provider, model });
  }
};

globalThis.AI_SDK_LOG_WARNINGS = logger;
```

Check the argument shape and warning type before accessing version-specific
fields. If AI SDK 5 is also installed, include its
`Experimental_LogWarningsFunction` parameter type and use `Array.isArray` to
handle its warning array before accessing the options object used by AI SDK 6
and 7.

### Deprecation Warnings

The default logger emits each deprecation code once per loaded SDK instance.
In Node.js, these warnings have the type `DeprecationWarning` and a stable
`AISDK_DEP_*` code, available as `warning.code` on `process.on('warning', handler)`.
Browser/edge console messages include the code in brackets.

| Code                                     | Deprecated usage / replacement                                             |
| ---------------------------------------- | -------------------------------------------------------------------------- |
| `AISDK_DEP_GENERATE_OBJECT`              | Use `generateText` with `output` instead of `generateObject`.              |
| `AISDK_DEP_STREAM_OBJECT`                | Use `streamText` with `output` instead of `streamObject`.                  |
| `AISDK_DEP_EXPERIMENTAL_GENERATE_SPEECH` | Use `generateSpeech` instead of `experimental_generateSpeech`.             |
| `AISDK_DEP_EXPERIMENTAL_TRANSCRIBE`      | Use `transcribe` instead of `experimental_transcribe`.                     |
| `AISDK_DEP_TOOL_RESULT_*`                | Use `file` with tagged `data` instead of legacy tool-result content types. |
| `AISDK_DEP_UI_MESSAGE_RAW_INPUT`         | Use `input` instead of `rawInput` in UI tool parts.                        |
| `AISDK_DEP_IMAGE_CONTENT_PART`           | Use `file` with an image `mediaType` instead of `image` message parts.     |

Run Node.js with `--throw-deprecation` to fail on deprecated usage,
`--no-deprecation` to suppress deprecations, or `--trace-deprecation` for stack
traces. These flags affect all `DeprecationWarning`s, rather than individual
codes, and apply only to the default Node.js logger.

## Error Handling

### Error Helper Object

Each AI SDK UI hook also returns an [error](/docs/reference/ai-sdk-ui/use-chat#error) object that you can use to render the error in your UI.
You can use the error object to show an error message, disable the submit button, or show a retry button.

We recommend showing a generic error message to the user, such as "Something
went wrong." This is a good practice to avoid leaking information from the
server.

```tsx title="app/page.tsx" {8,28-35,41}
'use client';

import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

export default function Chat() {
  const [input, setInput] = useState('');
  const { messages, sendMessage, error, regenerate } = useChat();

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    sendMessage({ text: input });
    setInput('');
  };

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}:{' '}
          {m.parts
            .filter(part => part.type === 'text')
            .map(part => part.text)
            .join('')}
        </div>
      ))}

      {error && (
        <>
          <div>An error occurred.</div>
          <button type="button" onClick={() => regenerate()}>
            Retry
          </button>
        </>
      )}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          disabled={error != null}
        />
      </form>
    </div>
  );
}
```

#### Alternative: replace the failed message

Alternatively, you can write a custom submit handler that replaces the failed
user message with new input. If the assistant response started streaming before
the error, remove both the partial assistant response and its user message.

```tsx title="app/page.tsx" {13-19,21-22,39}
'use client';

import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

export default function Chat() {
  const [input, setInput] = useState('');
  const { sendMessage, error, messages, setMessages } = useChat();

  function customSubmit(event: React.FormEvent<HTMLFormElement>) {
    event.preventDefault();

    if (error != null) {
      setMessages(messages =>
        messages.at(-1)?.role === 'assistant'
          ? messages.slice(0, -2)
          : messages.slice(0, -1),
      );
    }

    sendMessage({ text: input });
    setInput('');
  }

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}:{' '}
          {m.parts
            .filter(part => part.type === 'text')
            .map(part => part.text)
            .join('')}
        </div>
      ))}

      {error && <div>An error occurred.</div>}

      <form onSubmit={customSubmit}>
        <input value={input} onChange={e => setInput(e.target.value)} />
      </form>
    </div>
  );
}
```

### Error Handling Callback

Errors can be processed by passing an [`onError`](/docs/reference/ai-sdk-ui/use-chat#on-error) callback function as an option to the [`useChat`](/docs/reference/ai-sdk-ui/use-chat) or [`useCompletion`](/docs/reference/ai-sdk-ui/use-completion) hooks.
The callback function receives an error object as an argument.

AI SDK-created client errors use exported error classes with marker-based
`.isInstance()` guards:

| Error class                                                                                         | AI SDK UI failure                                                                         |
| --------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| [`APICallError`](/docs/reference/ai-sdk-errors/ai-api-call-error)                                   | A chat transport or completion request returns a non-successful HTTP response.            |
| [`EmptyResponseBodyError`](/docs/reference/ai-sdk-errors/ai-empty-response-body-error)              | A successful chat transport or completion response has no body.                           |
| [`UIMessageStreamError`](/docs/reference/ai-sdk-errors/ai-ui-message-stream-error)                  | A completion data stream reports an error or a UI message stream contains invalid chunks. |
| [`InvalidArgumentError`](/docs/reference/ai-sdk-errors/ai-invalid-argument-error)                   | An invalid stream protocol or message ID is used.                                         |
| [`UnsupportedFunctionalityError`](/docs/reference/ai-sdk-errors/ai-unsupported-functionality-error) | A `FileList` is used in an environment that does not support it.                          |

Errors thrown by custom fetch implementations, callbacks, and stream parsers
continue to propagate unchanged.

```tsx title="app/page.tsx" {2,9-15}
import { useChat } from '@ai-sdk/react';
import { APICallError, EmptyResponseBodyError } from 'ai';

export default function Page() {
  const {
    /* ... */
  } = useChat({
    // handle error:
    onError: error => {
      if (APICallError.isInstance(error)) {
        console.error('Request failed with status:', error.statusCode);
      } else if (EmptyResponseBodyError.isInstance(error)) {
        console.error('The server returned no response body.');
      } else {
        console.error(error);
      }
    },
  });
}
```

For AI SDK UI requests, `APICallError.requestBodyValues` is `undefined` so
prompts and messages are not copied into client-facing error objects. The
response text remains available as `message` and `responseBody`; display a
generic message to users to avoid leaking server information.

### Injecting Errors for Testing

You might want to create errors for testing.
You can easily do so by throwing an error in your route handler:

```ts title="app/api/chat/route.ts"
export async function POST(req: Request) {
  throw new Error('This is a test error');
}
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Generative User Interfaces
description: Learn how to build Generative UI with AI SDK UI.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/generative-user-interfaces"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Generative user interfaces (generative UI) is the process of allowing a large language model (LLM) to go beyond text and "generate UI". This creates a more engaging and AI-native experience for users.

At the core of generative UI are [ tools ](/docs/ai-sdk-core/tools-and-tool-calling), which are functions you provide to the model to perform specialized tasks like getting the weather in a location. The model can decide when and how to use these tools based on the context of the conversation.

Generative UI is the process of connecting the results of a tool call to a React component. Here's how it works:

1. You provide the model with a prompt or conversation history, along with a set of tools.
2. Based on the context, the model may decide to call a tool.
3. If a tool is called, it will execute and return data.
4. This data can then be passed to a React component for rendering.

By passing the tool results to React components, you can create a generative UI experience that's more engaging and adaptive to your needs.

## Build a Generative UI Chat Interface

Let's create a chat interface that handles text-based conversations and incorporates dynamic UI elements based on model responses.

### Basic Chat Implementation

Start with a basic chat implementation using the `useChat` hook:

```tsx title="app/page.tsx"
'use client';

import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

export default function Page() {
  const [input, setInput] = useState('');
  const { messages, sendMessage } = useChat();

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    sendMessage({ text: input });
    setInput('');
  };

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role === 'user' ? 'User: ' : 'AI: '}</div>
          <div>
            {message.parts.map((part, index) => {
              if (part.type === 'text') {
                return <span key={index}>{part.text}</span>;
              }
              return null;
            })}
          </div>
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          placeholder="Type a message..."
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

To handle the chat requests and model responses, set up an API route:

```ts title="app/api/chat/route.ts"
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  isStepCount,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';

export async function POST(request: Request) {
  const { messages }: { messages: UIMessage[] } = await request.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    instructions: 'You are a friendly assistant!',
    messages: await convertToModelMessages(messages),
    stopWhen: isStepCount(5),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

This API route uses the `streamText` function to process chat messages and stream the model's responses back to the client.

### Create a Tool

Before enhancing your chat interface with dynamic UI elements, you need to create a tool and corresponding React component. A tool will allow the model to perform a specific action, such as fetching weather information.

Create a new file called `ai/tools.ts` with the following content:

```ts title="ai/tools.ts"
import { tool as createTool } from 'ai';
import { z } from 'zod';

export const weatherTool = createTool({
  description: 'Display the weather for a location',
  inputSchema: z.object({
    location: z.string().describe('The location to get the weather for'),
  }),
  execute: async function ({ location }) {
    await new Promise(resolve => setTimeout(resolve, 2000));
    return { weather: 'Sunny', temperature: 75, location };
  },
});

export const tools = {
  displayWeather: weatherTool,
};
```

In this file, you've created a tool called `weatherTool`. This tool simulates fetching weather information for a given location. This tool will return simulated data after a 2-second delay. In a real-world application, you would replace this simulation with an actual API call to a weather service.

### Update the API Route

Update the API route to include the tool you've defined:

```ts title="app/api/chat/route.ts" {10,15,20}
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  isStepCount,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';
import { tools } from '@/ai/tools';

export async function POST(request: Request) {
  const { messages }: { messages: UIMessage[] } = await request.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    instructions: 'You are a friendly assistant!',
    messages: await convertToModelMessages(messages),
    stopWhen: isStepCount(5),
    tools,
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

Now that you've defined the tool and added it to your `streamText` call, let's build a React component to display the weather information it returns.

### Create UI Components

Create a new file called `components/weather.tsx`:

```tsx title="components/weather.tsx"
type WeatherProps = {
  temperature: number;
  weather: string;
  location: string;
};

export const Weather = ({ temperature, weather, location }: WeatherProps) => {
  return (
    <div>
      <h2>Current Weather for {location}</h2>
      <p>Condition: {weather}</p>
      <p>Temperature: {temperature}°C</p>
    </div>
  );
};
```

This component will display the weather information for a given location. It takes three props: `temperature`, `weather`, and `location` (exactly what the `weatherTool` returns).

### Render the Weather Component

Now that you have your tool and corresponding React component, let's integrate them into your chat interface. You'll render the Weather component when the model calls the weather tool.

To check if the model has called a tool, you can check the `parts` array of the UIMessage object for tool-specific parts. In AI SDK 5.0, tool parts use typed naming: `tool-${toolName}` instead of generic types.

Update your `page.tsx` file:

```tsx title="app/page.tsx" {4,9,14-15,19-46}
'use client';

import { useChat } from '@ai-sdk/react';
import { useState } from 'react';
import { Weather } from '@/components/weather';

export default function Page() {
  const [input, setInput] = useState('');
  const { messages, sendMessage } = useChat();

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    sendMessage({ text: input });
    setInput('');
  };

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role === 'user' ? 'User: ' : 'AI: '}</div>
          <div>
            {message.parts.map((part, index) => {
              if (part.type === 'text') {
                return <span key={index}>{part.text}</span>;
              }

              if (part.type === 'tool-displayWeather') {
                switch (part.state) {
                  case 'input-available':
                    return <div key={index}>Loading weather...</div>;
                  case 'output-available':
                    return (
                      <div key={index}>
                        <Weather {...part.output} />
                      </div>
                    );
                  case 'output-error':
                    return <div key={index}>Error: {part.errorText}</div>;
                  default:
                    return null;
                }
              }

              return null;
            })}
          </div>
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          placeholder="Type a message..."
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

In this updated code snippet, you:

1. Use manual input state management with `useState` instead of the built-in `input` and `handleInputChange`.
2. Use `sendMessage` instead of `handleSubmit` to send messages.
3. Check the `parts` array of each message for different content types.
4. Handle tool parts with type `tool-displayWeather` and their different states (`input-available`, `output-available`, `output-error`).

This approach allows you to dynamically render UI components based on the model's responses, creating a more interactive and context-aware chat experience.

## Expanding Your Generative UI Application

You can enhance your chat application by adding more tools and components, creating a richer and more versatile user experience. Here's how you can expand your application:

### Adding More Tools

To add more tools, simply define them in your `ai/tools.ts` file:

```ts
// Add a new stock tool
export const stockTool = createTool({
  description: 'Get price for a stock',
  inputSchema: z.object({
    symbol: z.string().describe('The stock symbol to get the price for'),
  }),
  execute: async function ({ symbol }) {
    // Simulated API call
    await new Promise(resolve => setTimeout(resolve, 2000));
    return { symbol, price: 100 };
  },
});

// Update the tools object
export const tools = {
  displayWeather: weatherTool,
  getStockPrice: stockTool,
};
```

Now, create a new file called `components/stock.tsx`:

```tsx
type StockProps = {
  price: number;
  symbol: string;
};

export const Stock = ({ price, symbol }: StockProps) => {
  return (
    <div>
      <h2>Stock Information</h2>
      <p>Symbol: {symbol}</p>
      <p>Price: ${price}</p>
    </div>
  );
};
```

Finally, update your `page.tsx` file to include the new Stock component:

```tsx
'use client';

import { useChat } from '@ai-sdk/react';
import { useState } from 'react';
import { Weather } from '@/components/weather';
import { Stock } from '@/components/stock';

export default function Page() {
  const [input, setInput] = useState('');
  const { messages, sendMessage } = useChat();

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    sendMessage({ text: input });
    setInput('');
  };

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>{message.role}</div>
          <div>
            {message.parts.map((part, index) => {
              if (part.type === 'text') {
                return <span key={index}>{part.text}</span>;
              }

              if (part.type === 'tool-displayWeather') {
                switch (part.state) {
                  case 'input-available':
                    return <div key={index}>Loading weather...</div>;
                  case 'output-available':
                    return (
                      <div key={index}>
                        <Weather {...part.output} />
                      </div>
                    );
                  case 'output-error':
                    return <div key={index}>Error: {part.errorText}</div>;
                  default:
                    return null;
                }
              }

              if (part.type === 'tool-getStockPrice') {
                switch (part.state) {
                  case 'input-available':
                    return <div key={index}>Loading stock price...</div>;
                  case 'output-available':
                    return (
                      <div key={index}>
                        <Stock {...part.output} />
                      </div>
                    );
                  case 'output-error':
                    return <div key={index}>Error: {part.errorText}</div>;
                  default:
                    return null;
                }
              }

              return null;
            })}
          </div>
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          type="text"
          value={input}
          onChange={e => setInput(e.target.value)}
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}
```

By following this pattern, you can continue to add more tools and components, expanding the capabilities of your Generative UI application.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Message Metadata
description: Learn how to attach and use metadata with messages in AI SDK UI
url: "https://ai-sdk.dev/docs/ai-sdk-ui/message-metadata"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Message metadata allows you to attach custom information to messages at the message level. This is useful for tracking timestamps, model information, token usage, user context, and other message-level data.

## Overview

Message metadata differs from [data parts](/docs/ai-sdk-ui/streaming-data) in that it's attached at the message level rather than being part of the message content. While data parts are ideal for dynamic content that forms part of the message, metadata is perfect for information about the message itself.

## Getting Started

Here's a simple example of using message metadata to track timestamps and model information:

### Defining Metadata Types

First, define your metadata type for type safety:

```tsx title="app/types.ts"
import { UIMessage } from 'ai';
import { z } from 'zod';

// Define your metadata schema
export const messageMetadataSchema = z.object({
  createdAt: z.number().optional(),
  model: z.string().optional(),
  totalTokens: z.number().optional(),
});

export type MessageMetadata = z.infer<typeof messageMetadataSchema>;

// Create a typed UIMessage
export type MyUIMessage = UIMessage<MessageMetadata>;
```

### Sending Metadata from the Server

Use the `messageMetadata` callback in `toUIMessageStream` to send metadata at different streaming stages:

```ts title="app/api/chat/route.ts" {21-29,31-37}
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
} from 'ai';
import type { MyUIMessage } from '@/types';

export async function POST(req: Request) {
  const { messages }: { messages: MyUIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({
      stream: result.stream,
      originalMessages: messages, // pass this in for type-safe return objects
      messageMetadata: ({ part }) => {
        // Send metadata when streaming starts
        if (part.type === 'start') {
          return {
            createdAt: Date.now(),
            model: 'your-model-id',
          };
        }

        // Send additional metadata when streaming completes
        if (part.type === 'finish') {
          return {
            totalTokens: part.totalUsage.totalTokens,
          };
        }
      },
    }),
  });
}
```

To enable type-safe metadata return object in `messageMetadata`, pass in the
`originalMessages` parameter typed to your UIMessage type.

### Accessing Metadata on the Client

Access metadata through the `message.metadata` property:

```tsx title="app/page.tsx" {8,20-23,33-36}
'use client';

import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';
import type { MyUIMessage } from '@/types';

export default function Chat() {
  const { messages } = useChat<MyUIMessage>({
    transport: new DefaultChatTransport({
      api: '/api/chat',
    }),
  });

  return (
    <div>
      {messages.map(message => (
        <div key={message.id}>
          <div>
            {message.role === 'user' ? 'User: ' : 'AI: '}
            {message.metadata?.createdAt && (
              <span className="text-sm text-gray-500">
                {new Date(message.metadata.createdAt).toLocaleTimeString()}
              </span>
            )}
          </div>

          {/* Render message content */}
          {message.parts.map((part, index) =>
            part.type === 'text' ? <div key={index}>{part.text}</div> : null,
          )}

          {/* Display additional metadata */}
          {message.metadata?.totalTokens && (
            <div className="text-xs text-gray-400">
              {message.metadata.totalTokens} tokens
            </div>
          )}
        </div>
      ))}
    </div>
  );
}
```

For streaming arbitrary data that changes during generation, consider using
[data parts](/docs/ai-sdk-ui/streaming-data) instead.

## Common Use Cases

Message metadata is ideal for:

- **Timestamps**: When messages were created or completed
- **Model Information**: Which AI model was used
- **Token Usage**: Track costs and usage limits
- **User Context**: User IDs, session information
- **Performance Metrics**: Generation time, time to first token
- **Quality Indicators**: Finish reason, confidence scores

## See Also

- [Chatbot Guide](/docs/ai-sdk-ui/chatbot#message-metadata) - Message metadata in the context of building chatbots
- [Streaming Data](/docs/ai-sdk-ui/streaming-data#message-metadata-vs-data-parts) - Comparison with data parts
- [UIMessage Reference](/docs/reference/ai-sdk-core/ui-message) - Complete UIMessage type reference

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Object Generation
description: Learn how to use the useObject hook.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/object-generation"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

`useObject` is available in React, Svelte, Vue, and the SolidJS community
integration. Angular provides the equivalent `StructuredObject` service.

The [`useObject`](/docs/reference/ai-sdk-ui/use-object) hook allows you to create interfaces that represent a structured JSON object that is being streamed.

In this guide, you will learn how to use the `useObject` hook in your application to generate UIs for structured data on the fly.

## Example

The example shows a small notifications demo app that generates fake notifications in real-time.

### Schema

It is helpful to set up the schema in a separate file that is imported on both the client and server.

```ts title='app/api/notifications/schema.ts'
import { z } from 'zod';

// define a schema for the notifications
export const notificationSchema = z.object({
  notifications: z.array(
    z.object({
      name: z.string().describe('Name of a fictional person.'),
      message: z.string().describe('Message. Do not use emojis or links.'),
    }),
  ),
});
```

### Client

The client uses [`useObject`](/docs/reference/ai-sdk-ui/use-object) to stream the object generation process.

The results are partial and are displayed as they are received.
Please note the code for handling `undefined` values in the JSX.

```tsx title='app/page.tsx'
'use client';

import { useObject } from '@ai-sdk/react';
import { notificationSchema } from './api/notifications/schema';

export default function Page() {
  const { object, submit } = useObject({
    api: '/api/notifications',
    schema: notificationSchema,
  });

  return (
    <>
      <button onClick={() => submit('Messages during finals week.')}>
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </>
  );
}
```

### Server

On the server, we use [`streamText`](/docs/reference/ai-sdk-core/stream-text) with [`Output.object()`](/docs/reference/ai-sdk-core/output#output-object) to stream the object generation process.

```typescript title='app/api/notifications/route.ts'
import { createTextStreamResponse, Output, streamText, toTextStream } from 'ai';
import { notificationSchema } from './schema';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const context = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    output: Output.object({ schema: notificationSchema }),
    prompt:
      `Generate 3 notifications for a messages app in this context:` + context,
  });

  return createTextStreamResponse({
    stream: toTextStream({ stream: result.stream }),
  });
}
```

## Enum Output Mode

When you need to classify or categorize input into predefined options, you can use the `enum` output mode with `useObject`. This requires a specific schema structure where the object has `enum` as a key with `z.enum` containing your possible values.

### Example: Text Classification

This example shows how to build a simple text classifier that categorizes statements as true or false.

#### Client

When using `useObject` with enum output mode, your schema must be an object with `enum` as the key:

```tsx title='app/classify/page.tsx'
'use client';

import { useObject } from '@ai-sdk/react';
import { z } from 'zod';

export default function ClassifyPage() {
  const { object, submit, isLoading } = useObject({
    api: '/api/classify',
    schema: z.object({ enum: z.enum(['true', 'false']) }),
  });

  return (
    <>
      <button onClick={() => submit('The earth is flat')} disabled={isLoading}>
        Classify statement
      </button>

      {object && <div>Classification: {object.enum}</div>}
    </>
  );
}
```

#### Server

On the server, use `streamText` with `Output.choice()` to stream the classification result:

```typescript title='app/api/classify/route.ts'
import { createTextStreamResponse, Output, streamText, toTextStream } from 'ai';

export async function POST(req: Request) {
  const context = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    output: Output.choice({ options: ['true', 'false'] }),
    prompt: `Classify this statement as true or false: ${context}`,
  });

  return createTextStreamResponse({
    stream: toTextStream({ stream: result.stream }),
  });
}
```

## Customized UI

`useObject` also provides ways to show loading and error states:

### Loading State

The `isLoading` state returned by the `useObject` hook can be used for several
purposes:

- To show a loading spinner while the object is generated.
- To disable the submit button.

```tsx title='app/page.tsx' {6,13-20,24}
'use client';

import { useObject } from '@ai-sdk/react';

export default function Page() {
  const { isLoading, object, submit } = useObject({
    api: '/api/notifications',
    schema: notificationSchema,
  });

  return (
    <>
      {isLoading && <Spinner />}

      <button
        onClick={() => submit('Messages during finals week.')}
        disabled={isLoading}
      >
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </>
  );
}
```

### Stop Handler

The `stop` function can be used to stop the object generation process. This can be useful if the user wants to cancel the request or if the server is taking too long to respond.

```tsx title='app/page.tsx' {6,14-16}
'use client';

import { useObject } from '@ai-sdk/react';

export default function Page() {
  const { isLoading, stop, object, submit } = useObject({
    api: '/api/notifications',
    schema: notificationSchema,
  });

  return (
    <>
      {isLoading && (
        <button type="button" onClick={() => stop()}>
          Stop
        </button>
      )}

      <button onClick={() => submit('Messages during finals week.')}>
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </>
  );
}
```

### Error State

Similarly, the `error` state reflects the error object thrown during the fetch request.
It can be used to display an error message, or to disable the submit button:

We recommend showing a generic error message to the user, such as "Something
went wrong." This is a good practice to avoid leaking information from the
server.

```tsx title="app/page.tsx" {6,13}
'use client';

import { useObject } from '@ai-sdk/react';

export default function Page() {
  const { error, object, submit } = useObject({
    api: '/api/notifications',
    schema: notificationSchema,
  });

  return (
    <>
      {error && <div>An error occurred.</div>}

      <button onClick={() => submit('Messages during finals week.')}>
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </>
  );
}
```

## Event Callbacks

`useObject` provides optional event callbacks that you can use to handle life-cycle events.

- `onFinish`: Called when the object generation is completed.
- `onError`: Called when an error occurs during the fetch request.

These callbacks can be used to trigger additional actions, such as logging, analytics, or custom UI updates.

```tsx title='app/page.tsx' {10-20}
'use client';

import { useObject } from '@ai-sdk/react';
import { notificationSchema } from './api/notifications/schema';

export default function Page() {
  const { object, submit } = useObject({
    api: '/api/notifications',
    schema: notificationSchema,
    onFinish({ object, error }) {
      // typed object, undefined if schema validation fails:
      console.log('Object generation completed:', object);

      // error, undefined if schema validation succeeds:
      console.log('Schema validation error:', error);
    },
    onError(error) {
      // error during fetch request:
      console.error('An error occurred:', error);
    },
  });

  return (
    <div>
      <button onClick={() => submit('Messages during finals week.')}>
        Generate notifications
      </button>

      {object?.notifications?.map((notification, index) => (
        <div key={index}>
          <p>{notification?.name}</p>
          <p>{notification?.message}</p>
        </div>
      ))}
    </div>
  );
}
```

## Configure Request Options

You can configure the API endpoint, optional headers and credentials using the `api`, `headers` and `credentials` settings.

```tsx {2-5}
const { submit, object } = useObject({
  api: '/api/use-object',
  headers: {
    'X-Custom-Header': 'CustomValue',
  },
  credentials: 'include',
  schema: yourSchema,
});
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Overview
description: An overview of AI SDK UI.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/overview"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK UI is designed to help you build interactive chat, completion, and assistant applications with ease. It is a **framework-agnostic toolkit**, streamlining the integration of advanced AI functionalities into your applications.

AI SDK UI provides robust abstractions that simplify the complex tasks of managing chat streams and UI updates on the frontend, enabling you to develop dynamic AI-driven interfaces more efficiently. With three main hooks — **`useChat`**, **`useCompletion`**, and **`useObject`** — you can incorporate real-time chat capabilities, text completions, streamed JSON, and interactive assistant features into your app.

- **[`useChat`](/docs/ai-sdk-ui/chatbot)** offers real-time streaming of chat messages and manages message, loading, and error state. Manage input state in your component for seamless integration into any UI design.
- **[`useCompletion`](/docs/ai-sdk-ui/completion)** enables you to handle text completions in your applications, managing the prompt input and automatically updating the UI as new completions are streamed.
- **[`useObject`](/docs/ai-sdk-ui/object-generation)** is a hook that allows you to consume streamed JSON objects, providing a simple way to handle and display structured data in your application.

These hooks are designed to reduce the complexity and time required to implement AI interactions, letting you focus on creating exceptional user experiences.

## UI Framework Support

AI SDK UI supports the following frameworks: [React](https://react.dev/), [Svelte](https://svelte.dev/), [Vue.js](https://vuejs.org/),
[Angular](https://angular.dev/), and [SolidJS](https://www.solidjs.com/).

Here is a comparison of the supported functions across these frameworks:

|                                                                 | [useChat](/docs/reference/ai-sdk-ui/use-chat) | [useCompletion](/docs/reference/ai-sdk-ui/use-completion) | [useObject](/docs/reference/ai-sdk-ui/use-object) | [MCP Apps](/docs/ai-sdk-core/mcp-apps) |
| --------------------------------------------------------------- | --------------------------------------------- | --------------------------------------------------------- | ------------------------------------------------- | -------------------------------------- |
| React `@ai-sdk/react`                                           | ✓                                             | ✓                                                         | ✓                                                 | ✓                                      |
| Vue.js `@ai-sdk/vue`                                            | ✓                                             | ✓                                                         | ✓                                                 | ✗                                      |
| Svelte `@ai-sdk/svelte`                                         | ✓ Chat                                        | ✓ Completion                                              | ✓ StructuredObject                                | ✗                                      |
| Angular `@ai-sdk/angular`                                       | ✓ Chat                                        | ✓ Completion                                              | ✓ StructuredObject                                | ✗                                      |
| [SolidJS](https://github.com/kodehort/ai-sdk-solid) (community) | ✓                                             | ✓                                                         | ✓                                                 | ✗                                      |

## Framework Examples

Explore these example implementations for different frameworks:

- [**Next.js**](https://github.com/vercel/ai/tree/main/examples/ai-e2e-next)
- [**Nuxt**](https://github.com/vercel/ai/tree/main/examples/nuxt-openai)
- [**SvelteKit**](https://github.com/vercel/ai/tree/main/examples/sveltekit-openai)
- [**Angular**](https://github.com/vercel/ai/tree/main/examples/angular)

## API Reference

Please check out the [AI SDK UI API Reference](/docs/reference/ai-sdk-ui) for more details on each function.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Reading UIMessage Streams
description: Learn how to read UIMessage streams.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/reading-ui-message-streams"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

`UIMessage` streams are useful outside of traditional chat use cases. You can consume them for terminal UIs, custom stream processing on the client, or React Server Components (RSC).

The `readUIMessageStream` helper transforms a stream of `UIMessageChunk` objects into an `AsyncIterableStream` of `UIMessage` objects, allowing you to process messages as they're being constructed.

## Basic Usage

```tsx
import { readUIMessageStream, streamText, toUIMessageStream } from 'ai';

async function main() {
  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    prompt: 'Write a short story about a robot.',
  });

  for await (const uiMessage of readUIMessageStream({
    stream: toUIMessageStream({ stream: result.stream }),
  })) {
    console.log('Current message state:', uiMessage);
  }
}
```

## Tool Calls Integration

Handle streaming responses that include tool calls:

```tsx
import { readUIMessageStream, streamText, toUIMessageStream, tool } from 'ai';
import { z } from 'zod';

async function handleToolCalls() {
  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    tools: {
      weather: tool({
        description: 'Get the weather in a location',
        inputSchema: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: ({ location }) => ({
          location,
          temperature: 72 + Math.floor(Math.random() * 21) - 10,
        }),
      }),
    },
    prompt: 'What is the weather in Tokyo?',
  });

  for await (const uiMessage of readUIMessageStream({
    stream: toUIMessageStream({ stream: result.stream }),
  })) {
    // Handle different part types
    uiMessage.parts.forEach(part => {
      switch (part.type) {
        case 'text':
          console.log('Text:', part.text);
          break;
        case 'tool-call':
          console.log('Tool called:', part.toolName, 'with args:', part.args);
          break;
        case 'tool-result':
          console.log('Tool result:', part.result);
          break;
      }
    });
  }
}
```

## Resuming Conversations

Resume streaming from a previous message state:

```tsx
import { readUIMessageStream, streamText, toUIMessageStream } from 'ai';

async function resumeConversation(lastMessage: UIMessage) {
  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: [
      { role: 'user', content: 'Continue our previous conversation.' },
    ],
  });

  // Resume from the last message
  for await (const uiMessage of readUIMessageStream({
    stream: toUIMessageStream({ stream: result.stream }),
    message: lastMessage, // Resume from this message
  })) {
    console.log('Resumed message:', uiMessage);
  }
}
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Stream Protocols
description: Learn more about the supported stream protocols in the AI SDK.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/stream-protocol"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

AI SDK UI functions such as `useChat` and `useCompletion` support both text streams and data streams.
The stream protocol defines how the data is streamed to the frontend on top of the HTTP protocol.

This page describes both protocols and how to use them in the backend and frontend.

You can use this information to develop custom backends and frontends for your use case, e.g.,
to provide compatible API endpoints that are implemented in a different language such as Python.

For instance, here's an example using [FastAPI](https://github.com/vercel/ai/tree/main/examples/next-fastapi) as a backend.

## Text Stream Protocol

A text stream contains chunks in plain text, that are streamed to the frontend.
Each chunk is then appended together to form a full text response.

Text streams are supported by `useChat`, `useCompletion`, and `useObject`.
To enable text streaming, use `TextStreamChatTransport` with `useChat`, or set
the `streamProtocol` option to `text` with `useCompletion`.

You can generate text streams with `streamText` in the backend.
Pass the result's `stream` to `toTextStream` and return it with
`createTextStreamResponse` to create a streaming HTTP response.

Text streams only support basic text data. If you need to stream other types
of data such as tool calls, use data streams.

### Text Stream Example

Here is a Next.js example that uses the text stream protocol:

```tsx title='app/page.tsx'
'use client';

import { useChat } from '@ai-sdk/react';
import { TextStreamChatTransport } from 'ai';
import { useState } from 'react';

export default function Chat() {
  const [input, setInput] = useState('');
  const { messages, sendMessage } = useChat({
    transport: new TextStreamChatTransport({ api: '/api/chat' }),
  });

  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {messages.map(message => (
        <div key={message.id} className="whitespace-pre-wrap">
          {message.role === 'user' ? 'User: ' : 'AI: '}
          {message.parts.map((part, i) => {
            switch (part.type) {
              case 'text':
                return <div key={`${message.id}-${i}`}>{part.text}</div>;
            }
          })}
        </div>
      ))}

      <form
        onSubmit={e => {
          e.preventDefault();
          sendMessage({ text: input });
          setInput('');
        }}
      >
        <input
          className="fixed dark:bg-zinc-900 bottom-0 w-full max-w-md p-2 mb-8 border border-zinc-300 dark:border-zinc-800 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={e => setInput(e.currentTarget.value)}
        />
      </form>
    </div>
  );
}
```

```ts title='app/api/chat/route.ts'
import {
  convertToModelMessages,
  createTextStreamResponse,
  streamText,
  toTextStream,
  UIMessage,
} from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
  });

  return createTextStreamResponse({
    stream: toTextStream({ stream: result.stream }),
  });
}
```

## Data Stream Protocol

A data stream follows a special protocol that the AI SDK provides to send information to the frontend.

The data stream protocol uses Server-Sent Events (SSE) format for improved standardization, keep-alive through ping, reconnect capabilities, and better cache handling.

When you provide data streams from a custom backend, you need to set the
`x-vercel-ai-ui-message-stream` header to `v1`.

The following stream parts are currently supported:

### Message Start Part

Indicates the beginning of a new message with metadata.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"start","messageId":"..."}

```

### Text Parts

Text content is streamed using a start/delta/end pattern with unique IDs for each text block.

#### Text Start Part

Indicates the beginning of a text block.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"text-start","id":"msg_68679a454370819ca74c8eb3d04379630dd1afb72306ca5d"}

```

#### Text Delta Part

Contains incremental text content for the text block.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"text-delta","id":"msg_68679a454370819ca74c8eb3d04379630dd1afb72306ca5d","delta":"Hello"}

```

#### Text End Part

Indicates the completion of a text block.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"text-end","id":"msg_68679a454370819ca74c8eb3d04379630dd1afb72306ca5d"}

```

### Reasoning Parts

Reasoning content is streamed using a start/delta/end pattern with unique IDs for each reasoning block.

#### Reasoning Start Part

Indicates the beginning of a reasoning block.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"reasoning-start","id":"reasoning_123"}

```

#### Reasoning Delta Part

Contains incremental reasoning content for the reasoning block.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"reasoning-delta","id":"reasoning_123","delta":"This is some reasoning"}

```

#### Reasoning End Part

Indicates the completion of a reasoning block.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"reasoning-end","id":"reasoning_123"}

```

### Reasoning File Part

Reasoning file parts contain references to files generated as part of reasoning, such as images produced during the reasoning process.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"reasoning-file","url":"data:image/png;base64,iVBOR...","mediaType":"image/png"}

```

### Source Parts

Source parts provide references to external content sources.

#### Source URL Part

References to external URLs.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"source-url","sourceId":"https://example.com","url":"https://example.com"}

```

#### Source Document Part

References to documents or files.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"source-document","sourceId":"https://example.com","mediaType":"file","title":"Title"}

```

### File Part

The file parts contain references to files with their media type.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"file","url":"https://example.com/file.png","mediaType":"image/png"}

```

### Custom Part

Custom parts represent provider-specific content that doesn't fit into the standard part types. The `kind` field identifies the specific custom content type in the format `{provider}.{provider-type}`.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"custom","kind":"openai.compaction","providerMetadata":{"openai":{"itemId":"cmp_123"}}}

```

### Data Parts

Custom data parts allow streaming of arbitrary structured data with type-specific handling.

Format: Server-Sent Event with JSON object where the type includes a custom suffix

Example:

```
data: {"type":"data-weather","data":{"location":"SF","temperature":100}}

```

The `data-*` type pattern allows you to define custom data types that your frontend can handle specifically.

### Error Part

The error parts are appended to the message as they are received.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"error","errorText":"error message"}

```

### Tool Input Start Part

Indicates the beginning of tool input streaming.

The optional `inputEncoding` field specifies how to interpret subsequent input
deltas. When omitted or set to `'json'`, the UI parses the accumulated text as
partial JSON. With `'text'`, the UI uses the accumulated text directly as the
string input. Use `'text'` for raw custom-tool input such as code or HTML.
The encoding is also retained on persisted `input-streaming` tool parts so
streaming can resume correctly.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"tool-input-start","toolCallId":"call_fJdQDqnXeGxTmr4E3YPSR7Ar","toolName":"getWeatherInformation"}

```

### Tool Input Delta Part

Incremental chunks of tool input as it's being generated.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"tool-input-delta","toolCallId":"call_fJdQDqnXeGxTmr4E3YPSR7Ar","inputTextDelta":"San Francisco"}

```

### Tool Input Available Part

Indicates that tool input is complete and ready for execution.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"tool-input-available","toolCallId":"call_fJdQDqnXeGxTmr4E3YPSR7Ar","toolName":"getWeatherInformation","input":{"city":"San Francisco"}}

```

### Tool Approval Request Part

Indicates that a tool call requires approval, or records that the approval decision was made automatically.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"tool-approval-request","toolCallId":"call_fJdQDqnXeGxTmr4E3YPSR7Ar","approvalId":"approval_123","approvalDescriptor":{"scope":"account:delete"},"reason":"Requires operator review"}

```

When `isAutomatic` is omitted, the request expects an explicit approval response
from the client. `reason` is optional and explains why the tool call requires
approval. `approvalDescriptor` is optional opaque metadata for the approval.
When the stream is processed into UI messages, it is available as
`part.approval.descriptor` and is retained through subsequent approval states.

### Tool Approval Response Part

Records the approval decision for a tool call.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"tool-approval-response","approvalId":"approval_123","approved":false,"reason":"User denied the request"}

```

For provider-executed tools, the response can also include `providerExecuted: true`.

### Tool Output Available Part

Contains the result of tool execution.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"tool-output-available","toolCallId":"call_fJdQDqnXeGxTmr4E3YPSR7Ar","output":{"city":"San Francisco","weather":"sunny"}}

```

### Tool Output Denied Part

Indicates that tool execution was denied after the approval flow completed.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"tool-output-denied","toolCallId":"call_fJdQDqnXeGxTmr4E3YPSR7Ar"}

```

### Start Step Part

A part indicating the start of a step.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"start-step"}

```

### Finish Step Part

A part indicating that a step (i.e., one LLM API call in the backend) has been completed.

This part is necessary to correctly process multiple stitched assistant calls, e.g. when calling tools in the backend, and using steps in `useChat` at the same time.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"finish-step"}

```

### Reset Step Part

Removes all message parts received since the most recent `start-step` part. If
there is no step boundary, it removes all parts from the current message. This
is useful when a streamed step is retried and partial output from the failed
attempt must be invalidated before replacement output is sent.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"reset-step"}

```

### Finish Message Part

A part indicating the completion of a message.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"finish"}

```

### Abort Part

Indicates the stream was aborted.

Format: Server-Sent Event with JSON object

Example:

```
data: {"type":"abort","reason":"user cancelled"}

```

### Stream Termination

The stream ends with a special `[DONE]` marker.

Format: Server-Sent Event with literal `[DONE]`

Example:

```
data: [DONE]

```

The data stream protocol is supported
by `useChat` and `useCompletion` on the frontend and used by default.
`useCompletion` only supports the `text` and `data` stream parts.

On the backend, you can pass the `streamText` result stream to `toUIMessageStream` and return it with `createUIMessageStreamResponse`.

### UI Message Stream Example

Here is a Next.js example that uses the UI message stream protocol:

```tsx title='app/page.tsx'
'use client';

import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

export default function Chat() {
  const [input, setInput] = useState('');
  const { messages, sendMessage } = useChat();

  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {messages.map(message => (
        <div key={message.id} className="whitespace-pre-wrap">
          {message.role === 'user' ? 'User: ' : 'AI: '}
          {message.parts.map((part, i) => {
            switch (part.type) {
              case 'text':
                return <div key={`${message.id}-${i}`}>{part.text}</div>;
            }
          })}
        </div>
      ))}

      <form
        onSubmit={e => {
          e.preventDefault();
          sendMessage({ text: input });
          setInput('');
        }}
      >
        <input
          className="fixed dark:bg-zinc-900 bottom-0 w-full max-w-md p-2 mb-8 border border-zinc-300 dark:border-zinc-800 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={e => setInput(e.currentTarget.value)}
        />
      </form>
    </div>
  );
}
```

```ts title='app/api/chat/route.ts'
import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  UIMessage,
} from 'ai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5.5",
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}
```

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Streaming Custom Data
description: Learn how to stream custom data from the server to the client.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/streaming-data"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

It is often useful to send additional data alongside the model's response.
For example, you may want to send status information, the message ids after storing them,
or references to content that the language model is referring to.

The AI SDK provides several helpers that allows you to stream additional data to the client
and attach it to the `UIMessage` parts array:

- `createUIMessageStream`: creates a data stream
- `createUIMessageStreamResponse`: creates a response object that streams data
- `pipeUIMessageStreamToResponse`: pipes a data stream to a server response object

The data is streamed as part of the response stream using Server-Sent Events.

## Setting Up Type-Safe Data Streaming

First, define your custom message type with data part schemas for type safety:

```tsx title="ai/types.ts"
import { UIMessage } from 'ai';

// Define your custom message type with data part schemas
export type MyUIMessage = UIMessage<
  never, // metadata type
  {
    weather: {
      city: string;
      weather?: string;
      status: 'loading' | 'success';
    };
    notification: {
      message: string;
      level: 'info' | 'warning' | 'error';
    };
  } // data parts type
>;
```

## Streaming Data from the Server

In your server-side route handler, you can create a `UIMessageStream` and then pass it to `createUIMessageStreamResponse`:

```tsx title="route.ts"
import { openai } from '@ai-sdk/openai';
import {
  convertToModelMessages,
  createUIMessageStream,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
} from 'ai';
import type { MyUIMessage } from '@/ai/types';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const stream = createUIMessageStream<MyUIMessage>({
    execute: ({ writer }) => {
      // 1. Start the assistant message before writing any message parts.
      writer.write({ type: 'start' });

      // 2. Send initial status (transient - won't be added to message history)
      writer.write({
        type: 'data-notification',
        data: { message: 'Processing your request...', level: 'info' },
        transient: true, // This part won't be added to message history
      });

      // 3. Send sources (useful for RAG use cases)
      writer.write({
        type: 'source',
        value: {
          type: 'source',
          sourceType: 'url',
          id: 'source-1',
          url: 'https://weather.com',
          title: 'Weather Data Source',
        },
      });

      // 4. Send data parts with loading state
      writer.write({
        type: 'data-weather',
        id: 'weather-1',
        data: { city: 'San Francisco', status: 'loading' },
      });

      const result = streamText({
        model: "anthropic/claude-sonnet-5.5",
        messages: await convertToModelMessages(messages),
        onEnd() {
          // 5. Update the same data part (reconciliation)
          writer.write({
            type: 'data-weather',
            id: 'weather-1', // Same ID = update existing part
            data: {
              city: 'San Francisco',
              weather: 'sunny',
              status: 'success',
            },
          });

          // 6. Send completion notification (transient)
          writer.write({
            type: 'data-notification',
            data: { message: 'Request completed', level: 'info' },
            transient: true, // Won't be added to message history
          });
        },
      });

      writer.merge(
        toUIMessageStream({ stream: result.stream, sendStart: false }),
      );
    },
  });

  return createUIMessageStreamResponse({ stream });
}
```

You can also send stream data from custom backends, e.g. Python / FastAPI,
using the [UI Message Stream
Protocol](/docs/ai-sdk-ui/stream-protocol#data-stream-protocol).

## Types of Streamable Data

### Data Parts (Persistent)

Regular data parts are added to the message history and appear in `message.parts`:

```tsx
writer.write({
  type: 'data-weather',
  id: 'weather-1', // Optional: enables reconciliation
  data: { city: 'San Francisco', status: 'loading' },
});
```

### Sources

Sources are useful for RAG implementations where you want to show which documents or URLs were referenced:

```tsx
writer.write({
  type: 'source',
  value: {
    type: 'source',
    sourceType: 'url',
    id: 'source-1',
    url: 'https://example.com',
    title: 'Example Source',
  },
});
```

### Transient Data Parts (Ephemeral)

Transient parts are sent to the client but not added to the message history. They are only accessible via the `onData` useChat handler:

```tsx
// server
writer.write({
  type: 'data-notification',
  data: { message: 'Processing...', level: 'info' },
  transient: true, // Won't be added to message history
});

// client
const [notification, setNotification] = useState();

const { messages } = useChat({
  onData: ({ data, type }) => {
    if (type === 'data-notification') {
      setNotification({ message: data.message, level: data.level });
    }
  },
});
```

## Data Part Reconciliation

When you write to a data part with the same ID, the client automatically reconciles and updates that part. This enables powerful dynamic experiences like:

- **Collaborative artifacts** - Update code, documents, or designs in real-time
- **Progressive data loading** - Show loading states that transform into final results
- **Live status updates** - Update progress bars, counters, or status indicators
- **Interactive components** - Build UI elements that evolve based on user interaction

The reconciliation happens automatically - simply use the same `id` when writing to the stream.

## Processing Data on the Client

### Using the onData Callback

The `onData` callback is essential for handling streaming data, especially transient parts:

```tsx title="page.tsx"
import { useChat } from '@ai-sdk/react';
import type { MyUIMessage } from '@/ai/types';

const { messages } = useChat<MyUIMessage>({
  api: '/api/chat',
  onData: dataPart => {
    // Handle all data parts as they arrive (including transient parts)
    console.log('Received data part:', dataPart);

    // Handle different data part types
    if (dataPart.type === 'data-weather') {
      console.log('Weather update:', dataPart.data);
    }

    // Handle transient notifications (ONLY available here, not in message.parts)
    if (dataPart.type === 'data-notification') {
      showToast(dataPart.data.message, dataPart.data.level);
    }
  },
});
```

**Important:** Transient data parts are **only** available through the `onData` callback. They will not appear in the `message.parts` array since they're not added to message history.

### Rendering Persistent Data Parts

You can filter and render data parts from the message parts array:

```tsx title="page.tsx"
const result = (
  <>
    {messages?.map(message => (
      <div key={message.id}>
        {/* Render weather data parts */}
        {message.parts
          .filter(part => part.type === 'data-weather')
          .map((part, index) => (
            <div key={index} className="weather-widget">
              {part.data.status === 'loading' ? (
                <>Getting weather for {part.data.city}...</>
              ) : (
                <>
                  Weather in {part.data.city}: {part.data.weather}
                </>
              )}
            </div>
          ))}

        {/* Render text content */}
        {message.parts
          .filter(part => part.type === 'text')
          .map((part, index) => (
            <div key={index}>{part.text}</div>
          ))}

        {/* Render sources */}
        {message.parts
          .filter(part => part.type === 'source')
          .map((part, index) => (
            <div key={index} className="source">
              Source: <a href={part.url}>{part.title}</a>
            </div>
          ))}
      </div>
    ))}
  </>
);
```

### Complete Example

```tsx title="page.tsx"
'use client';

import { useChat } from '@ai-sdk/react';
import { useState } from 'react';
import type { MyUIMessage } from '@/ai/types';

export default function Chat() {
  const [input, setInput] = useState('');

  const { messages, sendMessage } = useChat<MyUIMessage>({
    api: '/api/chat',
    onData: dataPart => {
      // Handle transient notifications
      if (dataPart.type === 'data-notification') {
        console.log('Notification:', dataPart.data.message);
      }
    },
  });

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    sendMessage({ text: input });
    setInput('');
  };

  return (
    <>
      {messages?.map(message => (
        <div key={message.id}>
          {message.role === 'user' ? 'User: ' : 'AI: '}

          {/* Render weather data */}
          {message.parts
            .filter(part => part.type === 'data-weather')
            .map((part, index) => (
              <span key={index} className="weather-update">
                {part.data.status === 'loading' ? (
                  <>Getting weather for {part.data.city}...</>
                ) : (
                  <>
                    Weather in {part.data.city}: {part.data.weather}
                  </>
                )}
              </span>
            ))}

          {/* Render text content */}
          {message.parts
            .filter(part => part.type === 'text')
            .map((part, index) => (
              <div key={index}>{part.text}</div>
            ))}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={e => setInput(e.target.value)}
          placeholder="Ask about the weather..."
        />
        <button type="submit">Send</button>
      </form>
    </>
  );
}
```

## Use Cases

- **RAG Applications** - Stream sources and retrieved documents
- **Real-time Status** - Show loading states and progress updates
- **Collaborative Tools** - Stream live updates to shared artifacts
- **Analytics** - Send usage data without cluttering message history
- **Notifications** - Display temporary alerts and status messages

## Message Metadata vs Data Parts

Both [message metadata](/docs/ai-sdk-ui/message-metadata) and data parts allow you to send additional information alongside messages, but they serve different purposes:

### Message Metadata

Message metadata is best for **message-level information** that describes the message as a whole:

- Attached at the message level via `message.metadata`
- Sent using the `messageMetadata` callback in `toUIMessageStream`
- Ideal for: timestamps, model info, token usage, user context
- Type-safe with custom metadata types

```ts
// Server: Send metadata about the message
return createUIMessageStreamResponse({
  stream: toUIMessageStream({
    stream: result.stream,
    messageMetadata: ({ part }) => {
      if (part.type === 'finish') {
        return {
          model: part.response.modelId,
          totalTokens: part.totalUsage.totalTokens,
          createdAt: Date.now(),
        };
      }
    },
  }),
});
```

### Data Parts

Data parts are best for streaming **dynamic arbitrary data**:

- Added to the message parts array via `message.parts`
- Streamed using `createUIMessageStream` and `writer.write()`
- Can be reconciled/updated using the same ID
- Support transient parts that don't persist
- Ideal for: dynamic content, loading states, interactive components

```ts
// Server: Stream data as part of message content
writer.write({
  type: 'data-weather',
  id: 'weather-1',
  data: { city: 'San Francisco', status: 'loading' },
});
```

For more details on message metadata, see the [Message Metadata documentation](/docs/ai-sdk-ui/message-metadata).

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Transport
description: Learn how to use custom transports with useChat.
url: "https://ai-sdk.dev/docs/ai-sdk-ui/transport"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

The `useChat` transport system provides fine-grained control over how messages are sent to your API endpoints and how responses are processed. This is particularly useful for alternative communication protocols like WebSockets, custom authentication patterns, or specialized backend integrations.

## Default Transport

By default, `useChat` uses HTTP POST requests to send messages to `/api/chat`:

```tsx
import { useChat } from '@ai-sdk/react';

// Uses default HTTP transport
const { messages, sendMessage } = useChat();
```

This is equivalent to:

```tsx
import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';

const { messages, sendMessage } = useChat({
  transport: new DefaultChatTransport({
    api: '/api/chat',
  }),
});
```

## Custom Transport Configuration

Configure the default transport with custom options:

```tsx
import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';

const { messages, sendMessage } = useChat({
  transport: new DefaultChatTransport({
    api: '/api/custom-chat',
    headers: {
      Authorization: 'Bearer your-token',
      'X-API-Version': '2024-01',
    },
    credentials: 'include',
  }),
});
```

### Dynamic Configuration

You can also provide functions that return configuration values. This is useful for authentication tokens that need to be refreshed, or for configuration that depends on runtime conditions:

```tsx
const { messages, sendMessage } = useChat({
  transport: new DefaultChatTransport({
    api: '/api/chat',
    headers: () => ({
      Authorization: `Bearer ${getAuthToken()}`,
      'X-User-ID': getCurrentUserId(),
    }),
    body: () => ({
      sessionId: getCurrentSessionId(),
      preferences: getUserPreferences(),
    }),
    credentials: () => 'include',
  }),
});
```

### Request Transformation

Transform requests before sending to your API:

```tsx
const { messages, sendMessage } = useChat({
  transport: new DefaultChatTransport({
    api: '/api/chat',
    prepareSendMessagesRequest: ({ id, messages, trigger, messageId }) => {
      return {
        headers: {
          'X-Session-ID': id,
        },
        body: {
          messages: messages.slice(-10), // Only send last 10 messages
          trigger,
          messageId,
        },
      };
    },
  }),
});
```

## WebSocket Transport

Use `WebSocketChatTransport` when you want chat turns to share a persistent
WebSocket connection instead of opening an HTTP request for every turn. This is
useful for chats with frequent client-side tool results or other multi-step
interactions.

```tsx
import { useChat } from '@ai-sdk/react';
import { WebSocketChatTransport } from 'ai';
import { useMemo } from 'react';

export default function Chat() {
  const transport = useMemo(
    () =>
      new WebSocketChatTransport({
        url: '/api/chat',
        params: () => ({ token: getShortLivedToken() }),
      }),
    [],
  );

  const { messages, sendMessage } = useChat({ transport });

  // ... render the chat
}
```

The transport multiplexes requests by `requestId`, validates incoming
`UIMessageChunk` values, forwards aborts to the server, and reuses the open
socket for later turns. HTTP and HTTPS URLs are automatically converted to WS
and WSS URLs.

The code that creates a transport owns its lifecycle. React and Vue chat hooks
do not close caller-supplied transports when options change or components are
disposed, because the same transport can be shared by multiple chats. Call
`transport.close()` after its last consumer is gone. If a directly managed
`Chat` instance exclusively owns the transport, call `chat.dispose()`; for a
shared transport, stop individual chats and close the transport only after all
consumers are finished.

Your WebSocket server must implement the
[`WebSocketChatTransport` protocol](/docs/reference/ai-sdk-ui/websocket-chat-transport#wire-protocol).
The exported `WebSocketChatTransportRequest` and
`WebSocketChatTransportResponse` types can be used to type server frames. Use
`safeValidateWebSocketChatTransportRequest` before handling an untrusted client
frame.

Browser WebSocket APIs cannot set arbitrary handshake headers. The `headers`
option is included inside each application-level request frame. Authenticate
the handshake with secure cookies, a short-lived query parameter, or a
subprotocol, and validate the connection origin and every frame on the server.

`sendAutomaticallyWhen` still starts a new chat turn after the preceding UI
message stream finishes. `WebSocketChatTransport` avoids opening another HTTP
connection for that turn, but it does not send tool results into an active
model generation.

If the socket closes during a response, the active stream errors. A later send
opens a new connection. The transport does not automatically replay requests,
because replaying a generation can duplicate model output or tool effects.
`reconnectToStream` requests a complete replay of the retained response from
sequence zero. Chat rebuilds the message with a fresh parser, so the replay must
include all earlier chunks and the original `start` chunk's `messageId`. That
stable message ID lets Chat replace the partial assistant message. Duplicate
sequence numbers are ignored within each response stream.

For complete options, protocol details, cancellation behavior, and a server
example, see the
[`WebSocketChatTransport` reference](/docs/reference/ai-sdk-ui/websocket-chat-transport).

## Direct Agent Transport

For scenarios where you want to communicate directly with an [Agent](/docs/reference/ai-sdk-core/agent) without going through HTTP, you can use `DirectChatTransport`. This transport invokes the agent's `stream()` method directly in-process.

This is useful for:

- **Server-side rendering**: Run the agent on the server without an API endpoint
- **Testing**: Test chat functionality without network requests
- **Single-process applications**: Desktop or CLI apps where client and agent run together

```tsx
import { useChat } from '@ai-sdk/react';
import { DirectChatTransport, ToolLoopAgent } from 'ai';

const agent = new ToolLoopAgent({
  model: "anthropic/claude-sonnet-5.5",
  instructions: 'You are a helpful assistant.',
  tools: {
    weather: weatherTool,
  },
});

const { messages, sendMessage } = useChat({
  transport: new DirectChatTransport({ agent }),
});
```

### How It Works

Unlike `DefaultChatTransport` which sends HTTP requests:

1. `DirectChatTransport` validates incoming UI messages
2. Converts them to model messages using `convertToModelMessages`
3. Calls the agent's `stream()` method directly
4. Returns the result as a UI message stream via `toUIMessageStream()`

### Configuration Options

You can pass additional options to customize the stream output:

```tsx
const transport = new DirectChatTransport({
  agent,
  // Pass options to the agent
  options: { customOption: 'value' },
  // Configure what's sent to the client
  sendReasoning: true,
  sendSources: true,
});
```

`DirectChatTransport` does not support stream reconnection since there is no
persistent server-side stream. The `reconnectToStream()` method always returns
`null`.

For complete API details, see the [DirectChatTransport reference](/docs/reference/ai-sdk-ui/direct-chat-transport).

## Workflow Transport

For chat apps built on the [Workflow SDK](/docs/agents/workflow-agent), `WorkflowChatTransport` from `@ai-sdk/workflow` provides automatic stream reconnection. It handles the common scenario where a workflow function times out mid-stream — the transport detects the missing `finish` event and reconnects to resume from where it left off.

```tsx
import { useChat } from '@ai-sdk/react';
import { WorkflowChatTransport } from '@ai-sdk/workflow/client';
import { useMemo } from 'react';

export default function Chat() {
  const transport = useMemo(
    () =>
      new WorkflowChatTransport({
        api: '/api/chat',
        maxConsecutiveErrors: 5,
        onChatEnd: ({ chatId, chunkIndex }) => {
          console.log(`Chat complete: ${chunkIndex} chunks`);
        },
      }),
    [],
  );

  const { messages, sendMessage } = useChat({ transport });

  // ... render chat UI
}
```

Key features:

- **Automatic reconnection**: Detects interrupted streams (no `finish` event) and reconnects via GET to `{api}/{runId}/stream`
- **Page refresh recovery**: `initialStartIndex` controls where the initial reconnection begins
- **Configurable retries**: `maxConsecutiveErrors` controls how many consecutive reconnection failures to tolerate
- **Lifecycle callbacks**: `onChatSendMessage` and `onChatEnd` for tracking chat state

Negative `initialStartIndex` values can fetch only the tail when the durable
server stream already stores `UIMessageChunk` objects. For raw `WorkflowAgent`
streams, use a non-negative cursor and follow the server-side conversion in the
WorkflowAgent guide.

For the full API reference, see [`WorkflowChatTransport`](/docs/reference/ai-sdk-workflow/workflow-chat-transport). For server-side endpoint setup, see the [WorkflowAgent guide](/docs/agents/workflow-agent#resumable-streaming-with-workflowchattransport).

## Building Custom Transports

To understand how to build your own transport, refer to the source code of the default implementation:

- **[DefaultChatTransport](https://github.com/vercel/ai/blob/main/packages/ai/src/ui/default-chat-transport.ts)** - The complete default HTTP transport implementation
- **[HttpChatTransport](https://github.com/vercel/ai/blob/main/packages/ai/src/ui/http-chat-transport.ts)** - Base HTTP transport with request handling
- **[ChatTransport Interface](https://github.com/vercel/ai/blob/main/packages/ai/src/ui/chat-transport.ts)** - The transport interface you need to implement

These implementations show you exactly how to:

- Handle the `sendMessages` method
- Process UI message streams
- Transform requests and responses
- Handle errors and connection management

The transport system gives you complete control over how your chat application communicates, enabling integration with any backend protocol or service.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Overview
description: An overview of foundational concepts critical to understanding the AI SDK
url: "https://ai-sdk.dev/docs/foundations/overview"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

This page is a beginner-friendly introduction to high-level artificial
intelligence (AI) concepts. To dive right into implementing the AI SDK, feel
free to skip ahead to our [quickstarts](/docs/getting-started) or learn about
our [supported models and providers](/docs/foundations/providers-and-models).

The AI SDK standardizes integrating artificial intelligence (AI) models across [supported providers](/docs/foundations/providers-and-models). This enables developers to focus on building great AI applications, not waste time on technical details.

For example, here’s how you can generate text with various models using the AI SDK:

To effectively leverage the AI SDK, it helps to familiarize yourself with the following concepts:

## Generative Artificial Intelligence

**Generative artificial intelligence** refers to models that predict and generate various types of outputs (such as text, images, or audio) based on what’s statistically likely, pulling from patterns they’ve learned from their training data. For example:

- Given a photo, a generative model can generate a caption.
- Given an audio file, a generative model can generate a transcription.
- Given a text description, a generative model can generate an image.

## Large Language Models

A **large language model (LLM)** is a subset of generative models focused primarily on **text**. An LLM takes a sequence of words as input and aims to predict the most likely sequence to follow. It assigns probabilities to potential next sequences and then selects one. The model continues to generate sequences until it meets a specified stopping criterion.

LLMs learn by training on massive collections of written text, which means they will be better suited to some use cases than others. For example, a model trained on GitHub data would understand the probabilities of sequences in source code particularly well.

However, it's crucial to understand LLMs' limitations. When asked about less known or absent information, like the birthday of a personal relative, LLMs might "hallucinate" or make up information. It's essential to consider how well-represented the information you need is in the model.

## Embedding Models

An **embedding model** is used to convert complex data (like words or images) into a dense vector (a list of numbers) representation, known as an embedding. Unlike generative models, embedding models do not generate new text or data. Instead, they provide representations of semantic and syntactic relationships between entities that can be used as input for other models or other natural language processing tasks.

In the next section, you will learn about the difference between model providers and models, and which ones are available in the AI SDK.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)

---
title: Prompts
description: Learn about the Prompt structure used in the AI SDK.
url: "https://ai-sdk.dev/docs/foundations/prompts"
docs_index: /llms.txt
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Prompts are instructions that you give a [large language model (LLM)](/docs/foundations/overview#large-language-models) to tell it what to do.
It's like when you ask someone for directions; the clearer your question, the better the directions you'll get.

Many LLM providers offer complex interfaces for specifying prompts. They involve different roles and message types.
While these interfaces are powerful, they can be hard to use and understand.

In order to simplify prompting, the AI SDK supports text, message, and system prompts.

## Text Prompts

Text prompts are strings.
They are ideal for simple generation use cases,
e.g. repeatedly generating content for variants of the same prompt text.

You can set text prompts using the `prompt` property made available by AI SDK functions like [`streamText`](/docs/reference/ai-sdk-core/stream-text) or [`generateText`](/docs/reference/ai-sdk-core/generate-text).
You can structure the text in any way and inject variables, e.g. using a template literal.

```ts {3}
const result = await generateText({
  model: "anthropic/claude-sonnet-5.5",
  prompt: 'Invent a new holiday and describe its traditions.',
});
```

You can also use template literals to provide dynamic data to your prompt.

```ts {3-5}
const result = await generateText({
  model: "anthropic/claude-sonnet-5.5",
  prompt:
    `I am planning a trip to ${destination} for ${lengthOfStay} days. ` +
    `Please suggest the best tourist activities for me to do.`,
});
```

## System Prompts

System prompts are the initial set of instructions given to models that help guide and constrain the models' behaviors and responses.
You can set system prompts using the `instructions` property.
System prompts work with both the `prompt` and the `messages` properties.
System messages in `prompt` or `messages` are rejected by default; use the `instructions` property for system instructions, or set `allowSystemInMessages: true` when you need to send existing message histories that contain system messages.

Opting in with `allowSystemInMessages` can create a prompt injection risk
where users can override or set the system prompt by injecting system
messages. In most cases, only trusted server-side code should set system
instructions via the `instructions` property.

```ts {3-6}
const result = await generateText({
  model: "anthropic/claude-sonnet-5.5",
  instructions:
    `You help plan travel itineraries. ` +
    `Respond to the users' request with a list ` +
    `of the best stops to make in their destination.`,
  prompt:
    `I am planning a trip to ${destination} for ${lengthOfStay} days. ` +
    `Please suggest the best tourist activities for me to do.`,
});
```

## Message Prompts

A message prompt is an array of user, assistant, and tool messages.
They are great for chat interfaces and more complex, multi-modal prompts.
You can use the `messages` property to set message prompts.

Each message has a `role` and a `content` property. The content can either be text (for user and assistant messages), or an array of relevant parts (data) for that message type.

```ts {3-7}
const result = await generateText({
  model: "anthropic/claude-sonnet-5.5",
  messages: [
    { role: 'user', content: 'Hi!' },
    { role: 'assistant', content: 'Hello, how can I help?' },
    { role: 'user', content: 'Where can I buy the best Currywurst in Berlin?' },
  ],
});
```

Instead of sending a text in the `content` property, you can send an array of parts that includes a mix of text and other content parts.

Not all language models support all message and content types. For example,
some models might not be capable of handling multi-modal inputs or tool
messages. [Learn more about the capabilities of select
models](./providers-and-models#model-capabilities).

### Provider Options

You can pass through additional provider-specific metadata to enable provider-specific functionality at 3 levels.

#### Function Call Level

Functions like [`streamText`](/docs/reference/ai-sdk-core/stream-text#provider-options) or [`generateText`](/docs/reference/ai-sdk-core/generate-text#provider-options) accept a `providerOptions` property.

Adding provider options at the function call level should be used when you do not need granular control over where the provider options are applied.

```ts
const { text } = await generateText({
  model: azure('your-deployment-name'),
  providerOptions: {
    openai: {
      reasoningEffort: 'low',
    },
  },
});
```

#### Message Level

For granular control over applying provider options at the message level, you can pass `providerOptions` to the message object:

```ts
const result = await generateText({
  model: "anthropic/claude-sonnet-5.5",
  instructions: {
    role: 'system',
    content: 'Cached system message',
    providerOptions: {
      // Sets a cache control breakpoint on the system message
      anthropic: { cacheControl: { type: 'ephemeral' } },
    },
  },
  prompt: 'Invent a new holiday and describe its traditions.',
});
```

#### Message Part Level

Certain provider-specific options require configuration at the message part level:

```ts
import { ModelMessage } from 'ai';

const messages: ModelMessage[] = [
  {
    role: 'user',
    content: [
      {
        type: 'text',
        text: 'Describe the image in detail.',
        providerOptions: {
          openai: { imageDetail: 'low' },
        },
      },
      {
        type: 'file',
        mediaType: 'image',
        data: 'https://github.com/vercel/ai/blob/main/examples/ai-functions/data/comic-cat.png?raw=true',
        // Sets image detail configuration for image part:
        providerOptions: {
          openai: { imageDetail: 'low' },
        },
      },
    ],
  },
];
```

AI SDK UI hooks like [`useChat`](/docs/reference/ai-sdk-ui/use-chat) return
arrays of `UIMessage` objects, which do not support provider options. We
recommend using the
[`convertToModelMessages`](/docs/reference/ai-sdk-ui/convert-to-model-messages)
function to convert `UIMessage` objects to
[`ModelMessage`](/docs/reference/ai-sdk-core/model-message) objects before
applying or appending message(s) or message parts with `providerOptions`.

### User Messages

#### Text Parts

Text content is the most common type of content. It is a string that is passed to the model.

If you only need to send text content in a message, the `content` property can be a string,
but you can also use it to send multiple content parts.

```ts {7-10}
const result = await generateText({
  model: "anthropic/claude-sonnet-5.5",
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'Where can I buy the best Currywurst in Berlin?',
        },
      ],
    },
  ],
});
```

#### Image Parts

User messages can include image parts. An image can be one of the following:

- base64-encoded image:
  - `string` with base-64 encoded content
  - data URL `string`, e.g. `data:image/png;base64,...`
- binary image:
  - `ArrayBuffer`
  - `Uint8Array`
  - `Buffer`
- URL:
  - http(s) URL `string`, e.g. `https://example.com/image.png`
  - `URL` object, e.g. `new URL('https://example.com/image.png')`

##### Example: Binary image (Buffer)

```ts {8-11}
const result = await generateText({
  model,
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe the image in detail.' },
        {
          type: 'file',
          mediaType: 'image',
          data: fs.readFileSync('./data/comic-cat.png'),
        },
      ],
    },
  ],
});
```

##### Example: Base-64 encoded image (string)

```ts {8-11}
const result = await generateText({
  model: "anthropic/claude-sonnet-5.5",
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe the image in detail.' },
        {
          type: 'file',
          mediaType: 'image',
          data: fs.readFileSync('./data/comic-cat.png').toString('base64'),
        },
      ],
    },
  ],
});
```

##### Example: Image URL (string)

```ts {8-12}
const result = await generateText({
  model: "anthropic/claude-sonnet-5.5",
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe the image in detail.' },
        {
          type: 'file',
          mediaType: 'image',
          data: 'https://github.com/vercel/ai/blob/main/examples/ai-functions/data/comic-cat.png?raw=true',
        },
      ],
    },
  ],
});
```

#### File Parts

Only a few providers and models currently support file parts: [Google
Generative AI](/providers/ai-sdk-providers/google), [Google Vertex
AI](/providers/ai-sdk-providers/google-vertex),
[OpenAI](/providers/ai-sdk-providers/openai) (for `wav` and `mp3` audio with
`gpt-4o-audio-preview`), [Anthropic](/providers/ai-sdk-providers/anthropic),
[OpenAI](/providers/ai-sdk-providers/openai) (for `pdf`).

User messages can include file parts. A file can be one of the following:

- base64-encoded file:
  - `string` with base-64 encoded content
  - data URL `string`, e.g. `data:image/png;base64,...`
- binary data:
  - `ArrayBuffer`
  - `Uint8Array`
  - `Buffer`
- URL:
  - http(s) URL `string`, e.g. `https://example.com/some.pdf`
  - `URL` object, e.g. `new URL('https://example.com/some.pdf')`

You need to specify the MIME type of the file you are sending.

##### Example: PDF file from Buffer

```ts {12-15}
import { google } from '@ai-sdk/google';
import { generateText } from 'ai';

const result = await generateText({
  model: google('gemini-3.8-flash'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is the file about?' },
        {
          type: 'file',
          mediaType: 'application/pdf',
          data: fs.readFileSync('./data/example.pdf'),
          filename: 'example.pdf', // optional, not used by all providers
        },
      ],
    },
  ],
});
```

##### Example: mp3 audio file from Buffer

```ts {12-14}
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';

const result = await generateText({
  model: openai('gpt-4o-audio-preview'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is the audio saying?' },
        {
          type: 'file',
          mediaType: 'audio/mpeg',
          data: fs.readFileSync('./data/galileo.mp3'),
        },
      ],
    },
  ],
});
```