Skip to content
AI guide

Add an AI Chatbot to Next.js with Streaming Responses

Build a chat page that streams answers word by word, using the Vercel AI SDK's `useChat` and a route handler, with one API key for GPT, Claude, Gemini and more.

20 min Intermediate Markdown for AI

What you'll build

  • A /api/chat route handler that streams answers with streamText
  • A chat page built with useChat, with a Stop button and error retry
  • A system prompt that keeps the bot on topic
  • A model picker that switches between GPT, Claude and Gemini by name
  • Cost limits, and optional chat history saved to Postgres

Before you start

  • A Next.js 16 app using the App Router
  • Basic React (state and event handlers)
  • A free CloudHike.dev account with credits for AI usage

How a streaming chatbot works

A chat page has two parts. In the browser, useChat keeps the list of messages and sends them to your server. On the server, a route handler passes them to a model with streamText and streams the answer back as it is written. The user sees words appear right away instead of waiting for the whole reply.

  • The model API key stays on the server. The browser only ever talks to your own /api/chat route.
  • CloudHike AI speaks OpenAI's Chat Completions format, so the AI SDK's OpenAI-compatible provider works with it. One key covers models from OpenAI, Anthropic, Google, xAI, DeepSeek and others.
  • Switching models is a one-word change: the model name, such as openai/gpt-5-mini or anthropic/claude-sonnet-4.5.

Step 1: Get an AI key and install the AI SDK

  1. Sign in at cloudhike.dev, open your project and click Deploy on the AI card.
  2. Click Copy as .env and paste AI_BASE_URL and AI_API_KEY into .env.local.
.env.local
Shell
AI_BASE_URL=https://...
AI_API_KEY=...
Shell
npm install ai @ai-sdk/react @ai-sdk/openai-compatible server-only

Create the provider once, in a file that can only be imported on the server:

lib/ai.ts
TypeScript
import "server-only";
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

export const models = createOpenAICompatible({
  name: "cloudhike",
  baseURL: process.env.AI_BASE_URL!,
  apiKey: process.env.AI_API_KEY,
});

The list of models your chat offers goes in a separate file with no secrets, so both the server and the browser can import it:

lib/chat-models.ts
TypeScript
export const CHAT_MODELS = [
  { id: "openai/gpt-5-mini", label: "GPT-5 mini" },
  { id: "anthropic/claude-sonnet-4.5", label: "Claude Sonnet 4.5" },
  { id: "google/gemini-2.5-flash", label: "Gemini 2.5 Flash" },
] as const;

export type ChatModelId = (typeof CHAT_MODELS)[number]["id"];

export const DEFAULT_MODEL: ChatModelId = "openai/gpt-5-mini";

// Only allow models from the list, whatever the browser sends.
export function pickModel(id: unknown): ChatModelId {
  return CHAT_MODELS.some((m) => m.id === id) ? (id as ChatModelId) : DEFAULT_MODEL;
}

Never prefix AI_API_KEY with NEXT_PUBLIC_. Every request is paid from your credits, so a leaked key lets anyone run up your bill.

Step 2: Stream answers from a route handler

useChat POSTs to /api/chat by default. The route turns the UI messages into model messages, adds the system prompt and streams the answer back in the format useChat reads.

app/api/chat/route.ts
TypeScript
import { convertToModelMessages, streamText, type UIMessage } from "ai";
import { models } from "@/lib/ai";
import { pickModel } from "@/lib/chat-models";

// Let long answers finish. Use what your host's plan allows.
export const maxDuration = 60;

const SYSTEM = `You are the friendly support assistant for Acme, a project management app.
Answer questions about using Acme in short, clear paragraphs.
If you don't know the answer, say so and suggest emailing support@acme.example.
Politely decline requests that have nothing to do with Acme.`;

export async function POST(request: Request) {
  const { messages, model }: { messages: UIMessage[]; model?: string } = await request.json();

  // Keep only user and assistant turns (the browser must not set the system
  // prompt) and the last 20 messages, so long chats stay cheap.
  const recent = messages.filter((m) => m.role === "user" || m.role === "assistant").slice(-20);

  const result = streamText({
    model: models(pickModel(model)),
    system: SYSTEM,
    messages: await convertToModelMessages(recent),
    maxOutputTokens: 1000,
    abortSignal: request.signal, // stop generating (and paying) when the user clicks Stop
  });

  return result.toUIMessageStreamResponse({
    onError: (error) => {
      console.error(error);
      return "Sorry, something went wrong. Please try again.";
    },
  });
}

The system prompt is where you set the bot's job, tone and limits. Keep it on the server: anything in the browser can be changed by the user. Be specific about what to do when the bot doesn't know, or it will guess.

onError decides what the user sees. The real error is logged on the server, and the browser gets a friendly message instead of internal details.

Step 3: Build the chat UI with useChat

useChat from @ai-sdk/react holds the messages, sends new ones and updates the last message as the stream comes in. Each message has parts; for a plain chatbot you only render the text parts.

components/chat.tsx
TSX
"use client";

import { useChat } from "@ai-sdk/react";
import type { UIMessage } from "ai";
import { useState } from "react";
import { CHAT_MODELS, DEFAULT_MODEL } from "@/lib/chat-models";

export function Chat({ id, initialMessages }: { id?: string; initialMessages?: UIMessage[] }) {
  const [input, setInput] = useState("");
  const [model, setModel] = useState<string>(DEFAULT_MODEL);
  const { messages, sendMessage, regenerate, status, stop, error } = useChat({ id, messages: initialMessages });

  const busy = status === "submitted" || status === "streaming";

  function onSubmit(e: React.FormEvent) {
    e.preventDefault();
    const text = input.trim();
    if (!text || busy) return;
    sendMessage({ text }, { body: { model } });
    setInput("");
  }

  return (
    <div className="mx-auto flex max-w-2xl flex-col gap-4 p-6">
      {messages.map((m) => (
        <div key={m.id} className={m.role === "user" ? "self-end rounded-lg bg-blue-50 p-3" : "p-3"}>
          {m.parts.map((part, i) =>
            part.type === "text" ? (
              <p key={i} className="whitespace-pre-wrap">
                {part.text}
              </p>
            ) : null,
          )}
        </div>
      ))}

      {status === "submitted" && <p className="text-gray-500">Thinking…</p>}
      {error && (
        <p role="alert" className="text-red-600">
          {error.message}{" "}
          <button type="button" onClick={() => regenerate({ body: { model } })} className="underline">
            Try again
          </button>
        </p>
      )}

      <form onSubmit={onSubmit} className="flex gap-2">
        <select value={model} onChange={(e) => setModel(e.target.value)} aria-label="Model">
          {CHAT_MODELS.map((m) => (
            <option key={m.id} value={m.id}>
              {m.label}
            </option>
          ))}
        </select>
        <input
          value={input}
          onChange={(e) => setInput(e.target.value)}
          placeholder="Ask anything about Acme…"
          className="flex-1 rounded border px-3 py-2"
          maxLength={4000}
        />
        {busy ? (
          <button type="button" onClick={() => stop()}>
            Stop
          </button>
        ) : (
          <button type="submit" disabled={!input.trim()}>
            Send
          </button>
        )}
      </form>
    </div>
  );
}
app/chat/page.tsx
TSX
import { Chat } from "@/components/chat";

export default function ChatPage() {
  return <Chat />;
}

Run npm run dev and open http://localhost:3000/chat. Ask a question and the answer streams in. Click Stop mid-answer and the stream ends; the AI usage shows on Credits & Billing in your CloudHike dashboard.

`status`Meaning
submittedThe message was sent and the first words have not arrived yet.
streamingThe answer is coming in.
readyDone. The user can send the next message.
errorThe request failed. error.message holds the text from onError.

Step 4: Switch models by name

The model picker sends model in the request body, and the route passes it through pickModel(). Changing the model is only a different string, because every model goes through the same key and the same API.

  • Add or remove models in lib/chat-models.ts. Names follow the provider/model pattern, such as x-ai/grok-4 or deepseek/deepseek-chat.
  • See every model you can call, with its price, under Models and prices, or fetch GET {AI_BASE_URL}/models from your server.
  • Always check the name on the server with an allowlist. Otherwise anyone could pick your most expensive model.

You can also choose the model on the server, without asking the user. For example, use a small fast model for short questions and a stronger one for long ones:

app/api/chat/route.ts (inside POST)
TypeScript
const last = recent.at(-1);
const lastText = last?.parts.map((p) => (p.type === "text" ? p.text : "")).join("") ?? "";
const modelId = lastText.length > 500 ? "anthropic/claude-sonnet-4.5" : "openai/gpt-5-mini";
// ...then pass models(modelId) to streamText()

Step 5: Keep costs under control

A public chatbot is an open door to your AI credits. Use a few layers of protection:

  • Cap each answer with maxOutputTokens, as in the route above.
  • Trim the history. Every message you send back is paid for again as input. The route keeps the last 20.
  • Limit input length with maxLength on the input, and check it on the server too.
  • Require sign-in for the chat page and the route. See protecting routes in Next.js.
  • Rate limit each user or IP address with Redis, as shown below.
  • Pick a cheaper default model. Small models answer most support questions well.

Here is a minimal fixed-window rate limit with Redis. The Redis rate limiting guide explains it and covers better algorithms.

lib/rate-limit.ts
TypeScript
import "server-only";
import Redis from "ioredis";

const globalForRedis = globalThis as unknown as { redis?: Redis };
const redis = globalForRedis.redis ?? new Redis(process.env.REDIS_URL!);
if (process.env.NODE_ENV !== "production") globalForRedis.redis = redis;

export async function allow(key: string, limit: number, windowSec: number) {
  const count = await redis.incr(`rate:${key}`);
  if (count === 1) await redis.expire(`rate:${key}`, windowSec);
  return count <= limit;
}
app/api/chat/route.ts (top of POST)
TypeScript
import { allow } from "@/lib/rate-limit";

// ...inside POST, before calling streamText():
const ip = request.headers.get("x-forwarded-for")?.split(",")[0]?.trim() ?? "unknown";
if (!(await allow(`chat:${ip}`, 20, 60))) {
  return new Response("Too many messages. Please wait a minute.", { status: 429 });
}

Your CloudHike account also has its own safety net. Requests get 402 when the account is out of credits or reaches its monthly spend cap, and 429 over the requests-per-minute limit. The numbers for each plan are in Pricing and limits and on the pricing page.

Step 6 (optional): Save chat history to Postgres

To let users come back to a conversation, give each chat an id and save its messages when an answer finishes. useChat already sends its id with every request. Store the messages as JSON in one row per chat.

schema.sql
SQL
create table chats (
  id text primary key,
  user_id text,
  messages jsonb not null default '[]',
  updated_at timestamptz not null default now()
);
lib/chat-store.ts
TypeScript
import "server-only";
import { Pool } from "pg";
import type { UIMessage } from "ai";

const globalForDb = globalThis as unknown as { pool?: Pool };
const pool = globalForDb.pool ?? new Pool({ connectionString: process.env.DATABASE_URL, max: 5 });
if (process.env.NODE_ENV !== "production") globalForDb.pool = pool;

export async function loadChat(id: string): Promise<UIMessage[]> {
  const { rows } = await pool.query("select messages from chats where id = $1", [id]);
  return rows[0]?.messages ?? [];
}

export async function saveChat(id: string, messages: UIMessage[]) {
  await pool.query(
    `insert into chats (id, messages) values ($1, $2)
     on conflict (id) do update set messages = excluded.messages, updated_at = now()`,
    [id, JSON.stringify(messages)],
  );
}

In the route, read id from the body, and give toUIMessageStreamResponse the original messages and an onFinish callback. It hands you the full list, including the new answer. generateMessageId gives saved messages stable ids.

app/api/chat/route.ts
TypeScript
import { convertToModelMessages, createIdGenerator, streamText, type UIMessage } from "ai";
import { models } from "@/lib/ai";
import { pickModel } from "@/lib/chat-models";
import { saveChat } from "@/lib/chat-store";

export const maxDuration = 60;

const SYSTEM = "You are the friendly support assistant for Acme...";

export async function POST(request: Request) {
  const { id, messages, model }: { id: string; messages: UIMessage[]; model?: string } = await request.json();
  const recent = messages.filter((m) => m.role === "user" || m.role === "assistant").slice(-20);

  const result = streamText({
    model: models(pickModel(model)),
    system: SYSTEM,
    messages: await convertToModelMessages(recent),
    maxOutputTokens: 1000,
  });

  // Keep generating even if the browser disconnects, so onFinish always runs.
  result.consumeStream();

  return result.toUIMessageStreamResponse({
    originalMessages: messages,
    generateMessageId: createIdGenerator({ prefix: "msg", size: 16 }),
    onFinish: async ({ messages }) => {
      await saveChat(id, messages);
    },
    onError: (error) => {
      console.error(error);
      return "Sorry, something went wrong. Please try again.";
    },
  });
}

This version drops abortSignal on purpose: a saved chat should have the whole answer, even if the user closed the tab. Pick one behavior or the other.

Finally, give each chat its own URL. /chat starts a new one, and /chat/<id> loads the saved messages into useChat:

app/chat/page.tsx
TSX
import { connection } from "next/server";
import { redirect } from "next/navigation";

export default async function NewChatPage() {
  await connection(); // a new id on every visit, never at build time
  redirect(`/chat/${crypto.randomUUID()}`);
}
app/chat/[id]/page.tsx
TSX
import { Chat } from "@/components/chat";
import { loadChat } from "@/lib/chat-store";

export default async function ChatPage({ params }: { params: Promise<{ id: string }> }) {
  const { id } = await params;
  const initialMessages = await loadChat(id);
  return <Chat id={id} initialMessages={initialMessages} />;
}

In a real app, save the signed-in user's id in user_id and check it in both loadChat and saveChat, so nobody can read or overwrite someone else's chat by guessing its id. CloudHike Postgres gives you a DATABASE_URL for this.

Production checklist

  • Add AI_BASE_URL and AI_API_KEY (and REDIS_URL and DATABASE_URL if you use them) to your host's environment variables.
  • Require sign-in for the chat route, and rate limit each user.
  • Check the model name against an allowlist on the server.
  • Set maxOutputTokens and trim old messages before sending them.
  • Set maxDuration on the route so long answers aren't cut off by your host.
  • Show a friendly message for 402 and 429 errors, and log the real error on the server.
  • Watch AI usage on Credits & Billing in the first days after launch, and set a spend cap you are comfortable with.

Frequently asked questions

How do I stream an OpenAI response in Next.js?

Call streamText from the Vercel AI SDK in a route handler and return result.toUIMessageStreamResponse(). On the client, useChat from @ai-sdk/react reads the stream and updates the message as words arrive. You can also use the openai package with stream: true and pipe the chunks into a ReadableStream yourself.

Can I use Claude or Gemini with the Vercel AI SDK useChat hook?

Yes. useChat does not care which model answers. Point an OpenAI-compatible provider at a gateway that serves several model families, then pass a name such as anthropic/claude-sonnet-4.5 or google/gemini-2.5-flash to streamText.

Should I call the AI API from the browser?

No. The API key would be visible to anyone who opens the developer tools. Keep the key in a route handler or Server Action and let the browser talk only to your own server.

How do I stop a chatbot from costing too much?

Cap each answer with maxOutputTokens, send only the last messages of a long chat, require sign-in, rate limit each user, allow only the models you choose, and use a smaller model by default.

How do I save chat history with the AI SDK?

Give each chat an id, pass originalMessages and an onFinish callback to toUIMessageStreamResponse, and save the messages it hands you to a database. Load them again and pass them to useChat as its starting messages.