Streaming AI Chat with Model Serving
Build a streaming AI chat experience using AI SDK and Databricks Model Serving endpoints.
Build with AI
- Copy the prompt below
- Paste into Cursor, Claude Code, Codex, or any coding agent
- Your agent builds it — asking questions along the way so the result is exactly what you want
New to templates? Learn more here
When done, you will have:
- A real-time streaming chat interface in your Databricks App
- Integration with Databricks Model Serving via AI Gateway
- Server-side chat transport and client-side chat UI wired together
- A deployed app where users can converse with a Databricks-hosted LLM
Prerequisites
Complete these prerequisite templates first:
- Set Up Your Local Dev Environment — install the Databricks CLI and authenticate a profile.
- Query AI Gateway Endpoints — confirm your workspace exposes a chat endpoint via the AI Gateway.
Then verify these Databricks workspace features are enabled. If any check fails, ask your workspace admin to enable the feature.
- An OpenAI-compatible chat endpoint in Model Serving. Run
databricks serving-endpoints list --profile <PROFILE>and confirm at least one OpenAI-compatible chat endpoint is listed (e.g.databricks-gpt-5-4-mini,databricks-meta-llama-3-3-70b-instruct, ordatabricks-claude-sonnet-4). Endpoint availability varies by workspace and region; note the one you plan to set asDATABRICKS_ENDPOINT. - Databricks Apps enabled. Run
databricks apps list --profile <PROFILE>and confirm the command succeeds (an empty list is fine). A permission ornot enablederror means Apps is not available to this identity in this workspace.
