Stream LLM Tokens with FastAPI SSE in Python (2026) — push tokens over HTTP with native Server-Sent Events, not a hand-rolled StreamingResponse.
You already have FastAPI mastery for APIs. This post is the AI-facing gap: first-class SSE (fastapi.sse) for token streams that browsers can resume.
TL;DR
- FastAPI 0.135+ :
from fastapi.sse import EventSourceResponse, ServerSentEvent - Set
response_class=EventSourceResponseandyieldevents - Use
id+Last-Event-IDto resume after drops - Always check
await request.is_disconnected()and clean up infinally
Install
pip install "fastapi>=0.135" uvicorn
# or: uv add "fastapi>=0.135" uvicorn
Minimal token stream
from collections.abc import AsyncIterable
import asyncio
from fastapi import FastAPI
from fastapi.sse import EventSourceResponse, ServerSentEvent
app = FastAPI()
@app.post("/chat/stream", response_class=EventSourceResponse)
async def chat_stream(prompt: str) -> AsyncIterable[ServerSentEvent]:
# Replace with your real LLM async iterator
words = (prompt or "hello from fastapi sse").split()
for i, token in enumerate(words, start=1):
yield ServerSentEvent(
data={"token": token},
event="token",
id=str(i),
retry=3000,
)
await asyncio.sleep(0.05)
yield ServerSentEvent(raw_data="[DONE]", event="done", id=str(len(words) + 1))
Resume with Last-Event-ID
from typing import Annotated
from fastapi import Header, Request
@app.get("/items/stream", response_class=EventSourceResponse)
async def stream_items(
request: Request,
last_event_id: Annotated[int | None, Header()] = None,
) -> AsyncIterable[ServerSentEvent]:
items = [{"sku": f"A{i}", "qty": i} for i in range(20)]
start = (last_event_id or -1) + 1
try:
for i, item in enumerate(items):
if await request.is_disconnected():
break
if i < start:
continue
yield ServerSentEvent(data=item, event="item_update", id=str(i))
await asyncio.sleep(0.1)
finally:
# release queues / subscriptions here
pass
When SSE vs WebSocket
- SSE — one-way server→client (LLM tokens, progress, logs); auto-reconnect in browsers
- WebSocket — bidirectional chats / collaborative editors
Pair this with typed agents from Pydantic AI or a multi-model client like LiteLLM when you harden production.
Production tips
- Prefer FastAPI ≥ 0.135 native SSE over extra SSE libraries for new apps
- Keep generators disconnect-aware; never leak per-client queues
- Put auth/validation before you start yielding
- Proxies: native SSE sets
Cache-Control: no-cacheand buffering-friendly headers
Wrap-up
In 2026, streaming LLM tokens is a FastAPI built-in. Yield ServerSentEvents, stamp ids, honor Last-Event-ID, and your clients survive flaky networks.