Streaming Responses from Node.js APIs

When a response takes time to generate — an LLM completion, a large CSV export, a real-time log feed — streaming lets clients start consuming data before the server is done producing it. This reduces time-to-first-byte and perceived latency without changing the underlying business logic.

Why Streaming Matters

For a conventional JSON response, the server collects all data, serializes it, then sends it. The client waits idle until the full response arrives. For a 10 MB dataset or a 5-second LLM response, that's a bad experience.

Streaming inverts this: the server sends chunks as they become available. The client can render partial results, process rows incrementally, or display typed text as it arrives.

Node.js Streams Refresher

Node's stream API has four types: Readable, Writable, Duplex, and Transform. For HTTP responses the relevant ones are Readable (data source) and Writable (the response).

The key rule: never write to a Writable faster than it can drain. When the internal buffer fills, write() returns false. Ignoring this causes unbounded memory growth — this is backpressure.

ts
import { createReadStream } from 'fs';
import { pipeline } from 'stream/promises';
import type { IncomingMessage, ServerResponse } from 'http';

export async function handler(req: IncomingMessage, res: ServerResponse) {
  res.setHeader('Content-Type', 'text/csv');
  res.setHeader('Transfer-Encoding', 'chunked');

  const fileStream = createReadStream('/data/export.csv');
  await pipeline(fileStream, res); // pipeline handles backpressure and cleanup
}

stream/promises pipeline is the correct way to connect streams. It handles backpressure, error propagation, and cleanup automatically. Never use stream.pipe() in new code — it leaks on error.

Streaming in Next.js Route Handlers

Next.js Route Handlers use the Web Streams API (ReadableStream) rather than Node streams:

ts
// app/api/export/route.ts
export async function GET() {
  const stream = new ReadableStream({
    async start(controller) {
      const cursor = db.post.findManyCursor(); // hypothetical cursor API

      for await (const post of cursor) {
        const line = `${post.id},"${post.title}"\n`;
        controller.enqueue(new TextEncoder().encode(line));
      }

      controller.close();
    },
  });

  return new Response(stream, {
    headers: {
      'Content-Type': 'text/csv',
      'Transfer-Encoding': 'chunked',
    },
  });
}

This keeps memory usage flat regardless of dataset size — the server never holds the full result in memory.

Server-Sent Events for Real-Time Updates

For push-based updates (status feeds, live logs, progress indicators), Server-Sent Events are a clean fit. SSE is one-directional (server to client), uses plain HTTP, and reconnects automatically. For a detailed comparison with WebSockets, see WebSockets vs Server-Sent Events for real-time apps.

ts
// app/api/progress/route.ts
export async function GET() {
  const encoder = new TextEncoder();

  const stream = new ReadableStream({
    async start(controller) {
      const send = (data: object) => {
        controller.enqueue(
          encoder.encode(`data: ${JSON.stringify(data)}\n\n`)
        );
      };

      send({ status: 'started', progress: 0 });
      await doWork((progress) => send({ status: 'running', progress }));
      send({ status: 'done', progress: 100 });
      controller.close();
    },
  });

  return new Response(stream, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      Connection: 'keep-alive',
    },
  });
}

On the client:

ts
const source = new EventSource('/api/progress');
source.onmessage = (event) => {
  const data = JSON.parse(event.data);
  setProgress(data.progress);
  if (data.status === 'done') source.close();
};

Streaming LLM Responses

Streaming is particularly valuable for LLM API calls where the model generates tokens over several seconds. Showing tokens as they arrive makes the UI feel responsive. The pattern is the same: pipe the LLM SDK's stream to your response stream:

ts
// app/api/generate/route.ts
import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

export async function POST(request: Request) {
  const { prompt } = await request.json();

  const stream = new ReadableStream({
    async start(controller) {
      const encoder = new TextEncoder();

      const response = await client.messages.stream({
        model: 'claude-sonnet-4-5',
        max_tokens: 1024,
        messages: [{ role: 'user', content: prompt }],
      });

      for await (const event of response) {
        if (
          event.type === 'content_block_delta' &&
          event.delta.type === 'text_delta'
        ) {
          controller.enqueue(encoder.encode(event.delta.text));
        }
      }

      controller.close();
    },
  });

  return new Response(stream, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
}

This is the foundation behind streaming LLM responses in a Next.js app — the UI side of the same pattern. The broader architecture for building AI features is covered in building your first AI feature with the Claude API.

Backpressure in Practice

With Web Streams, backpressure is handled through the queuing strategy and the desiredSize property on the controller. If the consumer is slow, desiredSize drops toward zero:

ts
const stream = new ReadableStream(
  {
    async pull(controller) {
      // pull is only called when the consumer wants more data
      const chunk = await getNextChunk();
      if (chunk) {
        controller.enqueue(chunk);
      } else {
        controller.close();
      }
    },
  },
  new CountQueuingStrategy({ highWaterMark: 5 })
);

Using pull instead of start lets the stream be demand-driven — it only generates data when the consumer is ready. This is the correct pattern for any producer that could generate data faster than it can be consumed.

Error Handling in Streams

Errors mid-stream are harder to recover from than errors in standard responses because headers (including the 200 status) have already been sent. The best approach is to signal errors in-band:

ts
// For SSE, send an error event before closing
controller.enqueue(encoder.encode(`event: error\ndata: ${JSON.stringify({ message: 'Processing failed' })}\n\n`));
controller.close();

For data streams (CSV, JSON Lines), consider a trailer or a convention like a final {"error": "..."} line. Document the protocol clearly so clients can handle it. This integrates with the error handling patterns from error handling patterns in Node.js APIs.

Conclusion

Streaming is a straightforward optimization with outsized impact on perceived performance. The key points: use pipeline in Node.js for backpressure-safe stream piping, use ReadableStream with pull in Next.js Route Handlers, prefer SSE for server-push scenarios, and handle errors in-band since headers are already committed. If you work on data-heavy or AI-backed applications, streaming is not optional — it is the correct default for any response that takes more than a second to generate. Explore the projects to see streaming patterns in production, or check out interview prep for how this topic is tested in backend engineering interviews.