JavaScript Streams API: ReadableStream, WritableStream and TransformStream

1. Why streams instead of just await response.json()?

For most API calls, await response.json() is the right choice: compact, readable and sufficient. But as soon as the data volume grows, large CSV exports, video streams, server-sent events, LLM responses, file uploads, a fundamental problem appears: the entire response has to be held in memory before a single byte can be processed. With a 500 MB CSV from a reporting API, that means 500 MB on the browser heap. The JavaScript Streams API solves this problem in a fundamentally different way: data flows through a pipeline as small chunks, and the developer processes each chunk immediately while the next one is still loading.

The second, often overlooked advantage is the improvement in time to first byte from the user's perspective. When a server sends a long response, an application using a ReadableStream can display the first data chunk immediately, even while the remaining 90% is still being transferred. That is the difference between an application that suddenly shows something after 3 seconds and one that displays the first lines after 300 milliseconds and loads the rest incrementally. LLM APIs use exactly this principle: the response appears token by token because the server sends a ReadableStream that the application can render right away.

2. ReadableStream: data sources as a stream

A ReadableStream is a data source that produces chunks sequentially. You create one with the constructor and a start(controller) function: inside this function you call controller.enqueue(chunk) to place data into the queue, and controller.close() once the source is exhausted. For asynchronous sources, database queries, file reads, external APIs, you use the optional pull(controller) method, which is called when the consumer is ready to receive the next chunk. That is the foundation for backpressure.

A ReadableStream can only be read once, it is not an observable that you can subscribe to any number of times. To send the same stream to two consumers, you use stream.tee(), which produces two identical ReadableStreams. This is useful when you want to write the same response into IndexedDB and display it on screen at the same time, without sending the request twice. Reading a ReadableStream happens either through the reader API (const reader = stream.getReader()) or via a for await...of loop, which treats the stream as an async iterator.

// ReadableStream, custom source with async generation
function createCounterStream(max) {
  let count = 0;
  return new ReadableStream({
    // start() called once, can enqueue immediately or return a Promise
    start(controller) {
      console.log('[Stream] Source initialized');
    },
    // pull() called when consumer is ready for more data (backpressure)
    async pull(controller) {
      if (count >= max) {
        controller.close();
        return;
      }
      // Simulate async data source (database, file, API)
      await new Promise((r) => setTimeout(r, 50));
      controller.enqueue({ index: count, value: count * count });
      count++;
    },
    cancel(reason) {
      console.log('[Stream] Consumer cancelled:', reason);
    },
  });
}

// Consume with for-await-of, clean, linear async code
async function processStream() {
  const stream = createCounterStream(10);

  for await (const chunk of stream) {
    console.log(`Chunk ${chunk.index}: ${chunk.value}`);
    // Process each chunk as it arrives, no need to wait for all 10
  }
  console.log('[Stream] Done');
}

3. Fetch API and streams: reading responses chunk by chunk

The response.body property of a fetch result is a ReadableStream. That means every HTTP response is already available as a stream, without having to create a stream of your own. For the most common use case, text streaming from an LLM API such as the OpenAI API or the Claude API, you read the stream with a TextDecoder-based reader and process each text chunk immediately. The user sees text appear while the server is still responding.

Streaming large binary files, downloads, video segments, zip archives, follows the same pattern with one difference: here the destination is not the DOM but a WritableStream or a Cache API entry. Instead of holding the entire ArrayBuffer in memory, you write each chunk straight into the target storage. The combination of response.body.pipeThrough() with a TransformStream for processing, and pipeTo() for the destination, is the central pattern for efficient data processing with the JavaScript Streams API.

4. TransformStream: transforming data in the pipeline

A TransformStream sits in the middle of a stream pipeline: it has a readable end and a writable end. Whatever comes in on the writable end is transformed and emitted on the readable end. The constructor accepts transformer methods: transform(chunk, controller) is called for every incoming chunk, can transform it and pass it on with controller.enqueue(). flush(controller) is called once the upstream has closed and lets you write out any remaining data from an internal buffer.

TransformStreams are composable: several TransformStreams can be chained into a pipeline, each with a clear, bounded responsibility. A common pattern for processing server-sent events: a first TransformStream decodes bytes to text (new TextDecoderStream()), a second splits the text into lines on the newline character, a third filters lines that start with data:, and a fourth parses the JSON content. Each of these transformations is a clear function that can be tested separately. The Streams API makes this pipeline architecture efficient at runtime through backpressure propagation across every stage.

// Streaming fetch with TransformStream pipeline
// Pattern: LLM token streaming (works with OpenAI, Claude, Ollama)
async function streamLLMResponse(prompt) {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt }),
  });

  if (!response.body) throw new Error('No body');

  // Pipeline: bytes → text → SSE lines → JSON data chunks
  const lineStream = new TransformStream({
    buffer: '',
    transform(chunk, controller) {
      this.buffer += chunk;
      const lines = this.buffer.split('\n');
      this.buffer = lines.pop(); // keep incomplete last line
      for (const line of lines) {
        if (line.startsWith('data: ') && line !== 'data: [DONE]') {
          try {
            const data = JSON.parse(line.slice(6));
            controller.enqueue(data.choices?.[0]?.delta?.content ?? '');
          } catch {
            // Skip malformed JSON lines
          }
        }
      }
    },
    flush(controller) {
      // Process any remaining buffered content
      if (this.buffer.startsWith('data: ')) {
        controller.enqueue(this.buffer.slice(6));
      }
    },
  });

  const readable = response.body
    .pipeThrough(new TextDecoderStream())
    .pipeThrough(lineStream);

  // Render tokens to DOM as they arrive
  const output = document.getElementById('output');
  for await (const token of readable) {
    output.textContent += token;
  }
}

5. WritableStream: writing chunks to destinations

A WritableStream is the destination of a stream pipeline. It defines what happens to each chunk as it arrives: write it to a file, send it to a WebSocket, store it in IndexedDB, pass it on to a service worker. The constructor accepts a sink object with the methods start(controller), write(chunk, controller), close(controller) and abort(reason). The write() method can return a promise, and if it does, the upstream waits for that completion signal before sending the next chunk. That is the backpressure mechanism on the sink side.

A practical example is the streaming upload pattern: instead of reading a large file fully into memory and then sending it as a single request, you read it with the File System Access API as a ReadableStream and send it with a WritableStream that internally builds a fetch request. This allows uploads of files larger than the available JavaScript heap. The WritableStream abstraction cleanly separates "how do I produce data?" (ReadableStream) from "where do I send the data?" (WritableStream).

6. Backpressure: controlling flow, saving memory

Backpressure is the most important concept in the JavaScript Streams API, and it is what sets it apart from simpler event-emitter models. The problem without backpressure: if a producer generates data faster than the consumer can process it, chunks pile up in memory, in the worst case up to an out-of-memory error. The Streams API solves this with a queuing-strategy system built around a high-watermark value.

Every stream has an internal queue with a fill level. When the queue exceeds the high-watermark value, the stream returns a desiredSize signal that tells the producer to slow down or stop production. For ReadableStreams, the browser stops calling the pull() method until the consumer has read chunks out of the queue. For WritableStreams, the write() method returns a promise that only resolves once the downstream is ready. This automatic backpressure propagation across an entire pipeline of ReadableStream to TransformStream to WritableStream is what makes the Streams API suitable for high-volume data processing.

// WritableStream with backpressure and pipeline composition
// Pattern: streaming CSV upload with progress tracking

function createProgressWritableStream(onProgress) {
  let bytesWritten = 0;

  return new WritableStream({
    write(chunk) {
      // write() returning a Promise applies backpressure:
      // upstream waits until this resolves before sending next chunk
      return new Promise((resolve) => {
        bytesWritten += chunk.length;
        onProgress(bytesWritten);
        // Simulate async I/O (IndexedDB write, WebSocket send, etc.)
        setTimeout(resolve, 10);
      });
    },
    close() {
      console.log(`[WritableStream] Completed: ${bytesWritten} bytes total`);
    },
    abort(reason) {
      console.error('[WritableStream] Aborted:', reason);
    },
  });
}

async function uploadFileWithProgress(file, onProgress) {
  const [progressStream] = [createProgressWritableStream(onProgress)];

  // File.stream() returns a ReadableStream, no need to load file into memory
  await file.stream()
    .pipeThrough(new TransformStream({
      // Example: count lines in CSV while uploading
      transform(chunk, controller) {
        controller.enqueue(chunk); // pass through unchanged
      },
    }))
    .pipeTo(progressStream);
}

7. Pipelines with pipeThrough and pipeTo

pipeThrough(transformStream) and pipeTo(writableStream) are the methods that make stream pipelines composable. pipeThrough() takes a TransformStream, connects its writable end to the current ReadableStream, and returns the readable end of the TransformStream. The result is a new ReadableStream on which further pipeThrough() calls can be chained. pipeTo() terminates the pipeline and returns a promise that resolves once the entire data flow has reached the destination, or an error has occurred.

Error handling in stream pipelines is explicit: if a TransformStream throws an exception or calls controller.error(), the error propagates through the entire pipeline. Both the upstream ReadableStream and the downstream WritableStream are terminated in an errored state. The pipeTo() promise is rejected with the error. Unlike deeply nested promise chains, the source of the error in a stream pipeline is clearly identifiable by the stage of the TransformStream involved. The AbortController pattern is also fully supported: an AbortSignal can be passed to pipeTo() and aborts the entire pipeline.

8. Streams in Node.js: Web Streams vs. Node Streams

Node.js has had its own stream system since its inception. The Web Streams API (ReadableStream, WritableStream, TransformStream) is a separate, browser-compatible API that has been available as stable since Node.js 18. Both systems exist in parallel but are interoperable via adapters: Readable.toWeb(nodeReadable) converts a Node.js Readable Stream into a Web ReadableStream. Readable.fromWeb(webReadable) does the reverse. This matters for libraries that need to run both in the browser and in Node.js.

For new Node.js projects that also need to run in a browser-like context (edge functions, Cloudflare Workers, Deno), consistently using the Web Streams API instead of Node.js-specific streams is recommended. Cloudflare Workers and Deno support only Web APIs, Node.js-specific streams are not available there. The JavaScript Streams API is therefore not just a browser API but the universal standard for streaming in modern JavaScript runtimes.

9. Stream types compared directly

The three stream types of the JavaScript Streams API have clearly separated roles. Here is an overview of their characteristics and use cases:

The built-in streams TextDecoderStream and TextEncoderStream are specialized TransformStreams that cover the most common encoding tasks without requiring a custom transformer implementation. For compression and decompression there are CompressionStream and DecompressionStream with support for gzip, deflate and deflate-raw. These built-in Streams API components can be used directly in pipeline chains and save considerable boilerplate compared to manual implementations.

10. Summary

The JavaScript Streams API, ReadableStream, WritableStream and TransformStream, is the standardized model for chunk-based data processing in the browser and in modern JavaScript runtimes. Instead of loading data fully into memory, it flows through a pipeline as chunks that is automatically regulated by backpressure. The Fetch API integrates streams seamlessly: response.body is a ReadableStream that can be processed immediately. TransformStreams chain together into pipelines for decoding, parsing, filtering and transformation in a readable, modular architecture.

Use cases range from LLM token streaming and server-sent-events parsing to large CSV exports, streaming uploads and real-time data processing. Compatibility with Node.js 18+, Cloudflare Workers, Deno and all modern browsers makes the Web Streams API the universal standard for streaming in JavaScript, regardless of where the code runs. Anyone who plans streams into their architecture today writes code that runs without modification across all current and future JavaScript runtimes.

11. FAQ: JavaScript Streams API