JavaScript Generator Functions: Lazy Iteration and Infinite Sequences

1. What makes generator functions fundamentally different

An ordinary JavaScript function starts, executes code, and returns a single value. A generator function, recognizable by the asterisk after the function keyword, can pause its execution context and produce a sequence of values one after another. The yield keyword pauses execution and returns the current value to the caller. On the next call to next(), the function resumes at exactly that point, with the same local scope, the same variables, the same call stack state.

This makes generator functions fundamentally different from regular functions or arrays: they don't compute values up front and store them, but instead produce each value only when it is requested. This property is called lazy evaluation. For an array with a million numbers, JavaScript needs a million memory slots. A generator function producing the same sequence needs constant memory, regardless of how long the sequence is. This is especially relevant for data transformation pipelines, pagination logic, file processing, and any case where you don't want or can't keep all the data in memory at once.

2. The iterator protocol: next(), return() and throw()

A generator function doesn't return a result when called, but a generator object that implements the iterator protocol. This protocol defines three methods: next(value) resumes execution and returns an object { value, done }. return(value) terminates the generator early and runs all finally blocks. throw(error) throws an exception at the currently paused yield point, as if it originated there, and the generator can catch it with try/catch.

The generator object also implements the iterable protocol: it has a [Symbol.iterator]() method that returns this. This means generator objects can be used directly in for...of loops, spread expressions, Array.from() and destructuring assignments. The for...of loop internally calls next() and stops when done: true comes back. return() is called on an early break, which ensures that finally blocks are also executed in that case, an important detail for resource cleanup in generator functions.

// Generator function basics, function* syntax and yield
function* counter(start = 0, step = 1) {
  let current = start;
  while (true) {
    // Pauses here and returns current value to caller
    const reset = yield current;

    // next(true) signals a reset back to start
    if (reset) {
      current = start;
    } else {
      current += step;
    }
  }
}

const gen = counter(0, 5);

console.log(gen.next().value);     // 0
console.log(gen.next().value);     // 5
console.log(gen.next().value);     // 10
console.log(gen.next(true).value); // 0, reset triggered
console.log(gen.next().value);     // 5

// Cleanup via return(), triggers finally blocks in the generator
gen.return('done'); // { value: 'done', done: true }

// Error injection via throw()
function* safeGen() {
  try {
    yield 1;
    yield 2;
  } catch (err) {
    console.error('Caught inside generator:', err.message);
    yield -1; // Recovery value after caught error
  }
}

const sg = safeGen();
sg.next();             // { value: 1, done: false }
sg.throw(new Error('oops')); // Caught inside generator: oops → { value: -1, done: false }

3. Lazy evaluation: computing values only when needed

The biggest practical advantage of generator functions is lazy evaluation: values are computed exactly when the consumer requests them, not earlier. Consider a transformation pipeline over a large dataset (filter, map, limit to the first ten): an array-based approach would filter and map every element before the take step kicks in. A generator pipeline stops computing as soon as the take limit is reached, the remaining elements are never computed.

This property has real performance consequences. When processing CSV files with millions of rows, an array-based approach loads the entire file into memory. A generator reads row by row, processes it, and frees the memory immediately. In Node.js, this connects elegantly with the streams API: a readable stream can be consumed as an async iterable, and generator functions can transform the incoming chunks without buffering them. This is the foundation for memory-efficient ETL pipelines in JavaScript.

4. Infinite sequences without memory problems

A generator function with an infinite loop isn't a bug, it's a useful pattern for infinite sequences. Fibonacci numbers, prime numbers, UUIDs, timestamps, pagination cursors: all of these can be modeled as an infinite sequence that the consumer consumes as needed. The memory problem of an array with infinite elements simply doesn't exist, because the generator only holds the current element in memory.

A practical example: a pagination function that automatically loads the next API page until all results have been consumed. Implemented as a generator function, the caller doesn't need to know how many pages there are; it iterates with for await...of, and the generator stops when no more pages are available. This decouples the pagination logic completely from the processing logic. The consumer can also stop midway with break, which properly terminates the generator and releases all resources.

// Infinite sequence, generates Fibonacci numbers on demand
function* fibonacci() {
  let [a, b] = [0, 1];
  while (true) {
    yield a;
    [a, b] = [b, a + b];
  }
}

// Take only what you need, rest is never computed
function take(n, iterable) {
  const result = [];
  for (const item of iterable) {
    result.push(item);
    if (result.length >= n) break;
  }
  return result;
}

console.log(take(8, fibonacci())); // [0, 1, 1, 2, 3, 5, 8, 13]

// Infinite primes using Sieve of Eratosthenes (lazy)
function* primes() {
  const composites = new Map();
  let n = 2;
  while (true) {
    if (!composites.has(n)) {
      yield n;
      composites.set(n * n, [n]);
    } else {
      for (const p of composites.get(n)) {
        const next = n + p;
        if (composites.has(next)) composites.get(next).push(p);
        else composites.set(next, [p]);
      }
      composites.delete(n);
    }
    n++;
  }
}

console.log(take(10, primes())); // [2, 3, 5, 7, 11, 13, 17, 19, 23, 29]

5. yield*: delegating to other iterables

yield* delegates iteration to another iterable: another generator, an array, a string, or any other iterable value. This is the tool for composing generator functions: a parent generator function can produce the values of multiple child generators one after another or nested. The difference from a normal loop with yield: yield* also forwards the return value of the delegated generator, which matters for bidirectional communication.

A practical use case for yield* is recursively traversing tree structures. A tree generator can call itself with yield* for child nodes, producing a flat stream of all nodes in depth-first order, without an explicit stack and without collecting all nodes into an array up front. This is more elegant and more memory-efficient than any array-based traversal, because at any point in time only the current path is held on the generator call stack.

6. Bidirectional communication with next(value)

A generator function can not only emit values but also receive values from the caller. The trick: next(value) passes a value that becomes available as the result of the current yield expression inside the generator. The first next() call has no value parameter (it is ignored, because no yield is yet waiting for a value). Only from the second next(value) onward does the value actually arrive inside the generator.

This pattern enables interactive generators that respond to outside control: a parser that receives characters; a state machine that processes events; a coroutine that waits for results of callbacks. Before async/await was introduced, this was exactly the foundation of co.js and other generator-based async libraries: a runner called next(), the generator yielded promises, and the runner waited for each promise and injected the result back via next(result).

// Bidirectional communication, generator as a coroutine
function* accumulator() {
  let total = 0;
  let count = 0;
  while (true) {
    // Receives value from next(n), returns running average
    const n = yield total === 0 ? null : total / count;
    if (n === null) break; // null signals termination
    total += n;
    count++;
  }
  return total / count; // Final average returned via done: true
}

const avg = accumulator();
avg.next();          // Start generator, yields null (no data yet)
avg.next(10);        // total=10, count=1 → yields 10
avg.next(20);        // total=30, count=2 → yields 15
avg.next(30);        // total=60, count=3 → yields 20
const final = avg.next(null); // done: true, value: 20

// yield* with return value, compose generators
function* inner() {
  yield 'a';
  yield 'b';
  return 'inner-done'; // Return value passed to outer via yield*
}

function* outer() {
  const result = yield* inner(); // Receives 'inner-done' as result
  console.log('Inner completed with:', result);
  yield 'c';
}

console.log([...outer()]); // ['a', 'b', 'c']
// Also logs: "Inner completed with: inner-done"

7. Async generators: asynchronous lazy iteration

Async generator functions (async function*) combine the lazy evaluation pattern with asynchronous operations. Instead of synchronous values, they produce promises that the consumer consumes with for await...of. This is the native pattern for asynchronous streams in modern JavaScript: API pagination, paged database queries, WebSocket messages, file reads in Node.js.

The crucial difference from regular generator functions: inside an async generator function you can use await to wait for promises before the next value is yielded. This allows mixing I/O operations and lazy evaluation without callback pyramids. An API pagination generator awaits the fetch request for each page and then yields each entry of the page individually, the consumer sees a continuous stream of entries while the generator loads the next page in the background.

/**
 * Async generator that paginates an API endpoint lazily.
 * Loads next page only when consumer requests more items.
 * @param {string} baseUrl - API base URL supporting ?page=N
 * @param {AbortSignal} [signal]
 * @yields {object} Individual items from each page
 */
async function* paginatedFetch(baseUrl, signal) {
  let page = 1;
  let hasMore = true;

  while (hasMore) {
    const response = await fetch(`${baseUrl}?page=${page}&per_page=100`, { signal });

    if (!response.ok) throw new Error(`HTTP ${response.status} on page ${page}`);

    const { data, meta } = await response.json();

    // Yield each item individually, consumer gets a flat stream
    for (const item of data) {
      yield item;
    }

    hasMore = page < meta.total_pages;
    page++;
  }
}

// Consumer, stops early after 250 items without loading more pages
const controller = new AbortController();
let count = 0;

for await (const order of paginatedFetch('/api/orders', controller.signal)) {
  await processOrder(order);
  if (++count >= 250) {
    controller.abort(); // Cancel in-flight request
    break;
  }
}

8. Generator pipelines for data transformation

The most elegant application of generator functions is transformation pipelines: a chain of generators, where each generator consumes values from the previous one, transforms them, and passes them on to the next. Every step is lazy, one row from the source travels through the entire pipeline and is fully processed before the next row begins. This results in constant-memory pipelines for datasets of arbitrary size.

Generator pipelines are composable and testable: every transformation generator function can be tested individually. Composition happens simply by nesting the calls. Unlike array methods such as .filter().map().reduce(), which each produce a new full array at every step, a generator pipeline never produces an intermediate array at any point. This is especially important in environments with a limited heap, such as edge functions or service workers.

9. Generators vs. arrays and streams compared

The choice between generator functions, arrays, and Node.js streams depends on the use case. Arrays are the right choice when all the data is needed and the amount is manageable: simple, searchable, iterable multiple times. Generators are optimal for lazy, once-iterable sequences of arbitrary length. Node.js streams support backpressure and are built for high-performance I/O, but are considerably more complex.

In modern JavaScript with for await...of and the Web Streams API, the boundaries between generator functions and streams blur. Node.js readable streams have implemented the async iterable protocol since version 16, so you can use them directly in for await...of. The Iterator Helpers proposal (Stage 3) adds native methods such as .map(), .filter(), .take() and .drop() directly to the iterator protocol, the same ergonomics as arrays, but lazy.

10. Summary

Generator functions are one of the most powerful and most underrated JavaScript features. function* and yield enable lazy evaluation, infinite sequences with constant memory usage, and bidirectional communication between generator and caller. The iterator protocol makes generators seamlessly compatible with for...of, spread syntax, and Array.from. yield* enables the composition of generators and elegant traversal of recursive structures. Async generators combine lazy evaluation with asynchronous I/O operations for API pagination, file processing, and database queries.

The most important takeaway: generator functions are not just academically interesting, they solve real problems better than arrays. Whenever you build an array only to subsequently filter and map it, a generator pipeline is the more memory- and compute-efficient alternative. Async generators are the native tool for asynchronous streams and replace complex event emitter patterns with understandable, synchronous-looking code.

11. FAQ: JavaScript Generator Functions