I'm having trouble grasping the utility of generator functions in JavaScript. While I understand they can be used to iterate over custom data types, I'm not convinced that they offer a significant advantage over traditional iteration methods.
For arrays, we can use the forEach loop:
const numbers = [1, 2, 3, 4, 5];
numbers.forEach(number => {
console.log(number);
});
And for objects, the for...in loop works well:
const person = {
name: 'Alice',
age: 30,
city: 'New York'
};
for (let key in person) {
console.log(key, person[key]);
}
I've also read that generator functions are useful for producing infinite data streams, such as the Fibonacci sequence. However, I can achieve the same result using a while loop and two variables:
function* fibonacci() {
let a = 0, b = 1;
while (true) {
yield a;
const next = a + b;
a = b;
b = next;
}
}
Could someone please elaborate on the specific benefits of generator functions and provide examples where they offer a clear advantage over traditional iteration methods?
Generators are a subcategory of iterators in JavaScript1.
Iterators in JavaScript are closely related to generators. The generators implement the iterable protocol in an easy to use manner. But JS itself uses iterables in various ways already which makes generators directly applicable.
In general, iterators generalise over enumerating things. This could be an array, in which case the difference is minimal:
const arr = [1, 2, 3, 4, 5];
for(let i = 0; i < arr.length; i++) {
const item = arr[i];
console.log(`do something with ${item}`);
}const arr = [1, 2, 3, 4, 5];
for(let it = arr[Symbol.iterator](), current = it.next(); !current.done; current = it.next()) {
const item = current.value;
console.log(`do something with ${item}`);
}This is a bit long-winded way of doing it but it is an illustrative example to directly compare it to a regular for loop. In reality you would use a for..of loop which handles the iteration in a much nicer fashion as long as the object implements the iterable protocol:
const arr = [1, 2, 3, 4, 5];
for(const item of arr) {
console.log(`do something with ${item}`);
}However, the iteration protocol does allow you to iterate over arbitrary data. For example, we can construct a linked list (minimal example) which exposes an iterator2. Then the same for (const item of iterable) can be used to go through it, without having to understand the implementation of the list:
class Node {
constructor (value, next = null) {
this.value = value;
this.next = next;
}
}
class LinkedList {
constructor(head) {
this.head = head;
}
[Symbol.iterator]() {
let next = this.head;
return {
next() {
const current = next;
next = current?.next;
return {
value: current?.value,
done: current === null
}
}
}
}
}
const links =
new Node(1,
new Node(2,
new Node(3,
new Node(4,
new Node(5)))));
const list = new LinkedList(links);
for(const item of list) {
console.log(`do something with ${item}`);
}The same can be done with other list implementations, or with trees, maps, sets, etc. All are handled uniformly. Otherwise, you need a completely different for loop for each.
The iterator protocol is used for more than just looping over. They are also essential for the so called array destructuring. It does not only work with arrays, it actually uses the iterable protocol to resolve the identifiers. So, you can use it with arrays:
const arr = [1, 2, 3, 4, 5];
const [a, b, ...rest] = arr;
console.log(a, b, rest);or lists:
class Node {
constructor (value, next = null) {
this.value = value;
this.next = next;
}
}
class LinkedList {
constructor(head) {
this.head = head;
}
[Symbol.iterator]() {
let next = this.head;
return {
next() {
const current = next;
next = current?.next;
return {
value: current?.value,
done: current === null
}
}
}
}
}
const links =
new Node(1,
new Node(2,
new Node(3,
new Node(4,
new Node(5)))));
const list = new LinkedList(links);
const [a, b, ...rest] = list;
console.log(a, b, rest);and so on.
Same can be said about the spread syntax3. It also relies on iterables:
spreading an array as arguments:
function doMath(a, b, c, d, e) {
return a + b + c + d + e;
}
const arr = [1, 2, 3, 4, 5];
console.log(doMath(...arr));spreading a list as arguments:
class Node {
constructor (value, next = null) {
this.value = value;
this.next = next;
}
}
class LinkedList {
constructor(head) {
this.head = head;
}
[Symbol.iterator]() {
let next = this.head;
return {
next() {
const current = next;
next = current?.next;
return {
value: current?.value,
done: current === null
}
}
}
}
}
const links =
new Node(1,
new Node(2,
new Node(3,
new Node(4,
new Node(5)))));
const list = new LinkedList(links);
function doMath(a, b, c, d, e) {
return a + b + c + d + e;
}
console.log(doMath(...list));and so on.
Iterables are widely used within JavaScript. They handle a lot of (by now) basic language features.
However, even more crucially, they allow for various ways of iteration that can even handle infinite data sources (e.g., reading a data stream) or maybe even read a very large dataset (e.g., a file which is gigabytes big) needing all of it at once (e.g., take a chunk -> process -> take another chunk, etc.).
These could be done without iterators. However, not without having to reinvent something close to the iterable protocol each and every time.
Generators in JavaScript produce iterables. Which makes them immediately useful to work with anything that handles the iterable protocol. For example, a generator massively simplifies making a list an iterable. From:
[Symbol.iterator]() {
let next = this.head;
return {
next() {
const current = next;
next = current?.next;
return {
value: current?.value,
done: current === null
}
}
}
}
to the much more digestible:
*[Symbol.iterator]() {
let current = this.head;
while(current !== null) {
yield current.value;
current = current.next;
}
}
class Node {
constructor (value, next = null) {
this.value = value;
this.next = next;
}
}
class LinkedList {
constructor(head) {
this.head = head;
}
*[Symbol.iterator]() {
let current = this.head;
while(current !== null) {
yield current.value;
current = current.next;
}
}
}
const links =
new Node(1,
new Node(2,
new Node(3,
new Node(4,
new Node(5)))));
const list = new LinkedList(links);
for(const item of list) {
console.log(`do something with ${item}`);
}But moreover, generators can directly plug into any existing usage of iterables in JavaScript:
Enumeration:
function *generator() {
yield 1;
yield 2;
yield 3;
yield 4;
yield 5;
}
for(const item of generator()) {
console.log(`do something with ${item}`);
}Destructuring assignment:
function *generator() {
yield 1;
yield 2;
yield 3;
yield 4;
yield 5;
}
const [a, b, ...rest] = generator();
console.log(a, b, rest);Spread syntax:
function doMath(a, b, c, d, e) {
return a + b + c + d + e;
}
function *generator() {
yield 1;
yield 2;
yield 3;
yield 4;
yield 5;
}
console.log(doMath(...generator()));Other than directly plugging into the JavaScript, generators are also coroutines allowing to pause and continue functions. There are two huge benefits for this:
I've also read that generator functions are useful for producing infinite data streams, such as the Fibonacci sequence. However, I can achieve the same result using a while loop and two variables.
You can do this easily
function fibonacci() {
let a = 0, b = 1;
const result = [];
result.push(a);
result.push(b);
while (true) {
const next = a + b;
result.push(next);
a = b;
b = next;
}
return result;
}
The problem being that this is not a useful function. If invoked, it will lead to infinite execution. While the generator function shown will not4:
function* fibonacci() {
let a = 0, b = 1;
while (true) {
yield a;
const next = a + b;
a = b;
b = next;
}
}
const fib = fibonacci();
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);
console.log(fib.next().value);The non-generator variant would need to be modified to terminate to be useful. But even with that modification, there are multiple usages which each would require a separate modification:
A generator-implemented fibonacci does not need to be altered to handle these. With a couple of helpers5 we can derive any value we really want:
function* skip(n, iterator) {
for (let i = 0; i < n; i++) //skip n
iterator.next();
yield* iterator; //return the rest
}
function* take(n, iterator) {
for (let i = 0; i < n; i++)
yield iterator.next().value;
}
function* fibonacci() {
let a = 0, b = 1;
while (true) {
yield a;
const next = a + b;
a = b;
b = next;
}
}
const fib = fibonacci();
//skip the first 10, take the next 5
const it = take(5,
skip(10, fib)
);
for(const item of it) {
console.log(`do something with ${item}`);
}The same skip and take can be used on any iterable. We could have some sort of ID generating function:
function* skip(n, iterator) {
for (let i = 0; i < n; i++) //skip n
iterator.next();
yield* iterator; //return the rest
}
function* take(n, iterator) {
for (let i = 0; i < n; i++)
yield iterator.next().value;
}
function* idGenerator() {
let i = 1;
while (true) {
yield `Some_random_id_${i}`;
++i;
}
}
const idGen = idGenerator();
//skip the first 10, take the next 5
const it = take(5,
skip(10, idGen)
);
for(const item of it) {
console.log(`do something with ${item}`);
}Or even objects implementing iterable6 and other iterable structures:
function* skip(n, iterator) {
for (let i = 0; i < n; i++) //skip n
iterator.next();
yield* iterator; //return the rest
}
function* take(n, iterator) {
for (let i = 0; i < n; i++)
yield iterator.next().value;
}
const arr = ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m",
"n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z"];
const arrayIterator = arr[Symbol.iterator]();
//skip the first 10, take the next 5
const it = take(5,
skip(10, arrayIterator)
);
for(const item of it) {
console.log(`do something with ${item}`);
}There are many extremely generic helpers that can be used on any iterable and implementing them as generators is simple5:
function* skip(n, iterator) {
for (let i = 0; i < n; i++) //skip n
iterator.next();
yield* iterator; //return the rest
}
function* take(n, iterator) {
for (let i = 0; i < n; i++)
yield iterator.next().value;
}
function* map(mappingFn, iterator) {
for (value of iterator)
yield mappingFn(value);
}
function* filter(predicate, iterator) {
for (value of iterator)
if (predicate(value))
yield value;
}
function* generator() {
let i = 1;
while (true) {
yield i;
++i;
}
}
const gen = generator();
//square the 2001th to 2006th even number
const it = map(x => x*x,
take(10,
skip(2000,
filter(x => x % 2 === 0, gen)
)
)
);
for(const item of it) {
console.log(`do something with ${item}`);
}Can you do the same with a loop? Sure. But you have to reimplement it again and again for consuming different data sources - arrays, maps, sets, reading files, getting values from a database, etc. Consider the case of trying to consume a large CSV file, for example: if you want to read and process line-by-line (loading the whole file in memory) it is possible. But if implemented as a generator, then it can be easily consumable by anything else that consumes iterators/generators:
//take 10 records that fulfil some criteria.
//A CSV line has to be read and processed to calculate the data to filter on:
const data = take(10,
filter(
record => record.someCalculatedField === 42,
map(
line => convertToRecord(line),
readCsvGeneratorFunction("some_file.csv")
)
);
This is trivial to do with the existing infrastructure to handle any iterables/generators
Implementing helpers by oneself is soon to not be needed at all, as the iterator helpers are soon to be included in the core language. When adopted by a browser, the the following would be an equivalent:
//square the 2001th to 2006th even number
const it = gen
.filter(x => x % 2 === 0)
.skip(2000)
.take(10)
.map(x => x*x);
//take 10 records that fulfil some criteria.
//A CSV line has to be read and processed to calculate the data to filter on:
const data = readCsvGeneratorFunction("some_file.csv")
.map(line => convertToRecord(line))
.filter(record => record.someCalculatedField === 42)
.take(10);
However, there might still be specialised cases where a non-standard (for JavaScript, but perhaps useful for a given project) iterator helper might need to be introduced.
async/awaitThis might be surprising but async/await and generators are intrinsically linked:
To implement either, you need to essentially mostly implement both. The core requirement is to be able to pausing and resuming the execution of a function.
In fact, async functions are implementable as generators. When ES2015 (ES6) was released, generators were included but await was released as part of ES2017. In the mean while, a usual way to handle it was to transpile the code. Given code like:
async function foo() {
await doAsyncStuff();
}
If transpiled with TypeScript against ES2015 the result is:
"use strict";
var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
return new (P || (P = Promise))(function (resolve, reject) {
function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
step((generator = generator.apply(thisArg, _arguments || [])).next());
});
};
function foo() {
return __awaiter(this, void 0, void 0, function* () {
yield doAsyncStuff();
});
}
while Babel produces:
function asyncGeneratorStep(n, t, e, r, o, a, c) { try { var i = n[a](c), u = i.value; } catch (n) { return void e(n); } i.done ? t(u) : Promise.resolve(u).then(r, o); }
function _asyncToGenerator(n) { return function () { var t = this, e = arguments; return new Promise(function (r, o) { var a = n.apply(t, e); function _next(n) { asyncGeneratorStep(a, r, o, _next, _throw, "next", n); } function _throw(n) { asyncGeneratorStep(a, r, o, _next, _throw, "throw", n); } _next(void 0); }); }; }
function foo() {
return _foo.apply(this, arguments);
}
function _foo() {
_foo = _asyncToGenerator(function* () {
yield doAsyncStuff();
});
return _foo.apply(this, arguments);
}
Both lean into generators to achieve the same as async/await.
1 Also important but not directly related to generators: Iteration protocols.
2 The implementation is intentionally a bit verbose.
3 When not used in object literals, e.g., foo = { a: 1, ...bar }.
4 Unless the code attempts to consume it all at once. Which is preventable. It is not for the non-generator function without modifying how it works.
5 The implementation of is intentionally simplified. In reality, they should also check if the iterator is not exhausted at each step. This is not much but the core of the logic is what is important. Finding the correct implementation is left as an exercise to the reader. compare and contrast with a full implementation without using generator functions.
6 Using an array here for illustrative purposes. While arr.slice(n, n+m) can be used instead of take(m, skip(n, arr) the former still has to return an array which can be hefty for large values of m. While the latter does not need to materialise the entire collection at once.