Generators, Iterators, and Iterables

  • Last updated on 24th Jun 2022

Introduction

In this article, I will give an overview of how to use Javascript (ES6+) to build sequential programs by using generators, iterators, iterables. I will also discuss how iterables and iterators work in conjunction with generators, but first I will explain them as standalone.

Generators

Generators are functions except that these don’t run to completion like normal functions do. Instead, the generator can be paused and resumed however we need. This allows us to have finer control flow in javascript. We define generators like so:

function* cards() {
  // more code here
}

Above, we denote the function using the *. As the name suggests, we use it to produce a series of values or to iterate over a collection of data. In combination with iterators/iterables is where it gets good. So let’s move on to that.

Iterators & Iterables

Iterators are objects that you can use to loop through a collection of data, like an array or a set, one item at a time. We have a next() method that gives us an object with two properties: value and done. The value property tells you what the current item in the collection is (while we iterate over it), while the done property tells you whether or not you’ve reached the end of the collection.

// Iterating over an array of different color socks
const socks = ['red', 'green', 'blue', 'yellow', 'purple
const socksIterator = socks.values()
let sock = socksIterator.next()
while (!sock.done) {
  console.log(_I have a ${sock.value} sock._)
  sock = socksIterator.next()
  console.log(sock)
}

Above, we are iterating over an array using the next() method of an iterator. So, the iterator is the object that performs that operation on the iterable (to loop over it), while the iterable is just an object that can be iterated over. Our console logs the value of each item in the collection being iterated over.

When we run the above, in our terminal we get:

{
  done: false,
  value: "red"
}
"I have a red sock."
{
  done: false,
  value: "green"
}
"I have a green sock."
{
  done: false,
  value: "blue"
}
"I have a blue sock."
{
  done: false,
  value: "yellow"
}
"I have a yellow sock."
{
  done: false,
  value: "purple"
}
"I have a purple sock."
{
  done: true,
  value: undefined
}

Objects are not iterable using the for…of loop (they are with for…in loop, but for the purpose of demo-ing I decided not to use it), so we have to make them iterable by defining a custom iterator using Symbol.iterator method. So it’s like having a blueprint for a machine to produce a specific product. Without the blueprint, the machine wouldn’t know what to produce. Similarly, without defining an iterator on an object, the program doesn’t know how to iterate over the object. So we use a custom iterator to translate the necessary instructions to iterate over the object, making it iterable. Below we print out the values of the shirt size, material, and color.

// Different color shirts
const shirts = {
  blue: {
    size: 'M',
    material: 'cotton',
  },
  red: {
    size: 'L',
    material: 'polyester',
  },
  green: {
    size: 'S',
    material: 'silk',
  },

  // Below we define a custom iterator
  [Symbol.iterator]: function () {
    //
    const values = Object.values(this)
    let index = 0
    return {
      next: () => {
        if (index < values.length) {
          return { value: values[index++], done: false }
        } else {
          return { done: true }
        }
      },
    }
  },
}

let shirtIterator = shirts[Symbol.iterator]()
let shirt = shirtsIterator.next()
while (!shirt.done) {
  console.log(_My ${shirt.value.size} ${shirt.value.material} shirt is ${shirt.value.color}._)
  shirt = shirtsIterator.next()
  console.log(shirt)
}

In our terminal we get:

{
  done: false,
  value: {
    material: "cotton",
    size: "M"
  }
}
"My M cotton shirt is undefined."
{
  done: false,
  value: {
    material: "polyester",
    size: "L"
  }
}
"My L polyester shirt is undefined."
{
  done: false,
  value: {
    material: "silk",
    size: "S"
  }
}
"My S silk shirt is undefined."
{
  done: true
}

Instead of returning an array with all values of the enumerable properties of shirts with Object.values(), we have to use Object.entries() to get both the keys and values of all the enumerable properties of the object.

const shirts = {
  blue: {
    size: 'M',
    material: 'cotton',
  },
  red: {
    size: 'L',
    material: 'polyester',
  },
  green: {
    size: 'S',
    material: 'silk',
  },
}

// Define Symbol.iterator, different syntax
shirts[Symbol.iterator] = function () {
  // Get an array of [key, value] pairs from the object
  const values = Object.entries(this)
  let index = 0
  return {
    next: () => {
      if (index < values.length) {
        // Add the key (which is the color) to the value object
        const [key, value] = values[index++]
        return { value: { ...value, color: key }, done: false }
      } else {
        return { done: true }
      }
    },
  }
}

let shirtIterator = shirts[Symbol.iterator]()
let shirt = shirtIterator.next()
while (!shirt.done) {
  // Print the color of the shirt along with the size and material
  console.log(_My ${shirt.value.size} ${shirt.value.material} ${shirt.value.color} shirt._)
  shirt = shirtIterator.next()
  console.log(shirt)
}

This way we can use the keys (which would be the color of the value object) and return it as a property of the values object in our iterator as we move from value to value. Then, we use the destructuring assignment to extract the key-value pairs from the values array returned by Object.entries(), and assign them to our key and value variables.

Below that line, we use destructuring with spread (object spread operator) to create a new object that then includes all the properties of the value object, and since our color property is the key, we return that too.

Below in our terminal we get:

{
  done: false,
  value: {
    color: "blue",
    material: "cotton",
    size: "M"
  }
}
"My M cotton blue shirt."
{
  done: false,
  value: {
    color: "red",
    material: "polyester",
    size: "L"
  }
}
"My L polyester red shirt."
{
  done: false,
  value: {
    color: "green",
    material: "silk",
    size: "S"
  }
}
"My S silk green shirt."
{
  done: true
}

Iterators and Iterables can also be used with Maps and String, since they are Symbol. Iterator is built-in by default for these, we don’t have to write our own, although if we need specific behavior, we can write a custom iterator on the method like we did before.

// Iterating over a map of different color belts
const belts = new Map([
  ['tan', { size: '32', material: 'leather', color: 'tan' }],
  ['black', { size: '36', material: 'synthetic', color: 'black' }],
  ['vanilla', { size: '28', material: 'canvas', color: 'white' }],
])

const beltsIterator = belts.values()
let belt = beltsIterator.next()
while (!belt.done) {
  console.log(
    _My ${belt.value.material} belt is ${belt.value.size} inches long and is ${belt.value.color} color._
  )
  belt = beltsIterator.next()
  console.log(belt)
}

In our terminal, we get:

{
  done: false,
  value: {
    color: "tan",
    material: "leather",
    size: "32"
  }
}
"My leather belt is 32 inches long and is tan color."
{
  done: false,
  value: {
    color: "black",
    material: "synthetic",
    size: "36"
  }
}
"My synthetic belt is 36 inches long and is black color."
{
  done: false,
  value: {
    color: "white",
    material: "canvas",
    size: "28"
  }
}
"My canvas belt is 28 inches long and is white color."
{
  done: true,
  value: undefined
}

If we use a string:

const clothing = 'CLOTHES'
const clothingIterator = clothing[Symbol.iterator]()
let letter = clothingIterator.next()
while (!letter.done) {
  console.log(_The letter is ${letter.value}._)
  letter = clothingIterator.next()
  console.log(letter)
}

In our terminal, each letter value is individually cycled through:

{
  done: false,
  value: "C"
}
"The letter is C."
{
  done: false,
  value: "L"
}
"The letter is L."
{
  done: false,
  value: "O"
}
"The letter is O."
{
  done: false,
  value: "T"
}
"The letter is T."
{
  done: false,
  value: "H"
}
"The letter is H."
{
  done: false,
  value: "E"
}
"The letter is E."
{
  done: false,
  value: "S"
}
"The letter is S."
{
  done: true,
  value: undefined
}

Now, like all of javascript there is some goofy goober behavior with iterables/iterators. So far, each of our initial console.log() statements is hardcoded and manually placed.

I did this because the way iterators/iterables operate is a bit nuanced. Essentially, the first time we call next() is to initiate the iteration in our while loop, and the second time (and onwards) is to cycle through the values. Thus, we will always have one “extra” next() call compared to our values (as we go beyond them) as the iteration continues to ensure that we have traversed all the sequence’s values, even if we have already arrived at the last value. This way, we can identify when we have reached the end.

End Thoughts

Hopefully this article has helped you learn and understand generators, iterables, and iteratos better. As well as how they work in standalone. Soon, I will cover how we can and why we often use yield with them to really add more utitility building sequential programs. Until then, have a good one!

Resources

MDN Generators, Iterators/Iterables