Demystifying Iterables and Iterators in Python

Two fundamental concepts in Python's iteration protocol are iterables and iterators. Although these terms may sound similar, they have distinct roles and behaviours. In this article, we will explore the key differences between iterables and iterators and how they help us efficiently traverse data in Python.

Understanding Iterables

In Python, an iterable is any object capable of returning its elements one at a time. It represents a collection of items that can be iterated upon. Examples of built-in iterables include lists, tuples, strings, and dictionaries.

Let's consider a simple example of a list:

my_list = [1,2,3,4]
for i in my_list:
    print (i)

Output:

1
2
3
4

Perhaps more elementarily, and for the purpose of explanation, we can consider an iterable as an object with an __iter__() method.

print(dir(my_list))

Output:

['__add__', '__class__', '__class_getitem__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']

This is where things get interesting!
The __iter__() method basically returns an Iterator object. This object is what is looped over by the for keyword.

Have a look at this:

my_list = [1,2,3,4]
my_iterator_list = my_list.__iter__()
print(i_my_list)

Output:

<list_iterator object at 0x7f3abd6eaa30>

But we are getting ahead of ourselves...

What are Iterators?

Iterators are objects that implement the iteration protocol, enabling iteration over an iterable object. They maintain the state of the iteration and provide a mechanism for accessing elements sequentially.

Woah! That's a lot of “Itera-”s. But what do we mean?

Essentially, Iterators are what make iteration (looping) possible. Your list or string (Iterable) is first converted to an Iterator object before it can be iterated over. We said earlier, this is done through the __iter__() method. Python also has a built-in iter() function that does the same thing.

my_list = [1,2,3,4]
my_iterator_list = iter(my_list)
print(i_my_list)

Output:

<list_iterator object at 0x7f3abd6eaa30>

An Iterator, in addition to having an __iter__() method like the Iterable, also has its characteristic __next__() method.

print(dir(my_iterator_list))

Output:

['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__length_hint__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__']

The __next__() method returns the next element in the iteration. This method raises the StopIteration exception when there are no more elements left to iterate. And of course, Python has a built-in next() function.

my_list = [1,2,3,4]
my_iterator_list = iter(my_list)

print(next(my_iterator_list))
print(next(my_iterator_list))
print(next(my_iterator_list))
print(next(my_iterator_list))
print(next(my_iterator_list))

Output:

1
2
3
4
Traceback (most recent call last):
  File "/home/kenn-stack/python test.py", line 32, in <module>
    print(next(my_iterator_list))
StopIteration

Iterator and Iterable Relationship

An important point to note is that an Iterator is also an Iterable, but not all Iterables are Iterators.

N.B: The __iter__() method of an Iterator just returns self since it is already an iterator.

A Practical Example

Let us consider this custom bit of code that mimics the Python for operation. It also catches the StopIteration exception.

my_list = [1,2,3,4]
my_iterator_list = iter(my_list)

while True:
    try:
        looped = next(my_iterator_list)
        print(looped)
    except StopIteration:
        break

Output:

1
2
3
4

Just as the first example!

And that's it. Don't forget to like and comment.
Ciao
Till next time…