Iterables in Python, The Buffet Table 🍽️

When you write a simple for x in something: in Python, there’s a lot happening behind the scenes. That “something” could be a list, a string, a dictionary, or even a file stream. But what exactly makes an object loop-able?

The answer: iterables.

In this article (Part 1 of a 3-part series), we’ll explore iterables with an easy-to-grasp metaphor the buffet table. We’ll peel back the layers of Python’s iteration protocol, look at memory details, pitfalls, and real-life use cases. By the end, you’ll never look at for item in data: the same way again.

---

🍴 The Buffet Table Metaphor

Imagine walking into a buffet restaurant. The setup looks like this:

• Buffet table = Iterable (the container of food items exist here)
• Plate = Iterator (your personal helper that keeps track of what you’ve taken)

Key idea:

• The buffet table itself never “runs out.” You can always grab a new plate and start again.
• Each plate knows its own position, so two people can eat from the same buffet independently.

---

What Is an Iterable in Python?

Formally, an object is iterable if:

1. It implements __iter__() that returns an iterator.
2. Or, if not, it implements __getitem__() with 0-based indexing, so Python can simulate iteration by fetching obj[0], obj[1], … until IndexError.

Think of iter(obj) as the universal doorbell:

• If obj.__iter__ exists, Python calls it.
• Otherwise, it tries the __getitem__ fallback.

---

Buffet in Code

lst = [1, 2, 3]          # list (iterable)
s = "hello"              # string (iterable)
st = {'x', 'y', 'z'}     # set (iterable, but unordered)
d = {'a': 1, 'b': 2}     # dict (iterable over keys)

# Python's iteration protocol under the hood
it = iter(lst)           # take a plate
print(next(it))          # 1
print(next(it))          # 2
print(next(it))          # 3

Notice: sets are iterable but not indexable (st[0] → TypeError).

---

🧠 Behind the Scenes Memory & Objects

• A list is a contiguous array of pointers to objects.

• An iterator (like list_iterator) is lightweight:

  • Stores a reference to the original list
  • Keeps a small integer cursor (next position)

👉 Iterators don’t copy data. They only hold a reference + position → O(1) memory overhead.

Example:

• range(10_000_000) → lazy, doesn’t allocate 10 million numbers.
• Iterators/generators let you stream data one-by-one with almost no memory cost.

---

Iterables via __getitem__ Fallback

class SquareSeq:
    def __getitem__(self, index):
        if index < 0:
            raise IndexError
        return index * index

sq = SquareSeq()
it = iter(sq)
print(next(it))  # 0
print(next(it))  # 1
print(next(it))  # 4

Here, Python keeps calling sq[0], sq[1], … until an IndexError.

⚠️ Warning: if you don’t raise IndexError, iteration could run forever.

---

Real-Life Use Cases

1. Processing huge log/CSV files → iterate line by line, don’t load all in memory.
2. Paginated API calls → each next() fetches a new page of results.
3. Pipelines → combine map, filter, itertools for efficient data flows.
4. Lazy computations → calculate expensive values only when needed.
5. Large numeric ranges → range() + slicing = efficient iteration.

---

Pitfalls & Gotchas

• Iterators get exhausted → once consumed, they don’t refill.
• Sets/dicts don’t guarantee order (unless you rely on CPython insertion-order).
• Infinite iterables (itertools.count()) → must slice or add stop conditions.

---

Custom Iterable The Right Way

Bad pattern (iterator = container):

class BadCities:
    def __init__(self):
        self._cities = ["Paris", "Berlin", "Rome"]
        self._index = 0
    def __iter__(self): return self
    def __next__(self):
        if self._index >= len(self._cities): raise StopIteration
        val = self._cities[self._index]
        self._index += 1
        return val

Problem: second loop finds nothing iterator already exhausted.

✅ Better pattern (separate container & iterator):

class CityIterator:
    def __init__(self, cities):
        self._cities, self._index = cities, 0
    def __iter__(self): return self
    def __next__(self):
        if self._index >= len(self._cities): raise StopIteration
        val = self._cities[self._index]; self._index += 1
        return val

class Cities:
    def __init__(self): self._cities = ["Paris", "Berlin", "Rome"]
    def __iter__(self): return CityIterator(self._cities)

Now you can loop over Cities() multiple times.

---

ASCII Mental Model

[Iterable (buffet table)]
       |
       | __iter__()  or __getitem__()
       V
[Iterator (plate)] --> reference + cursor
       |
       | __next__()
       V
   items one-by-one
   StopIteration -> stop

---

Quick Checklist ✅

• Multiple passes? → use iterable, not iterator.
• Memory-efficient? → use generator or iterator-based APIs.
• Need to test iterability? → try: iter(obj) / except TypeError.
• Special stop condition? → iter(callable, sentinel).

---

Wrap-Up

In Part 1, we learned:

• What an iterable is (the buffet).
• How iter() works (__iter__ or __getitem__ fallback).
• Why iterators are lightweight.
• Real-world iterable use cases.
• How to design reusable custom iterables.

👉 Next up (Part 2): Iterators the waiters with one-way tickets 🍽️. We’ll dive deep into __next__, StopIteration, and how generators fit into the picture.