DEV Community

loading...

Careful when using Python Generators!

tortellini
Big Data Software Engineer @ Trimble Inc. I am writing about my journey in tech and gems I find along the way!
・1 min read

Yes, they are memory efficient!

  • Python generators may prove themselves extremely useful in cases when you have large amounts of data and are trying to feed it to an iterator. In short, generators return a lazy iterator that does not store their contents in memory. WOW! Great, sounds like you save yourself the trouble of running into a MemoryError, right?

But, they hide something...

nums = (x for x in range(10000000000000))
for num in nums:
   print(num, end=" ")
>>> 1,2,3,4....
for num in nums:
   print(num*2, end=" ")
# What do you think is the output of the above for loop?

Generators are hungry hungry!

  • When iterating over a generator, the elements at each location are essentially being "consumed" and "discarded".
  • In other words, if you try to iterate AGAIN over the generator, it will look like all of your elements vanished!

Answer to the code above:

  • You will see no numbers printed to the console.

Leave your comments below!

Discussion (0)