loading...

30 Days of Python 👨‍💻 - Day 15 - Generators

arindamdawn profile image Arindam Dawn Originally published at tabandspace.com ・4 min read

30-days-of-python (30 Part Series)

1) 30 Days of Python 👨‍💻 - Day 1 - Introduction 2) 30 Days of Python 👨‍💻 - Day 2 - Data Types I 3 ... 28 3) 30 Days of Python 👨‍💻 - Day 3 - Data Types II 4) 30 Days of Python 👨‍💻 - Day 4 - Data Types III 5) 30 Days of Python 👨‍💻 - Day 5 - Conditions & Loops I 6) 30 Days of Python 👨‍💻 - Day 6 - Loops II & Functions 7) 30 Days of Python 👨‍💻 - Day 7 - Developer Environment 8) 30 Days of Python 👨‍💻 - Day 8 - OOP Basics 9) 30 Days of Python 👨‍💻 - Day 9 - OOP Pillars 10) 30 Days of Python 👨‍💻 - Day 10 - OOP Missing Pieces 11) 30 Days of Python 👨‍💻 - Day 11 - Functional Programming Basics 12) 30 Days of Python 👨‍💻 - Day 12 - Lambda Expressions & Comprehensions 13) 30 Days of Python 👨‍💻 - Day 13 - Decorators 14) 30 Days of Python 👨‍💻 - Day 14 - Error Handling 15) 30 Days of Python 👨‍💻 - Day 15 - Generators 16) 30 Days of Python 👨‍💻 - Day 16 - Module Basics 17) 30 Days of Python 👨‍💻 - Day 17 - External Modules 18) 30 Days of Python 👨‍💻 - Day 18 - File I/O 19) 30 Days of Python 👨‍💻 - Day 19 - Regular Expressions 20) 30 Days of Python 👨‍💻 - Day 20 - Debugging and Testing 21) 30 Days of Python 👨‍💻 - Day 21 - Scripting Basics 22) 30 Days of Python 👨‍💻 - Day 22 - Scripting Extras 23) 30 Days of Python 👨‍💻 - Day 23 - Web Scraping 24) 30 Days of Python 👨‍💻 - Day 24 - Web Development Basics 25) 30 Days of Python 👨‍💻 - Day 25 - Web Development Extras 26) 30 Days of Python 👨‍💻 - Day 26 - Machine Learning Basics 27) 30 Days of Python 👨‍💻 - Day 27 - ML & Data Science I 28) 30 Days of Python 👨‍💻 - Day 28- ML & Data Science II 29) 30 Days of Python 👨‍💻 - Day 29 - Automation Testing 30) 30 Days of Python 👨‍💻 - Day 30 - Free Python Resources

Today I explored all about the concept of generators in Python. The concept of generators exists in the JavaScript world as well. It was introduced in the ES6 version of JavaScript however I haven't used them in practical JavaScript projects much. While reading about them today, I realized they can be of great use and is used in several Python libraries and frameworks.

So what are generators?

A generator is a special kind of function that returns an iterable set of values, one at a time which means it can be looped over to get the values one by one. It is also sometimes referred to as functions that can be 'paused'. In theory, generators sound quite complicated and confusing. It is best to explain using code examples. We hare already used a built-in Python generator range to generate a range of values.

range_of_numbers = range(100) # a generator
for num in range_of_numbers: 
    print(num)
# Since range is a generator, it can be iterated or looped
def my_infinite_generator():
  num = 0
  while True:
    yield num
    num +=1

result = my_infinite_generator()
for i in result:
  print(i) # Kepps on printing values infinitely!

So what exactly is this generator function and how is it different from a normal function?

Functions can return only one value using the return statement. Once a function reaches a return statement, it returns the value and exits out. Whereas generator functions can return any number of values using a special keyword yield. Whenever the yield statement is reached, the function execution is paused and the control is passed to the whoever is calling the function.

The values of the generator can be extracted either by iteration or by using another built-in function called next on the generator. In the above block of code, the values of the generator are printed using iteration (for loop). The same can be done manually by using the next function.

def my_infinite_generator():
  num = 0
  while True:
    yield num
    num +=1

result = my_infinite_generator()
print(next(result)) # 0
print(next(result)) # 1
print(next(result)) # 2
print(next(result)) # 3
print(next(result)) # 4

Note how the generator function is able to remember the value of num and is able to increment it. Whenever yield statement is reached, it saves the local variables and their state and then the control is transferred to the caller.

def my_generator(max):
  num = 0
  while num < 3:
    yield num
    num +=1

result = my_generator(3)
print(next(result)) # 0
print(next(result)) # 1
print(next(result)) # 2
print(next(result)) # Stop Iteration Error

When an iteration is completed (in the above case once num = 3), the generator function automatically raises a StopIteration exception on further next calls to it. If the same generator is iterated using a for loop, the loop will automatically stop once the StopIteration is raised by the generator. The for loop internally handles this and terminates.

Performance benefits of using generators

Generator functions are memory efficient. Generators really come in handy when dealing with large sets of data that requires lot of processing to compute results. Memory is a finite resource and our systems can only hold a limited amount of data in memory.

For instance, if we have to create a function that accepts a number and prints the Fibonacci sequence till that number, the conventional approach would be something like this

def fibonacci(num):
  sequence = []
  a,b = 0,1
  for item in range(num):
    sequence.append(a)
    temp = a
    a = b
    b = temp + b
  return sequence

result = fibonacci(20)
print(result) # prints the fibonacci sequence

In the above function, the entire sequence is stored in memory as a list. The result is only printed once the entire sequence is stored in memory. This is usually fine when the number is small. However, if the number is large, the memory usage increases drastically and the process may even be killed if the memory overflows.

Using a generator the above function can be made more memory efficient

def fibonacci_generator(num):
  a,b = 0,1
  for item in range(num):
    yield a
    temp = a
    a = b
    b = temp + b

for num in fibonacci_generator(20):
  print(num)

In this case, only one value is stored in memory at a time, thus it is able to print values even if the provided number is very large.

Composition

Generators can be composed together or in other words, they can be piped together to combine results from different generators.

For instance, if we have to create a Fibonacci sequence with each value squared, they can be composed using two generator functions.

def fibonacci_generator(num):
  a,b = 0,1
  for item in range(num):
    yield a
    temp = a
    a = b
    b = temp + b

def square(nums):
  for num in nums:
    yield num**2

result = square(fibonacci_generator(5))

for num in result:
  print(num) # 0 1 1 4 9

This kind of composition can be very useful while computing large data sets and applying different operations on them in layers.

That's all about generators in Python in brief. Exploring generators today helped me solidify the mental model around the concept and I can now probably think of ways to implement them in JavaScript as well.

Tomorrow marks the start of the third week of this current Python challenge and as per my planned roadmap, I will be exploring another essential topic in Python - Modules. I am sure that will be quite interesting.

Have a great one!

30-days-of-python (30 Part Series)

1) 30 Days of Python 👨‍💻 - Day 1 - Introduction 2) 30 Days of Python 👨‍💻 - Day 2 - Data Types I 3 ... 28 3) 30 Days of Python 👨‍💻 - Day 3 - Data Types II 4) 30 Days of Python 👨‍💻 - Day 4 - Data Types III 5) 30 Days of Python 👨‍💻 - Day 5 - Conditions & Loops I 6) 30 Days of Python 👨‍💻 - Day 6 - Loops II & Functions 7) 30 Days of Python 👨‍💻 - Day 7 - Developer Environment 8) 30 Days of Python 👨‍💻 - Day 8 - OOP Basics 9) 30 Days of Python 👨‍💻 - Day 9 - OOP Pillars 10) 30 Days of Python 👨‍💻 - Day 10 - OOP Missing Pieces 11) 30 Days of Python 👨‍💻 - Day 11 - Functional Programming Basics 12) 30 Days of Python 👨‍💻 - Day 12 - Lambda Expressions & Comprehensions 13) 30 Days of Python 👨‍💻 - Day 13 - Decorators 14) 30 Days of Python 👨‍💻 - Day 14 - Error Handling 15) 30 Days of Python 👨‍💻 - Day 15 - Generators 16) 30 Days of Python 👨‍💻 - Day 16 - Module Basics 17) 30 Days of Python 👨‍💻 - Day 17 - External Modules 18) 30 Days of Python 👨‍💻 - Day 18 - File I/O 19) 30 Days of Python 👨‍💻 - Day 19 - Regular Expressions 20) 30 Days of Python 👨‍💻 - Day 20 - Debugging and Testing 21) 30 Days of Python 👨‍💻 - Day 21 - Scripting Basics 22) 30 Days of Python 👨‍💻 - Day 22 - Scripting Extras 23) 30 Days of Python 👨‍💻 - Day 23 - Web Scraping 24) 30 Days of Python 👨‍💻 - Day 24 - Web Development Basics 25) 30 Days of Python 👨‍💻 - Day 25 - Web Development Extras 26) 30 Days of Python 👨‍💻 - Day 26 - Machine Learning Basics 27) 30 Days of Python 👨‍💻 - Day 27 - ML & Data Science I 28) 30 Days of Python 👨‍💻 - Day 28- ML & Data Science II 29) 30 Days of Python 👨‍💻 - Day 29 - Automation Testing 30) 30 Days of Python 👨‍💻 - Day 30 - Free Python Resources

Posted on by:

arindamdawn profile

Arindam Dawn

@arindamdawn

Software Engineer who loves building user interfaces. In the quest to learn, unlearn and re-learn things.

Discussion

markdown guide
 

really nice tutorial, thank you!
You can replace

    temp = a
    a = b
    b = temp + b

with

    a, b = b, a + b
 

Thanks for the suggestion. I just kept it more verbose :)

 

Well explained, WELDONE and keep up the momentum.