What is generator in python?

I am new in python programming. Did not understand generators even after reading some tutorials. Can someone please explain me generators in simple words with some examples?

Comments

[Nested Software]

You can think of it as a special function that can return values repeatedly (i.e. more than once), usually in a loop. Instead of the return keyword, yield is used for this purpose.

I think the main use case for generators in python is as a substitute for lists. When you've got a list, that means you need to keep all of the items in the list in memory the whole time that you're using the list. Replacing a list with a generator means you just need to keep the current value in memory. The concept is very similar to an iterator, but the syntax is shorter/cleaner. Python also has generator expressions, which have a syntax very similar to list comprehensions. The difference is that a list comprehension creates a (potentially large) list in memory, whereas the generator expression only ever holds a single item in memory at a time.

[Austin S. Hemmelgarn]

A generator is a compact way of expressing a particular sequence of items as a function.

The most important aspects are:

• Generators are also inherently iterators. Anything that expects an iterator can be passed a generator instead. The simplest example is a for...in loop, but there are many others (for example the map() builtin function)
• Each item in the sequence is generated (hence the name) as it's requested.
• Because values are generated on demand, you can save time if you need to search a sequence of items that are computationally expensive to compute by avoiding the need to compute the whole list.
• Because values are generated on demand, it's insanely memory efficient (a generator that produces a million items only needs to take up as much space as the code itself and any variables the generator uses to store state).
• It doesn't have to terminate. Unlike looping over a list, looping over a generator doesn't have to stop at some point, it can just go on forever (this is also really useful for some applications).
• Generators are one-use only. Once you've used it, you have to make a new one to start over.

The important part of a generator is the yield statement. Each time whatever is iterating over the generator asks for the next item in the sequence, the generator function is run up until the next yield statement is encountered, then the value provided to that statement is returned s the next item in the sequence. The key point to understand here is that after this, the generator function is 'paused' until the next time the generator gets asked for a new item. When an item is asked for again, the generator function resumes execution from exactly where it was previously, retaining the values that any variables local to the generator function had previously.

Further explanation will require some code, so here's an example:

def fibonacci():
    last = 0     # Previous term
    current = 1  # Current term
    tmp = 0      # Temporary storage

    while true:
        yield current

        tmp = last
        last = current
        current += tmp

This is a trivial generator function that will spit out the terms of the Fibonacci sequence. There's no termination condition, so if you were to loop over it, it the loop would just keep going forever.

Assume we start looping over this generator like so (don't do this unless you want to have to manually kill the Python interpreter, it will loop forever):

for x in fibonacci():
    print(x)

The overall execution flow will look like this:

1. The generator function will start up by initializing the local variables at the top of the function body.
2. We'll move into the infinite loop.
3. We hit the yield statement, which returns the value of current as the next item.
4. The generator pauses, and the body of the for loop runs with x equal to to the value of current.
5. The for loop asks for the next item from the generator, restarting the generator function.
6. The generator function updates it's internal state, computing the next value to return.
7. We reach the end of the body of the infinite loop in the generator and return to step 3.