Python generator and iterator
What is Iterable
An iterable is an object that can be iterated upon, meaning that you can traverse through all the values. Examples of iterable objects include lists, tuples, dictionaries, strings, and sets. In Python, an iterable object must implement the
__iter__() method, which returns an iterator. This method is called when the iter() function is used on the object.
What is Iterator
An iterator is an object that represents a stream of data. It defines a method,
__next__(), which is used to access the next item in the stream. The iterator also keeps track of the current position in the stream, so that it knows where to start the next iteration. When there are no more items to return, the
__next__() method should raise a StopIteration exception.
What is Iteration
Iteration is the process of repeatedly extracting the next item from an iterator. It is often used in conjunction with loops, such as for loops or while loops. The
for loop in Python, for example, automatically creates an iterator for the iterable object, and repeatedly calls the
__next__() method until a StopIteration exception is raised. Iteration allows you to work with large data sets or infinite sequences without using up all of your memory, by only loading the next item when it is needed.
1itr = [5, 2, 3] 2 3# iterator from the list 4iterator = iter(itr) 5 6print(next(iterator)) # output: 5 7print(next(iterator)) # output: 2 8print(next(iterator)) # output: 3
values in list all iterated, try to do one more iteration, it will throws StopIteration exception
1print(next(iterator)) # output: StopIteration Exception
To create a custom iterator, you need to define a class that implements the iterator protocol. The iterator protocol consists of two methods:
__iter__() method should return the iterator object itself, and the
__next__() method should return the next item in the sequence.
Here is an example of a custom iterator that iterates over a range of numbers:
1class CreateRange: 2 def __init__(self, start, end): 3 self.start = start 4 self.end = end 5 6 def __iter__(self): 7 return self 8 9 def __next__(self): 10 if self.start >= self.end: 11 raise StopIteration 12 current = self.start 13 self.start += 1 14 return current 15 16for i in CreateRange(0, 5): 17 print(i)
In this example,
CreateRange is a class that implements the iterator protocol. The
__iter__ method returns the iterator object itself, which is
__next__ method returns the next number in the range and increments the start value. The
for loop iterates over the
CreateRange object and prints out the numbers from 0 to 4.
1m = CreateRange(0, 3) 2print(next(m)) # output: 0 3print(next(m)) # output: 1 4print(next(m)) # output: 2 5print(next(m)) # output: StopIteration Exception
What is Generator
A generator in Python is a special type of iterator that allows you to iterate over a sequence of items, but without the need to store the entire sequence in memory. Instead, generators use a special type of function called a generator function, which uses the
yield keyword to return items one at a time, as they are requested.
In normal function statements after return will not execute, function will terminates after return
1def normal_function(): 2 print("BEFORE - will execute") 3 return 1 4 print("AFTER - will not execute") 5 6normal_function() # output : BEFORE - will exceute
A generator function is defined like a normal function, but instead of returning a value, it uses the
yield keyword to return a generator object.
1def generator_function(): 2 for i in range(5): 3 yield i
1generator_function() # output : <generator object generator_function at 0x000001B87C217190>
1for j in generator_function(): 2 print(j) 3output : 40 51 62 73 84
Here is an another example of a simple generator function that generates the Fibonacci sequence:
1def fibonacci(n): 2 a, b = 0, 1 3 for i in range(n): 4 yield a 5 a, b = b, a + b 6 7for number in fibonacci(10): 8 print(number)
fibonacci function is a generator function that uses the
yield keyword to return the next number in the Fibonacci sequence each time it's called. The
for loop iterates over the generator object returned by the
fibonacci function and prints out the numbers of the sequence.
The main advantage of using generators is that they allow you to work with large data sets or infinite sequences without using up all of your memory. They are also more efficient than lists because they only return one item at a time and don't need to store the entire sequence in memory. Generators can also be used to make the code more readable and to make it more memory efficient.