current community

your communities

more stack exchange communities

Stack Exchange
tour help
careers 2.0
Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 3 down vote favorite

I've been looking for a good way to write a Python iterator that is based on a generator. I've found many tutorials on the topic of iterators and many on generators and the yield statement, but nothing that combines the two. I've built a small example that works and wondered if there is a better way to do this. 

class myIterator :

    def __init__(self, n) :
        self.last = n
        self.myGen = self.myGenerator() 

    def __iter__(self) :
        return self.myGenerator()

    def next(self) :
        return self.myGen.next()

    def myGenerator(self) :
        prev = 0
        fib = 1
        while fib < self.last :
            res = fib
            yield res
            fib = fib + prev
            prev = res

        raise StopIteration

I've used this technique in a real world program that can be found at SQLStatements.py in my Github repository

The most perplexing part of this was defining the next() function. The obvious solutions all returned the first element every time they were called. Storing an instance variable containing the generator works, but seems to be a kludge.

If you know a better way to do this or a good tutorial that covers this topic please let me know.

Edit: The third example posted by @MartijnPieters solves the problem completely. Saving the self.generator.next function in self.next provides a next function. I hope this helps someone else trying to solve this problem.

share|improve this question
2  
Why doesn't __iter__ return self.myGen? –  krlmlr Jan 6 '13 at 22:03
    
Thanks. That might work, but it still involves the kludge I was hoping to avoid. –  Peter Wooster Jan 6 '13 at 22:24
add comment

2 Answers

up vote 8 down vote accepted

The iterator protocol consists of two parts. The __iter__ method is the most important one, it is expected to return the iterator when you use iter() on an object.

Just replace the body of __iter__ with myGenerator; no need to raise StopIteration either:

class myIterator:
    def __init__(self, n):
        self.last = n

    def __iter__(self):
        prev = 0
        fib = 1
        while fib < self.last:
            res = fib
            yield res
            fib += prev
            prev = res

Now iter(myIterator(10)) is an iterator and has a .next() method.

If you want your class to be useful as an iterator directly, you need __iter__ to return self and provide a .next() method.

A .next() method produces one element per function body (and use return somevalue). A generator is just a function using yield, really, such a method itself implements the iterator protocol for you (which includes .next()).

If you were to use .next(), it would look like this instead:

class myIterator:
    def __init__(self, n):
        self.last = n
        self.prev = 0
        self.fib = 1

    def __iter__(self):
        return self

    def next(self):
        if self.fib < self.last:
            res = self.fib
            self.fib += self.prev
            self.prev = res
            return res

        raise StopIteration

or you can reuse the .next() method of your generator:

class myIterator:
    def __init__(self, n):
        self.last = n
        self.next = self.myGenerator().next  # Use the generator `.next`

    def __iter__(self):
        return self

    def myGenerator(self):
        prev = 0
        fib = 1
        while fib < self.last:
            res = fib
            yield res
            fib += prev
            prev = res
share|improve this answer
    
Thank you, that works exactly as I wished. And it will provide a clear example for others. –  Peter Wooster Jan 6 '13 at 22:23
    
Note that in Python 3 an iterator needs a __next__ method (instead of just next) and iterator.next() is no longer supported (next(iterator) should be used). –  poke Jan 6 '13 at 22:26
    
The first example provides the iterator based on a generator, but doesn't include a next() function. Is there a way short of the kludge I was using to get that to work? –  Peter Wooster Jan 6 '13 at 22:31
    
@PeterWooster: In the first example, __iter__ itself is the generator, and a generator provides it's own .next() method. __iter__ is supposed to return an iterator, and a generator fits that requirement. In the second form, we return self, and only then need to explicitly define a .next() method. –  Martijn Pieters Jan 6 '13 at 22:38
    
That's what I had hoped for, but when I try to use next() it tells me that "myIterator instance has no attribute 'next'" –  Peter Wooster Jan 6 '13 at 22:45
show 5 more comments
up vote 3 down vote

A generator is already an iterator. There is no need to wrap it:

>>> def gen():
...  yield 1
...  yield 2
...  yield 3
... 
>>> 
>>> a = gen()
>>> dir(a)
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__iter__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'close', 'gi_code', 'gi_frame', 'gi_running', 'next', 'send', 'throw']
>>> ai = a.__iter__()
>>> ai
<generator object gen at 0x542f2d4>
>>> a
<generator object gen at 0x542f2d4>
>>> a.next()
1
>>> a.next()
2
>>> a.next()
3
>>> a.next()
Traceback (most recent call last):
  File "<string>", line 1, in <module>
StopIteration
share|improve this answer
    
Actually in the real world case there were reasons to wrap it in a class. When I started this project I used a generator and it got a bit messy, so I decided to use a class to wrap the other parts of it. I'll give the generator only method another shot. / thanks –  Peter Wooster Jan 6 '13 at 22:33
    
Well, if you think you need to wrap the generator, you will probably have to tell us more about your use case –  oefe Jan 6 '13 at 22:39
    
The use case parses a file into SQL statements. There is considerable setup and I definitely need to use yield as the results are returned from the middle of a for loop. But I'm going to give the generator only version another try. / Thanks –  Peter Wooster Jan 6 '13 at 22:55
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.