Tutorial :python: attributes on a generator object



Question:

Is it possible to create an attribute on a generator object?

Here's a very simple example:

def filter(x):      for line in myContent:          if line == x:              yield x  

Now say I have a lot of these filter generator objects floating around... maybe some of them are anonymous... I want to go back later and interrogate them for what they are filtering for. Is there a way I can a) interrogate the generator object for the value of x or b) set an attribute with the value of x that I can later interrogate?

Thanks


Solution:1

Unfortunately, generator objects (the results returned from calling a generator function) do not support adding arbitrary attributes. You can work around it to some extent by using an external dict indexed by the generator objects, since such objects are usable as keys into a dict. So where you'd like to do, say:

a = filter(23)  b = filter(45)  ...  a.foo = 67  ...  x = random.choice([a,b])  if hasattr(x, 'foo'): munge(x.foo)  

you may instead do:

foos = dict()  a = filter(23)  b = filter(45)  ...  foos[a] = 67  ...  x = random.choice([a,b])  if x in foos: munge(foos[x])  

For anything fancier, use a class instead of a generator (one or more of the class's methods can be generators, after all).


Solution:2

Yes.

class Filter( object ):      def __init__( self, content ):          self.content = content      def __call__( self, someParam ):          self.someParam = someParam          for line in self.content:              if line == someParam:                  yield line  


Solution:3

If you want to interrogate them for debugging purposes, then the following function will help:

import inspect    def inspect_generator(g):      sourcecode = open(g.gi_code.co_filename).readlines()      gline = g.gi_code.co_firstlineno      generator_code = inspect.getblock(sourcecode[gline-1:])        output = "Generator %r from %r\n" % (g.gi_code.co_name, g.gi_code.co_filename)      output += "".join("%4s: %s" % (idx+gline, line) for idx, line in enumerate(generator_code))        output += "Local variables:\n"      output += "".join("%s = %r\n" % (key,value) for key,value in g.gi_frame.f_locals.items())        return output    print inspect_generator(filter(6))  """Output:  Generator 'filter' from 'generator_introspection.py'     1: def filter(x):     2:     for line in myContent:     3:         if line == x:     4:             yield x  Local variables:  x = 6  """  

If you want to interrogate them to implement functionality then classes implementing the iterator protocol are probably a better idea.


Solution:4

No. You can't set arbitrary attributes on generators.

As S. Lott points out, you can have a object that looks like a generator, and acts like a generator. And if it looks like a duck, and acts like a duck, you've got yourself the very definition of duck typing, right there.

It won't support generator attributes like gi_frame without the appropriate proxy methods, however.


Solution:5

Thinking about the problem, there is a way of having generators carry around a set of attributes. It's a little crazy--I'd strongly recommend Alex Martelli's suggestion instead of this--but it might be useful in some situations.

my_content = ['cat', 'dog days', 'catfish', 'dog', 'catalog']    def filter(x):      _query = 'I\'m looking for %r' % x        def _filter():          query = yield None          for line in my_content:              while query:                  query = yield _query                if line.startswith(x):                  query = yield line            while query:              query = yield _query        _f = _filter()      _f.next()      return _f    for d in filter('dog'):      print 'Found %s' % d    cats = filter('cat')  for c in cats:      looking = cats.send(True)      print 'Found %s (filter %r)' % (c, looking)  

If you want to ask the generator what it's filtering on, just call send with a value that evaluates to true. Of course, this code is probably too clever by half. Use with caution.


Solution:6

I realize this is a very belated answer, but...

Instead of storing and later reading some additional attribute, your code could later just inspect the generator's variable(s), using:

filter.gi_frame.f_locals

I guess Ants Aasma hinted at that.


Solution:7

I just wrote a decorator to do this here: http://code.activestate.com/recipes/577057-generator-attributes/


Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »