Yielding results instead of returning about ipyparallel

[minrk]

If I understand corrctly, this is how IPython parallel already works. You can iterate through an AsyncResult, and it will yield results as they arrive.

Can you describe a bit more detail about what you are trying to accomplish?

[mbraak]

Thanks for your answer.

I see in the documentation that the view.map does indeed have an AsyncResult. Let me see if that works in our situation.

[mbraak]

It seems that our use case does not work with view.map.

We want something like this:

from ipyparallel import Client

def p():
  for i in range(10):
    do_yield(i)

rc = Client()
view = rc[:]
result = view.apply(p)

for v in result:
  print v

I think this is different from view.map because in this case it's the applied code that is responsible for the yielding.

The view.map alternative is subtly different:

result = view.map(lambda v: v, range(10))

for v in result:
    print v

[minrk]

Just to clarify from your example above, you want all engines to run the same function p with the same arguments?

[mbraak]

No, we want 1 engine to run the p function.

This would be correct:

rc = Client()
view = rc[0]

[minrk]

Here's an example of running a generator remotely. The gist is to:

1. call the generator function on the engine
2. create an iterator on the generator
3. use apply(next, Reference('iterator')) to yield each item from the remote generator

Then you get a local generator yielding results from a remote generator.

[mbraak]

Thanks! This looks very promising.