DaskStreams CancelledErrors about streamz

[mrocklin]

What is s5?

[mrocklin]

In [1]: from distributed import Client
   ...: import streamz.dask as sd
   ...: from streamz import Stream
   ...: from tornado import gen
   ...: from tornado.ioloop import IOLoop
   ...: from distributed import sync
   ...: 
   ...: client = Client(processes=False)
   ...: 
   ...: def foo(x):
   ...:     print("incrementing")
   ...:     return x + 1
   ...: 
   ...: s = Stream()
   ...: s2 = sd.scatter(s)
   ...: s3 = s2.map(foo)
   ...: s4 = sd.gather(s3)
   ...: s4.sink(print)
   ...: 
Out[1]: <sink: print>

In [2]: s.emit(1)
incrementing
2
Out[2]: []

In [3]: s.emit(2)
incrementing
3
Out[3]: []

In [4]: s.emit(3)
incrementing
4
Out[4]: []

In [5]: history
from distributed import Client
import streamz.dask as sd
from streamz import Stream
from tornado import gen
from tornado.ioloop import IOLoop
from distributed import sync

client = Client(processes=False)

def foo(x):
    print("incrementing")
    return x + 1

s = Stream()
s2 = sd.scatter(s)
s3 = s2.map(foo)
s4 = sd.gather(s3)
s4.sink(print)
s.emit(1)
s.emit(2)
s.emit(3)

[jrmlhermitte]

thanks. i should have debugged on Client. The issue doesn't seem to be related to streams. Something more to do with the scheduler and worker on my machine. I'll look into it later.

Scheduler output:

distributed.scheduler - INFO - Receive client connection: Client-7d83d866-b02b-11e7-ae65-989096dbd8b9
distributed.scheduler - INFO - User asked for computation on lost data, foo-6fae4e6b1ff29357d2ffcc2568e2af02

[jrmlhermitte]

For the record: I fetched the latest version of distributed and dask and reinstalled and re-ran my scheduler and workers. No issue here anymore. So I'm not sure what went wrong. I will just leave it at that. If I see an issue again, I'll post on stackoverflow