Logging messages from engines about ipyparallel

[minrk]

I don't know what more robust means in this case. What exactly do you want it to do?

[mccarthyryanc]

I guess what I was looking for was something that extended the Logger class from the logging package so I could get logging messages from engines with something like:

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

def func(logger):
    logger.debug("hostname: %s", socket.gethostname())
    logger.warning("hostname: %s", socket.gethostname())

c = Client()
dv = c[:]
dv.apply_async(func,range(10),{'logger':logger})

Right now I just capture stdout from the engines in a similar manner to what's done in iopubwatcher.py. Perhaps that isn't the best method for logging, though.

[grantstephens]

Did you manage to get something working with the logger class?
Ran into a similar problem and it works in multiprocessing, but not ipyparallel.

[mccarthyryanc]

@RexFuzzle I didn't get a logger class working. What I am doing now is basically running main from iopubwatcher.py with multiprocessing.Process and writing output to a file instead of stdout.

[DrSAR]

I'm not going to pretend I understand what is going on here but in the [logging cookbook](git push --set-upstream origin logging.remastered) there is mention of a ZeroMQ ‘publish’ socket in conjunction with a ZeroMQ ‘subscribe’ socket. Maybe that's the the way to pass your engine produced log chatter back to the hub?

[minrk]

Hi! I’m going through and cleaning up old/stale issues on this repo. I’m especially closing old “how to” or debugging questions to keep the Issues list to bug reports and feature requests. These can be directed to the Jupyter forum if still relevant.

The 'modern' Jupyter approach I'd use for logging special messages is to use displaypub with custom mimetypes, which you can then watch for like the iopub watcher:

from IPython.display import publish_display_data
publish_display_data({"text/x-my-log": "log-message"})