jupyterhub not seeing running ipclusters about ipyparallel

[minrk]

The clusters tab is only aware of clusters that it has started. It doesn't have any awareness or control over clusters that it did not start, so you should be aware of that. I'm not aware of the clusters tab losing track of its own clusters, though. Maybe it's getting confused during the restart and thinking it should be removing the cluster that it just started. Can you show the logs of the notebook server when this happens (JupyterHub shouldn't be relevant, just the notebook server)?

I do plan to completely change how ipcluster works (make it a service instead of a script), at which point clusters started by any means should appear in the tab.

[mkgilbert]

Thank you so much for the reply! I apologize for taking so long getting
back to you. I wasn't aware that Notebook had a separate log file,
actually. I've been looking through the docs trying to find where it would
be stored, but I'm not having any luck. I also tried adding
"c.NotebookApp.log_file = /path/to/file", "c.NotebookApp.extra_log_file =
/path/to/file", and the same for "JupyterApp" but the log file never gets
generated. How do I go about specifying the log file for Notebook?

Thanks!

Mike

On Mon, Sep 28, 2015 at 10:58 AM, Min RK [email protected] wrote:

The clusters tab is only aware of clusters that it has started. It doesn't
have any awareness or control over clusters that it did not start, so you
should be aware of that. I'm not aware of the clusters tab losing track of
its own clusters, though. Maybe it's getting confused during the restart
and thinking it should be removing the cluster that it just started. Can
you show the logs of the notebook server when this happens (JupyterHub
shouldn't be relevant, just the notebook server)?

I do plan to completely change how ipcluster works (make it a service
instead of a script), at which point clusters started by any means should
appear in the tab.

—
Reply to this email directly or view it on GitHub
#44 (comment).

[mkgilbert]

I can't seem to find where the notebook sends its logs to...I don't see a spot for that in config. Is there a way to specify the notebook log file?

[polarizeme]

Having this same issue, just to throw another hat on the pile.

If any of our JupyterHub users start a number of engines through the clusters tab and then shut their servers down without stopping the engines first, the next time they start their hub server it won't see the running engines anymore in the clusters tab, but using

from ipyparallel import Client
c = Client(profile='default')
c.ids

in a notebook will still output the started engines from the previous session and I can still see them running w/ ps aux.

[minrk]

this is fixed in the new clusters extension in 7.0, which is aware of clusters started via any mechanism, be it the extension, ipcluster start, or the new ipp.Cluster API.