radiasoft / container-beamsim-jupyter-base

Base image container-beamsim-jupyter

This layer is added for efficiency on builds. See container-beamsim-jupyter for usage, etc.

container-beamsim-jupyter-base's Issues

Try out danielballan/jupyterhub-share-link

https://github.com/danielballan/jupyterhub-share-link is a tool for sharing notebooks dynamically, read-only, time limited, and statelessly within the same JuptyerHub.

Note radiasoft/jupyter.radiasoft.org#21

Add GPy to py3

ICFA ML Workshop

Document testing

@robnagler when you have some time would you mind documenting what your cycle for making changes and testing is? I tried following the steps in container-conf/README but ran into mkdir: cannot create directory ‘/root’: Permission denied. Those steps also look like they push to alpha which isn't right for my testing. For sirepo I build and tag an image and then test it by deploying with rsconf in my dev env. A similar workflow here would be great if possible.

Multiple kernels

On alpha with the jupyterlab 3.x changes there are now multiple kernels displayed. There should only be a single python 3 option.

add pandas and seaborn to py3

Document testing of container-beamsim-jupyter

JupyterLab is fairly unstable so need a manual test suite to see that what we expect is there.

Need to set jedi==0.17.2

jupyter/notebook#2435

autocomplete is broken

Describe all sockets

Try to understand all the communication between the sockets in the notebook server. Even with --transport=ipc there's a socket listening. Also look at securing the comm between the nodes so trusted between client and server. LLNL has done this (without containers).

add lume-genesis for uspas

https://github.com/slaclab/lume-genesis
https://github.com/ChristopherMayes/openPMD-beamphysics (has examples)
https://github.com/slaclab/zfel

need lz4-devel for beamline analysis notebooks

Independent of bluesky, we need the lz4-devel package installed so beamline analysis notebooks can read compressed h5 files. The raydata project needs this as soon as possible on production jupyter.

Install jupyterlab 1.0

Many fixes. including fix for radiasoft/devops#182

Add safeopt to py3

ICFA ML Workshop

import warp seeing args

jupyter_notebook_config.py used to have c.InteractiveShellApp.exec_lines because this worked at one point. However, this needs to be in ipython_notebook_config in the default profile:

c.InteractiveShellApp.exec_lines = ["import sys; sys.argv[1:] = []"]

build error: KeyError: '@jupyterlab/notebook'

During build

Installing collected packages: scipy, h5py, openPMD-viewer
Successfully installed h5py-2.9.0 openPMD-viewer-0.9.0 scipy-1.3.1
An error occured.
KeyError: '@jupyterlab/notebook'

setuptools race condition

The build is broken due to an unexpected error while upgrading a third party python package:

https://travis-ci.org/radiasoft/container-beamsim-jupyter/builds/205214779

The error seems to be related to:

pypa/setuptools#951

Add SDDS to py3

Likely ICFA-ML tutorial will use Elegant so they may want it in py3.

ipywidgets not displaying

jupyterlab/jupyterlab#4855 (comment)

explore pytorch vs tensorflow

@robnagler please explore the tradeoffs of running with pytorch or tensorflow when it comes to dependencies. This is in support of webcon.

graph-tool

https://git.skewed.de/count0/graph-tool/-/wikis/installation-instructions#native-installation

As it describes in the writeup, it's a python package that cannot be installed with pip. It has dependencies on Boost & OpenMP, and it suggests optimization.

Thanks!

include sirepo package with jupyter

The sirepo jupyter export relies on the sirepo.analysis.ml module.

travis: time limit exceeded on build

https://travis-ci.org/radiasoft/container-beamsim-jupyter/builds/203632476

Synergia is the difficult one. It may make sense to build an RPM for it, since the install is tricky anyway.

Successfully installed numpy-1.12.0 rssynergia-20170216.220723
yum clean all
4099+1 records in
4099+1 records out
2098784 bytes (2.1 MB) copied, 0.00933809 s, 225 MB/s
The job exceeded the maximum time limit for jobs, and has been terminated.

Create startup_bashrc

fete error finding RadiaSoft rc styles from Jupyter notebook

On my jupyter server instance, I'm unable to load the RadiaSoft rc file.

The matplotlib config directory is correctly listed as /home/vagrant/.config/matplotlib, and the appropriate style files exist.

However, the none of the "rs_" styles appear as available options from within Jupyter notebooks, and attempting to load them produces an error. The same issue occurs when running Python or IPython from the terminal.

Jupyter Server coming in JupyterLab 3.0

Likely this will be another breaking change.

Upgrade to JupyterLab 3.1

We are doing a webinar Oct. 6 to show off using JupyterLab on Sirepo, it would be good to do a version upgrade before then to have the latest features available, if possible.

https://jupyterlab.readthedocs.io/en/stable/extension/extension_migration.html#jupyterlab-2-x-to-3-x

JPY_USER is deprecated

There's now way to get the user, I think, without looking at the notebook config. Need to verify.

In any event, want to avoid unbound variable error.

add plotly

Add bluesky to beamsim-jupyter

RayData could benefit from having this available for testing/learning purposes.

jupyter lab build fails and doesn't output errors

See the log file for details: /tmp/jupyterlab-debug-w0jacocd.log

allow one time server init

when the container starts, it should run a script for the user before jupyterlab starts. That was done via init-from-git, but that's no longer secure as we are not using github auth so we need a file to init similar to bashrc.

add botorch

For uspas https://botorch.org/

Install is downgrading numpy-1.21.1 to numpy-1.20.3

Maybe caused by botorch:

Collecting tenacity>=6.2.0
  Downloading tenacity-8.0.1-py3-none-any.whl (24 kB)
Collecting numpy<1.21,>=1.17
  Downloading numpy-1.20.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.3 MB)

/srv/mpi_worker/cmd: line 11: /usr/sbin/sshd: No such file or directory

Enable Jupyter Lab on RadiaSoft Jupyter servers

JupyterLab has been available a for a few months now, and I've had several recommendations from colleagues at ICAP that including this resource on our Jupyter servers may be useful for our work, as well as for educational outreach and attracting new users.

It would be great if we could include this on our Jupyter server builds.

cannot find georges-core for zgoubidoo==2020.1

%matplotlib widget: Error displaying widget: model not found

The %matplotlib widget option seems to have recently broken on Jupyter. Trying the following just gives an error:

%matplotlib widget

import matplotlib.pyplot as plt

plt.figure()
plt.plot(1, 1)
plt.show()

Error displaying widget: model not found

jupyterlab AttributeError: 'NotebookWebApplication' object has no attribute 'append'

NotebookWebApplication doesn't support "append". Need to review history to see what version works...

[I 23:36:07.049 LabApp] JupyterLab application directory is /home/vagrant/.pyenv/versions/3.7.2/envs/py3/share/jupyter/lab
[W 23:36:07.051 LabApp] Error loading server extension jupyterlab
    Traceback (most recent call last):
      File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/notebook/notebookapp.py", line 1991, in init_server_extensions
        func(self)
      File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/jupyterlab/extension.py", line 222, in load_jupyter_server_extension
        add_handlers(web_app, config)
      File "/home/vagrant/.pyenv/versions/3.7.2/envs/py3/lib/python3.7/site-packages/jupyterlab_server/handlers.py", line 165, in add_handlers
        handlers.append((url_pattern, LabHandler))
    AttributeError: 'NotebookWebApplication' object has no attribute 'append'

Replace labhub cmd with lab

See radiasoft/jupyter.radiasoft.org#53 for more details

Add PyTorch for USPAS

terminado settings patch not included

jupyterlab terminal not working in safari?

I just logged in from safari, and launched a terminal which was "all black". Works in Firefox and Chrome.

py3 default version and other py3 fixes

f29 upgrade

Need to fix:

build_yum install http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-27.noarch.rpm
build_yum install https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-27.noarch.rpm

make /lab default url

Broken Link in README

Link to 'radiasoft/container-beamsim' (https://github.com/radisoft/container-beamsim) in README is missing second 'a' in radiasoft in the url.

add botorch

for uspas

mpmd Widgets not loading: Plugin 'matplotlib-jupyter:main' failed to activate

This is happening in container-beamsim-jupyter and in @mkeilman's VM in his dev environment. I've tried to clean up the build.sh so clearer, but probably have to lock versions of something...

vendors~main.71830be4eac39a5864d4.js:41013 Plugin 'matplotlib-jupyter:main' failed to activate.
(anonymous) @ vendors~main.71830be4eac39a5864d4.js:41013
vendors~main.71830be4eac39a5864d4.js:41014 Error: No provider for: jupyter.extensions.jupyterWidgetRegistry.
    at JupyterLab.push.2dC9.Application.resolveRequiredService (vendors~main.71830be4eac39a5864d4.js:40928)
    at vendors~main.71830be4eac39a5864d4.js:40886
    at Array.map (<anonymous>)
    at JupyterLab.push.2dC9.Application.activatePlugin (vendors~main.71830be4eac39a5864d4.js:40886)
    at vendors~main.71830be4eac39a5864d4.js:41012
    at Array.map (<anonymous>)
    at JupyterLab.push.2dC9.Application.start (vendors~main.71830be4eac39a5864d4.js:41011)
    at main (main.6fcd977b391c16459117.js:1541)

The openPMD interactive viewer is not working in our Jupyter notebooks

The template notebook for the openPMD GUI is here,
https://github.com/openPMD/openPMD-viewer/blob/master/opmd_viewer/notebook_starter/Template_notebook.ipynb

You need to generate (or download) some data in order to test it.
This 2D data set is suitable,
https://github.com/openPMD/openPMD-example-datasets/blob/draft/example-2d.tar.gz

In order for the template notebook with GUI to work under Python2, I think you have to do the following:
pip install --upgrade openPMD-viewer
pip install --upgrade ipympl

Under Python3, I think it's the same, except that you don't need the ipympl install.

In both cases, you have to modify the first cell of the template openPMD notebook to specify:
%matplotlib widget

python incorrect in subprocess from jupyter notebook

PYENV_VERSION=jupyter so subprocesses get the wrong python. The terminal has a workaround. Need to set PYENV_VERSIONS in kernel.json env.

cookie_name has changed

This may be related to jupyterhub/wrapspawner#11. Need to figure out changes in minor release (0.8) that cause this error:

/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/traitlets/config/configurable.py:84: UserWarning: Config option `cookie_name` not recognized by `HubOAuth`.
  self.config = config
Traceback (most recent call last):
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/traitlets/traitlets.py", line 528, in get
    value = obj._trait_values[self.name]
KeyError: 'oauth_client_id'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/vagrant/.pyenv/versions/jupyter/bin/jupyterhub-singleuser", line 6, in <module>
    main()
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 455, in main
    return SingleUserNotebookApp.launch_instance(argv)
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/jupyter_core/application.py", line 267, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/traitlets/config/application.py", line 657, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-7>", line 2, in initialize
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/traitlets/config/application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/notebook/notebookapp.py", line 1364, in initialize
    self.init_webapp()
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 393, in init_webapp
    self.init_hub_auth()
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/jupyterhub/singleuser.py", line 388, in init_hub_auth
    if not self.hub_auth.oauth_client_id:
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/traitlets/traitlets.py", line 556, in __get__
    return self.get(obj, cls)
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/traitlets/traitlets.py", line 535, in get
    value = self._validate(obj, dynamic_default())
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/traitlets/traitlets.py", line 593, in _validate
    value = self._cross_validate(obj, value)
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/traitlets/traitlets.py", line 599, in _cross_validate
    value = obj._trait_validators[self.name](obj, proposal)
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/traitlets/traitlets.py", line 907, in __call__
    return self.func(*args, **kwargs)
  File "/home/vagrant/.pyenv/versions/3.5.2/envs/jupyter/lib/python3.5/site-packages/jupyterhub/services/auth.py", line 439, in _ensure_not_empty
    raise ValueError("%s cannot be empty." % p