unidata / python-workshop

A series of Jupyter Notebooks on exploring Unidata technology with Python. See website for more information.

This repository holds the archived content for Unidata's Python Training Workshops.

It has been superseded by Unidata's Python Training repository.

python-workshop's Issues

Update welcome slides

Update welcome slides

Workshop materials

• git/github #16
• conda/binstar #17
• Ipython notebook #22
• numpy/matplotlib #18
• Geoscience vis w/ matplotlib/cartopy #19
• NetCDF case study #21
• Pythonic #23
• Siphon #24
• MetPy #25
• Point obs #20
• WMS #26
• Radar composites #27

Set up period?

If we are going to have the users do the install, should we have a set up period scheduled?

Update setup.md

Update setup.md

get_latest_dods_url fails

the pytds function get_latest_dods_url fails on several datasets, for example:

http://thredds.ucar.edu/thredds/catalog/grib/NCEP/DGEX/CONUS_12km/catalog.xml
http://thredds.ucar.edu/thredds/catalog/grib/NCEP/DGEX/CONUS_12km/files/catalog.xml

http://thredds.ucar.edu/thredds/catalog/nexrad/level2/KFTG/catalog.xml
http://thredds.ucar.edu/thredds/catalog/nexrad/level2/KFTG/20130701/catalog.xml

---

In [16]: pytds.util.get_latest_dods_url(url)

UnboundLocalError Traceback (most recent call last)
in ()
----> 1 pytds.util.get_latest_dods_url(url)

/Users/sarms/dev/unidata/tds-python-workshop/pytds/util/catalog_util.py in get_latest_dods_url(dataset_url)
76 """Given a top level dataset URL, return the latest dataset."""
77 o = urlparse(dataset_url)
---> 78 resolver = get_resolver_xml_url(dataset_url)
79 latest = None
80 if (resolver is not None):

/Users/sarms/dev/unidata/tds-python-workshop/pytds/util/catalog_util.py in get_resolver_xml_url(dataset_url)
53 latest = s.get("name")
54 latest_base = s.get("base")
---> 55 ds = find_dataset(root,latest)
56 if (ds is not None):
57 # TODO: generalize this to handle relatives paths starting with a '/'

/Users/sarms/dev/unidata/tds-python-workshop/pytds/util/catalog_util.py in find_dataset(root, service_name)
39 return ds
40 else:
---> 41 if find_dataset(ds, service_name) is not None:
42 return find_dataset(ds, service_name)
43

/Users/sarms/dev/unidata/tds-python-workshop/pytds/util/catalog_util.py in find_dataset(root, service_name)
36 for ds in root.findall(xmlns_prefix + 'dataset'):
37 sn = ds.find(xmlns_prefix + "serviceName")
---> 38 if ((sn is not None) and (service_name == sn.text)):
39 return ds
40 else:

UnboundLocalError: local variable 'sn' referenced before assignment

Update notebooks

• Port syntax, etc. to Python 3
• Move from pyudl to siphon
• Make sure they work

Regardless, they should be saved to bring them up to IPython's latest notebook format (and eliminate the scary message when you load them on IPython 3).

Need to check:

• CompositeRadar.ipynb
• Geocoded_METAR.ipynb
• matplotlib.ipynb
• netcdf-by-coordinates.ipynb
• numpy.ipynb
• reading_netCDF.ipynb
• wms_sample.ipynb
• writing_netCDF.ipynb

Update figshare links for conda and github

Update figshare links for conda and github

Materials for git/GitHub

Just a tracker for reviewing/enhancing what we have for git and github.

Explain how to start the OS X Terminal and Windows cmd.exe

Explain how to start the OS X Terminal and and Windows cmd.exe

Improve colormaps for materials

Should look at improving colormaps across the board, using viridis by default. I want jet eradicated.

This depends on Unidata/MetPy#75

Materials for radar composites

Tracker for reviewing and enhancing materials for Accessing radar composites.

Materials for using WMS

Tracker for reviewing/enhancing materials for Exploring WMS (w/ ncWMS & Cartopy extensions)

Materials for conda/binstar

Tracker to review and/or enhance materials about conda and binstar.

Materials for MetPy

Need to create materials for MetPy. Should start with the exercise created for the Users' workshop.

Materials for Pythonic section

Need to find Johnny Lin's materials and see what this actually covered at SEA.

SIPHON_NCSS_Example_Cartopy notebook broken

Currently complains about missing time1 variable

xray is now called xarray

replace xray references with xarray

CompositeRadar notebook broken

Complains about a color table issue near end of NB

Update workshop schedule links in README

/ipython-notebook-examples probably needs an update

/ipython-notebook-examples probably needs an update

Fix environment instructions.

At a minimum, we need to put back the correct instructions for setting up the ipython/jupyter notebook in our environment. This means installing ipython-notebook rather than a hand accumulated list of tornado, jinja, pyzmq, and ipython, which is no longer accurate.

Better yet, we could restore the instructions to the use conda env create which references the up to date environment.yml file.

Add new counties map to cartopy notebook.

Delete cruft

Delete files in github repo that are no longer needed or relevant.

matplotlib notebook broken

currently complains about a DAP server error

Materials for matplotlib and cartopy

Tracker to review/enhance materials on matplotlib/cartopy.

Add some more resources to site

• Resources on types in Python (mypy, Cython?) Maybe PEPs?
• Bash config stuff for git prompt modifications
• git/Python cheatsheet
• Links to documentation for libraries

Improve Unidata Python Dockerfile

Improve Unidata Python Dockerfile so that it may be used for the workshop, if someone wants.

Materials for NetCDF case study

Tracker for reviewing/enhancing materials for the NetCDF case study.

Materials for IPython notebook

Tracker for reviewing/enhancing materials for IPython notebook section.

vagrant

Decide if we should keep or remove the vagrant material. Also, if interested, I have a Docker container that runs the workshop material. It is quite nice actually, nice and slim.

reading_netCDF notebook is failing on sfctmp = gfs.variables['Temperature_surface']

chokes on

sfctmp = gfs.variables['Temperature_surface']

tag repo when workshop is over

tag repo when workshop is over w/ 'summer-2015-workshop'

Materials for numpy/matplotlib

Tracker to review and enhance materials for the intro to numpy with matplotlib section.

Add `environment.yml`

We can put all the channels and packages in this file and just use:
conda env create
from the directory with the file. This should prevent having dependencies duplicated everywhere, and ease installation.

ipython notebook deprecated

remove references to ipython notebook and replace with jupyter notebook

Prepare the workshop image ASAP

The machine is leaving Tuesday July 14!

Provide git clone alternative

You don't necessarily need git to download the notebooks, you can click on the Download ZIP button.

matplotlib magic inserted after matplotlib import (numpy notebook)

For some reason this generates an error:

import matplotlib.pyplot as plt
%matplotlib inline
ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory

but this does not

%matplotlib inline
import matplotlib.pyplot as plt

make sure matplotlib magic is inserted before the matplotlib import

Materials for METAR/Point obs

Tracker for reviewing and enhancing materials for Point observations w/ TDS

This one especially needs some love to make good use of Siphon and get rid of a lot of ugly hand-coding.

Materials for Siphon

Need to create new materials to talk about Siphon. Steal liberally from the existing examples.

Verbiage for git client

Per Ben, point users to git client installation instructions in the README.

binstar -> anaconda

Binstar has been rebranded to anaconda. This means all channel references to conda.binstar.org should be updated to anaconda.org (the former is not working for me today, but I'm not sure if it's a permanent shutdown). Also, the command line tools are under anaconda now, not binstar.

Jupyter Intro Material

• Need to update to the latest versions from Jupyter
• No need to experiment with plotting in jupyter-examples/ 3. Plotting.ipynb
• Eradicate all mention of pylab

Send out email to workshop attendees by Friday (7/17)

Send out email to workshop attendees by Friday (7/17).

Improve query docs

These need the include_inherited (or whatever Sphinx uses) so that the subclasses of DataQuery have a full listing of useful methods. This is especially important given that the subclasses are all over the place.

Thoughts for next time.

Go ahead and capture any take aways from this time.

SIPHON_Radar_Server_Level_3 notebook broken

Currently fails on second code line

Notes from Madison workshop

Here's a dump of the notes we took while giving the workshop in Madison; these need to be turned into individual issues at some point:

Workshop notes

TextWrangler for mac if users do not want to use command line - add as a suggestion for mac users? Notepad for windows, gedit for linux?
Types in Python - dangerous or not?
git add - didn’t we already add?
Do not have them make their own plot in the jupyter-examples/ 3. Plotting.ipynb , as this is covered in intro to numpy and matplotlib notebook
intro to numpy and matplotlib notebook - reword the “reduce by faster of three” exercise prompt. Maybe “For each dimension, select every third element”
busted link in “sharing notebooks” jupyter example
Link to Docs
Presenter notes - scope of exercise, how interactive, how detailed

Things to add to site:

Resource on types in Python
Add bash config stuff for git prompt modifications
Find GIT cheat sheet
PEPs dealing with types in base python?
git ssl issues: git -c http.sslVerify=false clone …
Anaconda (old version clashes???)

Day 2:

Might be good to do pythonic data analysis at before read/write netCDF?

Pythonic data analysis
User getting error reading netCDF. LD_LIBRARY_PATH is set to look in several places for libs, and the end result is that it is finding an older copy of HDF5, which is causing it to be loaded instead of the condo version. This causes the notebook kernel to restart when trying to open a netCDf files, due to a library incompatibility.
make variable names and linestyles separate lists (around cell 10 in solutions notebook)? Makes it more readable.
states dict - capital not capitol
show adding scalar to array with units.
MetPy Units and Upper Air
Remove StringIO from import in cell 2
add hodograph
Siphon HRRR Carroty
list variables from ncss object: ncss.variables
MetPy Surface
Delete time delta from “now”
Radar
Sort stations
Remove carroty bug code

Round 2:

Day 1:

pint<0.7 in environment.yml -> version needs to go
Why isn’t the interactive widgets (or all from jupyter) example more domain specific?
Seriously, we need to murder pylab from the jupyter examples
We should also try to standardize a few more things in the notebooks (subplots vs. add_subplots vs. axes())

Siphon Carroty HRRR

remove calls to plot and use ax, fig objects instead

add colorer and label

ax.set_title("time[t_step].isoformat()")

cax = fig.colorbar(mesh)

cax.set_label(ncvar.units)

Pythonic data analysis lists language features, then jumps into an advanced Matplotlib example with no warning—need to tie that first section into introduction

Ryan really needs to re-examine exercise questions

The workshop page should have a top-level set of resources, cheatsheets, and links to relevant documentation for numpy, matplotlib, metpy, etc.

Why Vagrant instead of native Anaconda install?

For the workshop, why have folks use a virtual machine via Vagrant instead of native Anaconda install? Is it because you are using conda modules that are not built yet for Mac and/or Windows?
If so, which ones haven't been built yet?

Siphon_XRAY_Cartopy_HRRR currently giving errors

Siphon_XRAY_Cartopy_HRRR currently giving errors. see
http://mybinder.org/repo/Unidata/unidata-python-workshop