As well as pandas and the standard numpy/scipy stack, the library has the following dependencies: lxml, openpyxl, gdata, pods
pip install lxml
pip install openpyxl
pip install gdata
pip install pods
In 2014 Corinna Cortes and I
were NIPS program Co-Chairs. Alan Saul was our Program Manager. As part of the
process we wrote a lot of scripts for processing the data. The scripts I wrote
used the IPython notebook (now Project Jupyter) and
pandas. It was always my intention to summarise this work in case others find
it useful. It is also quite a good document for summarising what is involved in
program chairing a major conference like NIPS.
In May 2021, I separated out the utility files used for the notebooks into a separate python module. The library, cmtutils, which manages the submissions. For
reviewer management (which was the first thing written) the scripts are based
around a local mirror of the CMT user data base in SQLite. For review management
we moved things much more towards pandas and used CMT as the central
repository of reviews, exporting them on a daily basis.
A lot of communication was required between CMT through imports and exports. Some of the links used for CMT exports are available here. The various tasks are structured in IPython notebooks in the conference repo. The code used was first written for the NIPS 2014 conference, but ideas were based on experience from using CMT for AISTATS 2012 and some preliminary code written then (for example for importing the XML formatted version of Excel that CMT uses).
Right from the start it was felt that being able to import and export
information to Google spreadsheets would be very useful. With this in mind an
interface between pandas and Google sheets was created (initially just for
reading, then later for updating). This made it much easier to import reviewer
suggestions and export information about paper statuses to reviewers. That
software has been spun out as part of a suite of tools for Open Data
Science that is
available on github here. These notebooks are
also available in their own github repository for conference
software.
A note on the code. A lot of this code was written 'live' as reviews were coming
in or as a crisis required averting. The original code for sharing information
via Google spreadsheets was written across two or three days whilst on a family
holiday in the Catskills. Much of the code could do with rewriting, and this is
an ongoing process that I hope other conference chairs or program managers will
contribute to. It is shared here as a record of the work required for a
conference like NIPS as well as in the hope that it will be useful for others.
It is not shared as an example of 'best practice' in python coding. There are
some parts I'm proud of and others I'm not. However, I think it is a very good
example of how the notebook can be used with python and pandasto do 'live'
data processing of some importance whilst under a great deal of pressure. I
can't imagine having done it quite like this with a different suite of tools.
As well as the installed files, you need to create a file called .cmt_user.cfg in your home directory and give it the following fields:
# This is a user's personal configuration file for CMT
[conference]
short_name = NIPS
year = 2014
chair_email = [email protected]
[cmt]
export_directory =
[gmail]
account =
name =
password =
[google docs]
# Here include the spreadsheet keys of program committee and reviewer candiates
program_committee_key =
reviewer_candidate_key =
buddy_pair_key =
global_results_key =
[review data]
directory =
file = all_reviews.pickle
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent