tommeagher / pycar14

The Python mini boot camp at NICAR 2014 in Baltimore.

 888888ba            a88888b.  .d888888   888888ba     d88  dP   dP
 88    `8b          d8'   `88 d8'    88   88    `8b     88  88   88
a88aaaa8P' dP    dP 88        88aaaaa88a a88aaaa8P'     88  88aaa88
 88        88    88 88        88     88   88   `8b.     88       88
 88        88.  .88 Y8.   .88 88     88   88     88     88       88
 dP        `8888P88  Y88888P' 88     88   dP     dP    d88P      dP
oooooooooooo~~~~.88~oooooooooooooooooooooooooooooooooooooooooooooooo
            d8888P

#For the current version of the PyCAR class, visit the maintained repo here.

##Python mini boot camp at #NICAR14 in Baltimore

Let's use Wakari.io bundles for this class. At the end, we'll give you everything working in Github.

As a backup plan in case the internet is bad, we'll have the bundles on the machines, and we can use the Anaconda distro of Python's interactive interpreter in Windows PowerShell.

###Schedule for the day ####Introduction

• Who are you, what do you do, what do you want to learn?
• What will we learn
• what can I do with it?
• The goal of the day is to start to learn how to think about solving problems.

The Wakari bundle

####Project #1 Start with a CSV. Let's write some basic scripts to see what we're working with. Then we'll introduce some key Python types like strings, integers, lists, slicing and loops. After a white board exercise, we'll start with a file of pseudocode, and we'll walk through writing the program in Python code, running the file at the command line. We'll hold your hand through it, step by step.

####Project #2 We have a CSV of baseball player salaries. Let's figure out who makes the most money and examine some other biographical information about them, using dictionaries.

LUNCH

####How to debug your code, aka "Google and StackOverflow are your new best friends"

####Project #3 This section covers webscraping with Python using a get request, writing the data to a CSV.

####Project #4 Let's use an API to get information programatically to create a spreadsheet. Our data comes in a new format: JSON. Show on the whiteboard how it's basically a combination of data structures we already know about: Lists and dicts (arrays and objects).

At the end of the day, we'll send you home with with:

• A lightning dash through basic variables, types and functions.
• the working, commented code for each project from our git repo
• a walkthrough for setting up your machine at home with Python, version control and virtualenvs.
• Good libraries to explore with strong tutorials
• lifetime support guarantee from us & PythonJournos

###Help! If you're working through this code at home and have trouble, please let us know. The best option is to file an issue report of the bug so we can help you. You can also reach out to any of us on Twitter or ping PythonJournos.

pycar14's Issues

Next year

Hey everybody,
One last thought. While NICAR is still relatively fresh in our minds, what do you think we should do differently next year?

Are there any other projects, or a different schedule we should look at? Do you want to refactor any of your code now, rather than waiting until we're in the hotel bar in Atlanta at the end of next February?

What do you think?

Takehome

Any thoughts on how best to collect and organize all of this material we'd like to give the class? I have just that iPython notebook for the walkthrough of the basics and a markdown file that's just a list of stuff we want to collect.

Do you have any favorite resources or tutorials you want to share here? @esagara, I think you said you had some scripts with a collection of helpful functions you'd put in there, right?

Project #2

Okay guys, I finally got Project 2 mostly done (I still need to write the output to a file, but that's simple enough). It's not optimized code by any means because I wanted to be able to talk about things step-by-step, but I was thinking of creating an optimized version that people could compare to.

Any feedback welcome -- sorry it's taken me this long to get something going!

The database

From the bootcamp description:

The daylong class will introduce language basics and useful libraries in the course of a typical reporting project: scraping data from the Web, inserting it in a database and analyzing the results.

http://ire.org/events-and-training/event/973/1025/

So, should we add an element of dealing with a database? And if so, how do we want to handle that? Wakari's terminal does have support for SQLite. If we have to use the Windows PowerShell backup, that's going to be considerably more complicated. Thoughts?

Deadlines?

I promised Jaimi that we'd get her the files to load on the computers (the back up copies of the commented pseudocode for each project) by 2/25.

Should we set an earlier deadline for at least rough drafts of everything? 2/21?

Also, should we all plan to meet up Weds evening to review everything and make sure we feel confident we're prepared for Thursday?

Wakari.io

On Monday, I'm going to send a note to everyone registered for both sections of the class, asking them to sign up for a free account at Wakari.io. Ideally, everyone will be signed up ahead of time.

I called Wakari yesterday. While they can't guarantee service for the free tier of accounts, it shouldn't be a problem for us all to access it simultaneously. If anyone has a problem, they can use their local iPython, if they need to.

In Wakari, I can easily pull down the Github repo (maybe we should make a "comments" branch that just includes the empty files we'll use during the class) and share the files and environment as a bundle with everyone. Through the Wakari dashboard, everyone will get a text editor and a terminal to run files. If we need it, it also includes iPython interpreter and notebooks. It already has requests installed. We may need to pip install bs4 or other 3rd party modules.

This should be a very easy platform for teaching. If you have a moment this weekend, please sign up for your own free account at Wakari.io. Go and kick the tires a bit. And help us spread the word on Monday to get the students to create their accounts before they get to Baltimore.

Compare notes with others

Turns out @chrislkeller is teaching a similar, albeit shorter, intro class on Python. I asked him to chat with us so we can steal ideas from one another.

training suggestion

I think it would help beginners a lot if you named variables in a way that makes it much easier to see what's a variable and what's not. For example, something like "vrow" or "vfilename" or "varfilename" or "var_row". Anything that sets them apart from everything else. It gets confusing when you have a variable named "writer' and then a command that includes "writer" and stuff like that.

confused about the state of project 3

I'm not quite sure I am looking at the official version of the repository, and am a bit confused about what is expected for project no. 3.

I was going through this with the understanding that there was a narrative readme intro to scraping with BeautifulSoup and then 2 additional json/csv related project files (get_json, json_to_csv)

Now I am just seeing the Readme and a scraper file, which seems like an exercise file of the same info in the Readme file. Could I get an update on what is generally expected for project 3? Thanks.

Breakfast

Biscuits and gravy stat.

The introduction

I added a few things today.

Here's the URL for the slide deck I put together for the beginning of the class: http://bit.ly/intropycar14

I think we can sprint through this piece. Just go round the room and introduce ourselves and set expectations for the day. I thought having the first hands-on task of the day being "import this" would be a good way to get people into it. So much of the Zen of Python I think really will speak to the reporters in the room.

After that, I thought we'd do a mad dash through the basics of variables and elementary types, saying as we did so that we'd be repeating these concepts over and over through the day. I did it as an iPython notebook on Wakari. As the students type each line, I can just press the run button and show the output and talk about each one.

The big problem I'm wrestling with at this point is this: I don't think we can do all of this in a half-hour. So, should I trim down the first lesson and we try to work all those ideas into the projects? Or should we adjust the timing of the day to make more time for this piece of it? What do you think?

Any and all feedback and suggestions will be much appreciated.

Thanks!

CONTRIBUTORS

I added all your names to this file. Wasn't sure what contact info you all would want to list. Are we cool with just including Twitter handles?

Project3 - Feedback, suggestions and comments

Hello all,

Just pushed a new commit with a simple web scrape and accompanying readme. Please review and post feedback on both the scraper and the really long walkthrough in the readme. I plan to write a second scraper using post requests as well.

Modules to cover

What are the most useful standard library or third-party modules we want to point people at first?

Maybe this is something we go through later in the day or just a list of resources we send them home with.

A few I thought of on the train:
Csv
Re
Requests
Bs4
mechanize
pee-wee
python-twitter
Csvkit
Xlrd http://www.python-excel.org/
Json

And a great resource of quick overviews on standard library modules, because the Python docs can be really daunting to newbies.
http://pymotw.com/2/contents.html

Project 2

I've been working on Project 2, but I'm having difficulty finding a decent national dataset that can reasonably be split into multiple CSVs without being enormous. (This is a little outside my wheelhouse, heh.) Anyone have suggestions?

Best practices or ease of reading for beginners?

A lot of "good" code has distracting parts. Which shall we prefer in writing these lessons? Some examples I'm thinking of:

shebang at beginning of file e.g. #!/usr/bin/env python

differentiate between import and directly running file:

if __name__ == '__main__':
    main()

with for automatically closing files:

with open('path/to/file.csv', 'r') as f:
    pass

Let's take assignments

I roughed out the schedule in the repo. Obviously we can all collaborate on each piece, but maybe we should take the lead on each piece of the day. Take a look at the repo and the README and let us know which sections you want to start working on. Let's shoot to have all our pseudocode and working code by 2/7. Is that reasonable, @hbillings & @esagara?

Feedback on Project 4

After looking at Project 3, I may have made Project 4 a bit too basic. Thoughts?

Project 1

Before I get too far ahead of things, I wanted some feedback on the approach I was taking for lesson 1.

I just started working logically through the steps I would take to learn how to use python to do some fundamental things with a csv. Am I on the right track, or did you all have something else in mind?

Debugging advice

I know @hbillings was wrapping up a few other things the last few days. This is just a reminder that we'll want to iron out this section too.

Lots of questions

We can break these up into separate issues however you like:

• What kind of environment do we want to use?
  ipython with what libraries?
  Wakari using the terminal? (if so, does that negate making a web page? Probably)
• Structure of the day?
  Start with the basics (types, assignment, iteration)
  Target a data set either a CSV we have or a website to scrape.
  Then what?

-Prework/postwork/tutorials?
What do we want to send people home with? A virtualenv and pip walkthrough is probably a good starter.

....then meetings swept me away. We can parse this out into the wiki