tommeagher / pycar14 Goto Github PK

A lot of "good" code has distracting parts. Which shall we prefer in writing these lessons? Some examples I'm thinking of:

shebang at beginning of file e.g. #!/usr/bin/env python

differentiate between import and directly running file:

if __name__ == '__main__':
    main()

with for automatically closing files:

with open('path/to/file.csv', 'r') as f:
    pass

Okay guys, I finally got Project 2 mostly done (I still need to write the output to a file, but that's simple enough). It's not optimized code by any means because I wanted to be able to talk about things step-by-step, but I was thinking of creating an optimized version that people could compare to.

Any feedback welcome -- sorry it's taken me this long to get something going!

Any thoughts on how best to collect and organize all of this material we'd like to give the class? I have just that iPython notebook for the walkthrough of the basics and a markdown file that's just a list of stuff we want to collect.

Do you have any favorite resources or tutorials you want to share here? @esagara, I think you said you had some scripts with a collection of helpful functions you'd put in there, right?

Turns out @chrislkeller is teaching a similar, albeit shorter, intro class on Python. I asked him to chat with us so we can steal ideas from one another.

On Monday, I'm going to send a note to everyone registered for both sections of the class, asking them to sign up for a free account at Wakari.io. Ideally, everyone will be signed up ahead of time.

I called Wakari yesterday. While they can't guarantee service for the free tier of accounts, it shouldn't be a problem for us all to access it simultaneously. If anyone has a problem, they can use their local iPython, if they need to.

In Wakari, I can easily pull down the Github repo (maybe we should make a "comments" branch that just includes the empty files we'll use during the class) and share the files and environment as a bundle with everyone. Through the Wakari dashboard, everyone will get a text editor and a terminal to run files. If we need it, it also includes iPython interpreter and notebooks. It already has requests installed. We may need to pip install bs4 or other 3rd party modules.

This should be a very easy platform for teaching. If you have a moment this weekend, please sign up for your own free account at Wakari.io. Go and kick the tires a bit. And help us spread the word on Monday to get the students to create their accounts before they get to Baltimore.

I added all your names to this file. Wasn't sure what contact info you all would want to list. Are we cool with just including Twitter handles?

I've been working on Project 2, but I'm having difficulty finding a decent national dataset that can reasonably be split into multiple CSVs without being enormous. (This is a little outside my wheelhouse, heh.) Anyone have suggestions?

I promised Jaimi that we'd get her the files to load on the computers (the back up copies of the commented pseudocode for each project) by 2/25.

Should we set an earlier deadline for at least rough drafts of everything? 2/21?

Also, should we all plan to meet up Weds evening to review everything and make sure we feel confident we're prepared for Thursday?

Hello all,

Just pushed a new commit with a simple web scrape and accompanying readme. Please review and post feedback on both the scraper and the really long walkthrough in the readme. I plan to write a second scraper using post requests as well.

I know @hbillings was wrapping up a few other things the last few days. This is just a reminder that we'll want to iron out this section too.

I roughed out the schedule in the repo. Obviously we can all collaborate on each piece, but maybe we should take the lead on each piece of the day. Take a look at the repo and the README and let us know which sections you want to start working on. Let's shoot to have all our pseudocode and working code by 2/7. Is that reasonable, @hbillings & @esagara?

Biscuits and gravy stat.

After looking at Project 3, I may have made Project 4 a bit too basic. Thoughts?

I'm not quite sure I am looking at the official version of the repository, and am a bit confused about what is expected for project no. 3.

I was going through this with the understanding that there was a narrative readme intro to scraping with BeautifulSoup and then 2 additional json/csv related project files (get_json, json_to_csv)

Now I am just seeing the Readme and a scraper file, which seems like an exercise file of the same info in the Readme file. Could I get an update on what is generally expected for project 3? Thanks.

From the bootcamp description:

The daylong class will introduce language basics and useful libraries in the course of a typical reporting project: scraping data from the Web, inserting it in a database and analyzing the results.

http://ire.org/events-and-training/event/973/1025/

So, should we add an element of dealing with a database? And if so, how do we want to handle that? Wakari's terminal does have support for SQLite. If we have to use the Windows PowerShell backup, that's going to be considerably more complicated. Thoughts?

Hey everybody,
One last thought. While NICAR is still relatively fresh in our minds, what do you think we should do differently next year?

Are there any other projects, or a different schedule we should look at? Do you want to refactor any of your code now, rather than waiting until we're in the hotel bar in Atlanta at the end of next February?

What do you think?

I added a few things today.

Here's the URL for the slide deck I put together for the beginning of the class: http://bit.ly/intropycar14

I think we can sprint through this piece. Just go round the room and introduce ourselves and set expectations for the day. I thought having the first hands-on task of the day being "import this" would be a good way to get people into it. So much of the Zen of Python I think really will speak to the reporters in the room.

After that, I thought we'd do a mad dash through the basics of variables and elementary types, saying as we did so that we'd be repeating these concepts over and over through the day. I did it as an iPython notebook on Wakari. As the students type each line, I can just press the run button and show the output and talk about each one.

The big problem I'm wrestling with at this point is this: I don't think we can do all of this in a half-hour. So, should I trim down the first lesson and we try to work all those ideas into the projects? Or should we adjust the timing of the day to make more time for this piece of it? What do you think?

Any and all feedback and suggestions will be much appreciated.

Thanks!

I think it would help beginners a lot if you named variables in a way that makes it much easier to see what's a variable and what's not. For example, something like "vrow" or "vfilename" or "varfilename" or "var_row". Anything that sets them apart from everything else. It gets confusing when you have a variable named "writer' and then a command that includes "writer" and stuff like that.

Before I get too far ahead of things, I wanted some feedback on the approach I was taking for lesson 1.

I just started working logically through the steps I would take to learn how to use python to do some fundamental things with a csv. Am I on the right track, or did you all have something else in mind?

We can break these up into separate issues however you like:

• What kind of environment do we want to use?
  ipython with what libraries?
  Wakari using the terminal? (if so, does that negate making a web page? Probably)
• Structure of the day?
  Start with the basics (types, assignment, iteration)
  Target a data set either a CSV we have or a website to scrape.
  Then what?

-Prework/postwork/tutorials?
What do we want to send people home with? A virtualenv and pip walkthrough is probably a good starter.

....then meetings swept me away. We can parse this out into the wiki

What are the most useful standard library or third-party modules we want to point people at first?

Maybe this is something we go through later in the day or just a list of resources we send them home with.

A few I thought of on the train:
Csv
Re
Requests
Bs4
mechanize
pee-wee
python-twitter
Csvkit
Xlrd http://www.python-excel.org/
Json

And a great resource of quick overviews on standard library modules, because the Python docs can be really daunting to newbies.
http://pymotw.com/2/contents.html