taggartk / 2016-nicar-adv-sql

The queries and data for an Advanced SQL class at NICAR 2016

2016-NICAR-Adv-SQL

The queries and data for an Advanced SQL class at the NICAR 2016 conference.

This is a slightly modified version of the Advanced SQL class Liz Lucas taught at NICAR 2015. We're going to be using SQLite instead of MySQL. We're also going to use a free Firefox plugin that makes it easier to use SQLite on your computer. It should already be on the classroom computers, but you can also download it here.

All the SQL for the class is in this file: SQL_queries.md

We're going to use inspection data from the Department of Labor/Occupational Safety and Health Administration.

Here's the description from DOL's website: "The dataset consists of inspection case detail for approximately 100,000 OSHA inspections conducted annually. The dataset includes information regarding the impetus for conducting the inspection, and details on citations and penalty assessments resulting from violations of OSHA standards. Additionally, accident investigation information is provided, including textual descriptions of the accident, and details regarding the injuries and fatalities which occurred."

inspection, accident, accident_injury were downloaded from the DOL's site on 02/11/2016.

For this class, I limited the data to Jan 1, 2011 through December 31, 2015.

Record layouts:

• Data dict from OSHA
• Layouts for the tables we'll be using

What we'll cover:

• Truthing data tables and joins
• DISTINCT
• CREATE TABLE
• ALTER TABLE to add columns
• UPDATE to populate new columns
• Dealing with dates in SQLite (strftime)
• How to calculate difference in days between two dates
• Wildcards for more complex filtering
• Aliases in table names
• Subqueries

2016-nicar-adv-sql's Issues

add count distinct

SELECT count (DISTINCT activity_nr)
FROM osha_inspection_2011_2015_test4

load clean data files for 2011 to 2015

note about accident table dates

looks like it only goes through 2014?

Add a couple more examples of subqueries

Tell class about the record that was messing up importing into SQLite

SQLite failed to import

Notes — I was getting an error on this row because of the ‘R’ House
300753688,950615,,'R' HOUSE,540 MIDDLE RINCON ROAD,SANTA ROSA,CA,95409.0,A,0.0,N,S,8211.0,611110.0,A,B,,N,,,,,,,,PO BOX 2587,SANTA ROSA,CA,95405.0,N055 000020269,88.0,2011-07-29,2012-12-27,2011-10-05,2012-11-23,2012-12-30 04:29:49.429856,2011-07-29

Opened in Komodo and took out the apostrophes

SELECT *
FROM inspection
WHERE activity_nr = '300753688'

push data from local computer to repo

question about OSHA data

Most recent event date in accident table appears to be 2014-01-21 (whereas inspections run through 2015). Ask someone more familiar with OSHA data if this kind of lag is common,

Do we want to get into indexing?

http://www.w3schools.com/sql/sql_create_index.asp