18f / crime-data-explorer

Moved to https://github.com/fbi-cde

Crime Data Explorer

The FBI collects and publishes Uniform Crime Reporting (UCR) data on an annual basis. Over 18,000 law enforcement agencies across the country voluntarily participate in the program by submitting data through a state UCR program or directly to the FBI. This open data project is part of our ongoing efforts to improve the accuracy and timeliness of the nation’s crime statistics.

The Crime Data Explorer is a collaboration between 18F and the FBI Criminal Justice Information Services (CJIS) Division. It aims to make national, state, and local crime data more accessible and understandable to the public.

Available datasets

The FBI collects several types of data. These datasets are available for download via the Crime Data Explorer and the underlying API:

• Summary Reporting System (SRS): monthly offense, arrest, and clearance data for crimes known to law enforcement from 1960–2014.
• National Incident-Based Reporting System (NIBRS): reports from individual crime incidents with offense, location, victim and offender demographics, property damage, and arrestee details from 1991–2014.
• Estimated data: Summary (SRS) data that has been estimated to create trends for crime occurrences on the state and national level. The FBI has traditionally included estimates in their annual publications.
• Police Employee Data: totals for full-time sworn and civilian law enforcement employees by state from 1960–2014.
• Uniform Crime Reporting Participation Data: SRS and NIBRS participation rates for individual law enforcement agencies by state and year from 1960–2014. These datasets are available for download; we’re working on adding them to the API:
• Law Enforcement Officers Killed and Assaulted (LEOKA): totals for officers who were killed or assaulted in the line of duty from 1974–2014.
• Hate Crime: totals by state, year, and bias motivation for hate crimes across the country from 1991–2014.
• Cargo Theft: totals by state, year, property type, and property value for cargo thefts from 2013–2014. This is a relatively new dataset for the FBI.
• Human Trafficking: totals by state and offense for commercial sex acts and involuntary servitude from 2013–2014. This is also a new dataset for the FBI.

About the data

The FBI’s role in collecting this data

To ensure crime data is uniformly reported across the nation, the FBI provides contributing law enforcement agencies with documentation that explains how to classify and score offenses. Though crime definitions vary from state to state, the FBI asks agencies to report offenses according to Uniform Crime Reporting guidelines.

Although the FBI makes every effort through its editing procedures, training practices, and correspondence to ensure the validity of the data it receives, the accuracy of the data depends on the adherence of each contributor to the established standards of reporting. It is the responsibility of each state UCR program or individual contributing law enforcement agency to submit accurate data or correct existing errors.

Avoid rankings and comparisons

Since crime is a sociological phenomenon influenced by a variety of factors, the FBI discourages ranking locations or making comparisons between states, counties, or agencies as a way of measuring law enforcement effectiveness. Some of this data may not be comparable to previous years because of differing levels of participation over time.

Unestimated data

While the FBI has traditionally included estimates in their annual publications, the Crime Data Explorer provides data without estimates or modifications through downloads and in agency-level views. As a result, the data available for download may vary from the FBI’s annual reports.

Working with NIBRS data

Each incident-based (NIBRS) ZIP file includes a group of CSVs that you can join together with the internal incident ID or the law enforcement agency’s Originating Agency Identifier (ORI).

The incident IDs are generated to make it easier to work with the data. These numbers are not specific to local law enforcement agencies and bear no relation to other identifiers that could be linked to a real case number.

Related resources

We encourage you to explore, make suggestions, and contribute to our code. This repository, Crime Data Explorer, is the best place to submit general feedback about the site.

• Crime data API: code for the RESTful API behind the Crime Data Explorer
• Crime data style: visual style guide and user interface component library
• Uniform Crime Reporting (UCR) documentation: user manuals developed by the FBI
• API documentation: technical documentation for the crime data API

Get involved

We’re thrilled you want to get involved!

• Read our contributing guidelines. Then, file an issue or submit a pull request.
• If you’re a developer, follow the installation instructions in the README.md page of each repository to run the apps on your computer.
• If you have questions about the data, including potential errors or discrepancies, please email the FBI Criminal Justice Information Services Division at [email protected]. Copyright and licensing This project is in the public domain within the United States, and we waive worldwide copyright and related rights through CC0 universal public domain dedication. Read more on our license page.

Other project repositories

• 18f/crime-data-api
• 18f/crime-data-frontend
• 18f/crime-data-prototypes
• 18f/crime-data-style

crime-data-explorer's Issues

Explore new glossary icon

@brendansudol and I discussed that our current glossary icon feels kind of heavy and looks more like a rectangle than a book. We want to explore another, lighter and more book-y option, similar to FEC perhaps.

Implement an "All violent crime" and "All property crime view" for national and state level views

Build out view that includes all violent crime or property crime and mini-trend lines of all the sub categories of crime. Include the interactivity between charts that was explored through the prototype.

https://crime-data-prototypes.fr.cloud.gov/demos/city-explore/

Pre-launch test plan

Develop a plan and timeline for pre-launch testing and validation of the site content with expert users.

Agency perspective: Explore adding state and national data to agency page

As a user, I view agency data in context because i have a sense of how it compares to state and national trends, even if the numbers are in a different format.

Agency perspective: Refine overview content

As a user, I understand what I am looking at when viewing an agency. Consider adding a sentence that speaks to what the state and national trends are.

Incident details: Add ethnicity to offender/victim demographics

Display ethnicity detail in addition to race in the NIBRS details.

Review trend charts after adding estimated data

If necessary, refine trend chart design based on estimated data.

Content: Review table 8 footnotes

Review table 8 footnotes to make sure we are accounting for everything the FBI tells users of the UCR data.

Add new crime types to the explorer

agency view: show state and national sparklines

As a user, I can compare agency data with state and national trends for that offense.

Notes:

Clicking on these trend lines will take you to the page for that result (either the state or national perspective).

Explore loading state for intro paragraph on Explorer

There is a loading icon for the visualizations, but not for the intro paragraph, so there is a jump from landing to when it loads which makes the page look broken. We should explore an option maybe similar to facebook's content loading visual.

http://cloudcannon.com/deconstructions/2014/11/15/facebook-content-placeholder-deconstruction.html

Add NIBRS dimensions based on offense selected (needs definition)

As a user, I see varying NIBRS attributes by crime type. For example, if I look at Homicides, I may see weapon types or for Motor Vehicle theft, I may see types of vehicles stolen.

Estimated data: Refine estimation text

As a user, I understand that I am viewing estimated data for states and the nation. I can dig deeper to find out the methodology behind the FBI's estimations.

Update Trend and NIBRS chart designs to match latest spec

Annotated files coming...

Agency perspective: Link to agency NIBRS download

Refine design for download button. Aviva, please link to your updated file.

agency view: download agency offense data as a CSV (bar chart downloads)

As a user I can download the agency data displayed on the page as a CSV. This issue covers two separate download actions, one for the bar chart (counts) and another for the aggregated NIBRS dimensions (table). The formatting of the CSVs should be similar to how we handle these downloads at the state level, with the exception being that the agency counts will include a column for clearance.

Content: Estimated data

Add definition / methodology as a glossary item. (This may be a repeat of another issue.)

Provide context for questions for Cynthia

We setup a recurring call with Cynthia to discuss technical questions that either affect our implementation, or which will help us build confidence in how data is presented by the CDE.

We said we would send the discussion topics in advance, but should provide necessary context when doing so, by for instance, explaining how we propose defining urban vs. rural so they can weigh in.

https://docs.google.com/document/d/1nqLc-6QzDrspMRJgrVbvth4e9auAFe6HVovsK6n9kXI/edit

Sprint 10 research plan

Draft a research plan to better understand:

1. Users needs when navigating to an agency
2. Usability and effectiveness of the content on the agency page
3. The impact of variations in data types to the user experience.

Make territory data accessible via the CDE as a download

As a user, I can access data for US territories that participate in the UCR program via the CDE.

agency view: add programmatically-generated overview text

As a user I can view introductory text about the agency I’m viewing that explains what data type it reports (SRS or NIBRS) and whether it is classified as urban or suburban.

Note:

We need to specify how we will determine if an agency is classified as urban or suburban.

Trend chart exploration—try year dots

Explore & prototype pathways for agency navigation (search)

Explore and implement designs for allowing users to navigate to a specific agency.

Update trend chart to new spec

Annotated file here

https://drive.google.com/open?id=0B9bDBpzeNNaXektaaVlsOVBKNHM

Content: Glossary definitions

Add definitions for cleared, reported, estimated data

Incident details: Location

Change name of "Incident Location" to reflect the data source that the information is being pulled from. If it is from an offense table it should be Offense location. Pair with @jharris

Content: Define agency or jurisdiction

Review UCR manuals definition of jurisdiction to craft a definition of agency and add it to the site where appropriate.

Incident details: Race and Ethnicity displays

Check OMB's guidelines for displaying race and ethnicity to ensure the CDE is in compliance.

Add new geographical perspectives

Agency perspective: Incident details table

Implement incident details table. Aviva, please add a link to the annotated table files.

Agency perspective: Navigation with no data available

As a user, when I search for an agency that does not participate or has no data available, the agency name appears light grey in the dropdown. (Will this ever happen?) If we don't have data why would it be in the list?

Adjust red line for lists to match line height of text

Content: Considerations for interpretation

Consider adding content provided by the Idaho state rep on agency level pages and in the readme or about pages.

Considerations for Interpretation: Crime activity in a particular community can be deceiving unless specific factors are examined and taken into consideration before conclusions are made. Some conditions affecting the type and volume of crime are:* population size, density, and degree of urbanization* population variations in composition and stability* economic conditions and employment availability* cultural factors and educational, recreational, and religious characteristics* effective strength of the police force* administrative and investigative emphases of law enforcement* prosecutorial, judicial, and/or correctional policies* values established by family, community, and society in general* citizens’ attitude toward crime and police* crime reporting practices of citizens

Crime in Idaho
https://isp.idaho.gov/BCI/CrimeInIdaho/CrimeInIdaho2015/Introduction.pdf

About: adjust content

Agency perspective: Design for "covered by"

As a user, I am aware when an agencies data includes that of other agencies. I am able to see a list of agencies that an agency covers. If I search for an agency that is covered by another, I am sent to the page of the agency that covers it. If an agency is covered by for less than 12 months but reports 12 months of data, the additional data is aggregated into the covered by agency. (Verify with Cindy and Sammy.)

Content: API documentation

As a user of the API, I understand what the endpoints will return. Pair with Jeremia and Jacob.

Content: Incident details: Remove year range from title

As a user, I am mislead when I see a year range in the incident title and that differs for when NIBRS data is available.

Content: Data may differ or change

As a user, I understand how and why data may differ in the CDE over time or in comparison to state publications.

Agency perspective: Refine bar chart

Refine bar chart design. Address issues around cleared and reported. Considerations: covered by agencies, population...

Incident details: Make dynamic numbers bold

As a user, I notice dynamic numbers in the incident details description because they are bold.

agency view: implement related links

As a user I can view the agency website as a related link for the agency I'm viewing for.

Download & docs: reduce first column font size

Explore & define opportunities for improving the download experience

agency view: view agency counts as a bar chart

As a user, I can view the number of offenses reported and cleared for the agency I am viewing based on the time range I’ve selected.

Notes:

--The default view is 10 years
--Historical data is limited to 1995
--Users can select individual years within the range to view counts for that year
--if the agency reports SRS data only, this will be the only chart displayed on the page. If the agency reports NIBRS data, the chart will reflect summarized NIBRS counts, just like we handle state reporting.

Explore new design for header divider lines

Thin lines as header dividers sometimes look strange with shorter paragraph content—explore other design options for header styles/dividers

Incidents details: Relationship to offender

As a user, Relationship to offender categories are organized in a logical manner and facilitate easier interpretation of the data.

Incident details: Display all race & ethnicity elements even if they have 0 reported

Display elements, such as offender/victim ethnicity and race, even if they have 0 reported.

CJIS visit planning

Define goals, make plan, prepare necessary materials.

Agency perspective: About the data

As a user, I am given information I need to responsibly interrupt the data from the explorer view. Consider whether we need to add considerations for interpretation or other caveats based on the granularity of the data. Also, this is the only place we are showing raw data. Should that be explained?

agency view: view aggregated agency incident data as a table

As a user, I can view related incident data for the agency I am viewing, assuming that agency submits NIRBS data.