johntigue / trumpest

We know datas. We have the best datas.

Trumpest

The goals of this project is to provide technology to accelerate investigations into Trump corruption. The world needs more tools such as those mentioned in 4 approaches to building collaborative data infrastructures for journalism.

The core novelty of this project is to interconnect data sources on the web, rather than collect them together. The main benefit is the interconnection of public data. A corollary of this approach is that private data can be interconnected with public data.

Action items include:

• collect best practices for data authors for harmonizing multiple repositories of data related to Trump
• use CSVW to make the harmonization explicit to software
• write software which can be used by data authors and researchers to make them more productive

The core of the idea is to use CSVW to create web structures which allows separately developed data repositories (“databases”) to be interlinked over the Web. This essentially creates a web-tech-based distributed “database” of Trump related data.

For example, if two repositories of data have CSVs with columns containing individuals names, then CSVW can be used to map between those two repos so that “joins” can be performed to find individuals on which there is data within both repositories.

The term “database” as used herein explicitly does not mean RDBMS/SQL tech. This is only about how data is expressed on the web, which may very well have a RDBMS behind it - or be based in a Google Doc spreadsheet - but such details are irrelevant.

trumpest's Issues

Check up on CPI's license

PublicI/pfd-parser#2

Geocoding

See

• docs/best_practices.org
• geocoding.org

WH staff financial disclosures: needs CSV and csv-metadata.json

Timeline machinery: survey

Some candidates

• https://technotarek.com/timeliner/
  • themoscowproject.com uses this
• https://github.com/zz85/timeliner
  • Timeline.js by Marcin Ignac
  • Keytime Editor by Matt DesLauriers
  • Frame.js by Ricardo Cabello (Side note: mrdoob's talk on this also showcase interesting editors used by the demoscene)
  • TweenTime by idflood.
• http://schoolofdata.org/handbook/courses/timeline-tools/
• http://timemapper.okfnlabs.org/
• http://propublica.github.io/timeline-setter/
• http://timeline.knightlab.com/docs/index.html

TimelineJS: could upcast from their proprietary multi-column time format to ISO 8601

Modifying the code for TimelineJS seems foolish. Better simply to look at TimelineJS spreadsheets as a build artifact transform of the core data.

So, core data has times (start and optional stop) as ISO 8601 and some transformer hacks each of those single column times into the multiple columns
that TimelineJS wants.

The only loss here is that non-tech folks will not be able to enter data in Google Docs and see the results... Could upcast from TimelineJS time formats to ISO 8601. See code in tools/timelines/timeline_dumper.js for much of that already available.

See this repos notes on TimelineJS for more details.

HXL probably can offer value

• http://hxlstandard.org/
• http://hxlstandard.org/standard/1_0final/dictionary/
  • 1.1. Places:
    • #region
    • #country
    • #geo
    • +lat, +lon, +coord, etc.
    • etc.
  • Dates
    • +start, +end,
  • #org+name (the name of an organisation)
  • +url

ProPublica's Beachhead Epic

Research: can a stock browser load CSVs hosted on GitHub?

BuzzFeed's TrumpWorld Epic

See this repo's README on BuzzFeed's TrumpWorld for more info.