aashish24 / data

A list of open data repositories or interesting data sets.

Data

This page contains links to interesting open data or lists of interesting open data.

Lists

• Amazon Public Data Sets
• caesar0301/awesome-public-datasets
• Google Public Data Directory
• Google BigQuery Public Datasets
• Quora large public datasets
• DataPortals.org datacatalogs.org
• Inside-r.org
• Open Access Data simmons.edu

Personal (about you)

• Google Takeout
• Green Button Energy Use Data

Academic

• The Dataverse Project
• Dryad
• Academic Torrents
• Open Science Framework
• Open Science Data Cloud
• Nature.com Scientific Data
• DataCite.org

Financial

• NxCore
• Quandl
• Worldbank
• Kiva
• Bitcoin
  • Blockparser
  • Abe
• Black-market archives
• Open Corporates API

Crypto Exchanges

• Kraken API

Event

• GDELT
• AP Breaking News API
• Google Trends

Social Media

• Twitter
  • Tweepy
  • Twython
• Github
• Reddit
  • Stuck_in_the_matrix post
  • Internet Archive Reddit Corpus
• Facebook
• Instagram
• GNIP

Cyber

• Internet Census 2012
• National Vulnerability Database
• Defcon CTF Pcaps
• Netresec Pcaps List
• Simpleweb Traces
• CTU-13 Botnet
• ANT
• SecRepo
• Los Alamos CyberSec Events

Health

• Medicare
• NCBI Genome
• Open Payments Data
• Healthdata.gov
• Vanderbilt University Department of Biostatistics
• Cancer Genome Atlas
• cBioPortal Cancer Genomics Data
• WHO
• List of biological databases
• Broad Institute Cancer Datasets

Weather and Climate

• Berkeley Earth
• NOAA
• NASA Earth Exchange (NEX)

Machine Learning Specific

• Kaggle
• UC Irvine Machine Learning Repository
• mldata.org
• Google Ngram
• NIST Trec Data
• Netflix Prize
• Google Youtube Dataset
• Standford Question Answer Dataset

Machine Translation

• Opus Open Parallel Corpus
• UN Corpus

Computer Vision

• Labeled Faces in the Wild
• Computer Vision Online
• Berkeley Segmentation

Government Data

• Data.gov
• City of Berkeley Open Data
• courts.gov
• Google Public Transit Feeds
• European Union Data
• data.gov.uk
• open.canada.ca
• Homeland Infrastructure Foundation Data
• China Vitae
• Trade.gov api

Other

• UFO Reports
• NYC Taxi
• Uber TLC FOIL
• Microsoft Research GeoLife GPS

APIs (non-producing)

• Clarif.ai
• IBM Watson API

Organizations

• Harvard Institute for Quantitative Social Science
• Center for Open Science
• UC Berkeley DLab
• UC Berkeley AMP Lab

Knowledge Graph

• dbpedia

Privacy

Gov

• DHS Privacy Treshold Analysis
• DHS Privacy Impact Analysis Template
• NASA Privacy Impact Analysis
• HHS Privacy Impact Analysis

---

Submit new or interesting data links via pull request or email to [email protected]