Note: No longer updated. I have resigned from Wikipedia in protest at content deletion.
This public repository archives data over time from various public sources on the web.
Data is presented as timestamped CSV files, for maximum compatibility.
It is hoped that this data will be useful to those producing visualizations or analyses.
Code is included.
Shown here in GIF format. There is a better (smaller/higher resolution) webm format also generated.
Generates static SVGs.
Source images were this one and this one.
For a China map, the following command sequence will grab data from DXY and render it.
./build china
You now have timestamped JSON, CSV and SVG files in the data-sources/dxy/data/ subdirectory.
For a world map, the process is similar.
./build world
You now have timestamped CSV and SVG files in data-sources/bno/data.
Probably an incomplete list:
- bash
- perl
- php
- imagemagick
- ffmpeg
- wget
Includes detail on foreign sources, individual provincial update URLs. Updated once per day or so.
- Data is timestamped in US Eastern Standard Time (ET) timezone
- Direct link to latest data
High level information without specific source URLs. However, this is updated frequently and appears to be the best available data.
- Data is timestamped in Beijing (CST) timezone
- Direct link to latest data
- Add historical data from personal archives and Alihealth (thanks to @aboutmydreams / #3)
- Add key to global view
- Convert disparate data sources in to singular SQLite database
- Add cities (needs an improved visualization system)
- More visualization options:
- amcharts (icky licensing but looks cool)
- d3.js and d3-china-map (good option but possibly overkill)
- jvectormap and jvectormap china (good option but no cities)
- kartograph (good option)
- Add other sources, eg.
- National Health Commission daily reports
- Tencent live monitoring page
- English Wikipedia (although it generally lags DXY on updates, the summary table is good for international data citations)
- Fuuuuuuu's map which seems to have many international cases missed by BNO
- Potential for different source SVGs, eg. naturalearthdata
- Potential for global spread of a novel coronavirus from China (2020-01-27)
- Ranks Burma, Cambodia, India, Indonesia, Philippines as relatively vulnerable.
- Real-time nowcast and forecast on the extent of the Wuhan CoV outbreak, domestic and international spread (2020-01-27)
- Hong Kong University professors estimate 43,590 infections as of 2020-01-25. (ie. ~20x 'confirmed cases')
- 2019-nCoV may be about to become a global epidemic
- Self-sustaining human-to-human spread is already present in all major Chinese cities
- Pattern of early human-to-human transmission of Wuhan 2019-nCoV (2020-01-24)
- We found the basic reproduction number, R0, to be around 2.2 (90% high density interval 1.4–3.8), indicating the potential for sustained human-to-human transmission. Transmission characteristics appear to be of similar magnitude to severe acute respiratory syndrome-related coronavirus (SARS-CoV) and the 1918 pandemic influenza.
- Novel coronavirus 2019-nCoV: early estimation of epidemiological parameters and epidemic predictions (2020-01-23)
- Only 5% of cases are likely reported in official figures of confirmed cases.
This section is written for the curious / non-technical user.
The general approach to problems such as these is as follows:
- Gather the data
- Modify and store it
- Do something with it.
The area of programming surrounding gathering data from websites that were not explicitly designed for it is called web scraping.
In general, web scraping consists of making an HTTP (web) request to the website in question, parsing (or interpreting) the response, and extracting the data of interest. Thereafter some modification may be required.
We translate some Chinese and English information (toponyms or geographic region names) in to a known format by matching against a static database file for countries and a similar file for regions in or near China.
We then store the data in various formats, mostly CSV and JSON, which are timestamped in a most to least significance format, inspired by the ISO 8601 standard, to aid in sorting.
Finally, we further interpret and process the data in two stages.
First, we transform some reference SVG maps gathered from Wikimedia Commons by applying the data we have captured.
Finally we animate multiple such resulting images in to two formats, animated GIF and the greatly superior and far more modern webm container format with VP9 encoding. This is done using the open source tools imagemagick and ffmpeg.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent
