GithubHelp home page GithubHelp logo

datasciencecampus / google-mobility-reports-data Goto Github PK

View Code? Open in Web Editor NEW
79.0 11.0 33.0 22.41 MB

Archive of data extracted from the google community mobility reports

covid mobility google dsc-projects

google-mobility-reports-data's Introduction

google-mobility-reports-data

Update

04/09/2020 New lookup table and boundaries file have been added to the geography folder to enable analysis to be done at the Local Authority level which has recently been added to the published Google Mobility data.

07/05/2020 Revised lookup tables and boundaries file have been provided in the geography folder to reflect the changes made in the published Google Mobility data in the separation of Nottingham and Nottinghamshire.

17/04/2020: Google now make this data available in csv format format, please source the data from there and note the additional documentation. There will be small differences between the data we have extracted up until now and that published, usually that is around +/- 1% due to rounding.

Introduction

Archive of data related to the Google Community Mobility Reports published 03/04/2020 and 10/04/2020.

This repository contains data extracted from Google Community Mobility Reports published on the 03/04/2020 and 10/04/2020, together with supporting boundaries based on the current UK administrative geography to allow mapping and linking with other sources of data.

Data

The latest data is available in Microsoft Office .xlsx format here, which includes the entire data extracted from the UK report, as well as other international reports. Other datasets include:

LATEST DATA 16/04/2020 (covers the period 29/02 - 11/04)

Original data (covers the period 16/02 - 29/03)

Headline data for all countries is available from Matt Kerlogue's repository

Background

The Data Science Campus has extracted the data from Google’s UK report, and other international reports, to make them publicly available to all. We developed this Python tool (mobius) to achieve this.

The insights are drawn from changes in visits and lengths of stay for different types of locations indexed against median levels, for the same day of the week, over the period 3rd January - 6th February 2020 (for the original data). Google uses aggregated, anonymized sets of data from users who have turned on the Location History setting (off by default). Google caution that the location accuracy and the understanding of categorized places will vary between regions and so advice against using it to compare changes between countries, or between regions with different characteristics (e.g. rural versus urban areas) - please see Google's documentation for more details.

trend-chart

Figure 1: UK trends chart from Google Community Mobility Reports published on the 03/04/2020.

Data and methodology cannot be QA’d directly by the Campus and is provided on an ‘as-is’ basis.

Python tool: Mobility Report Data Extractor (mobius)

Logo

The Data Science Campus has created a python tool (https://github.com/datasciencecampus/mobility-report-data-extractor) for extracting the data points from the graphs for Google Community Mobility Report PDFs. To use this tool, the PDF documents need to be converted to Scalable Vector Graphic (SVG) format. The Campus has converted all PDF documents to SVG documents for this tool. These SVG files are used in mobius to obtain a Comma Separated Value (CSV) file for each graph in the original report.

This repository contains the CSV files for all G20 countries for national and regional graphs.

Boundary data for the UK

The set of boundaries provided in the geopackage is draft, and has been created by ONS in order to promote information sharing and analysis of the effect of COVID19.

The boundaries have been tailored specifically to present ‘Community Mobility’ data (first published by Google on 3 April 2020) recast to administrative boundaries.

The data published by Google covers all of the UK based on the normal Government Statistical Service (GSS) assignment to 2019 administrative areas - with 3 exceptions. The Unitary Authorities of Bournemouth, Christchurch and Poole; and Telford and Wrekin are missing. In building the boundaries provided here we have assumed that Google have included data for these missing areas within Dorset and Shropshire. Although they do not reflect the current administrative geography we have made use of Ceremonial County versions of Dorset and Shropshire to ensure full geographic coverage of the UK. These boundaries do not reflect current administrative boundaries so care should be taken in how they are used and they should only be used to understand the results published by Google. Further versions of these boundaries may be made available if we are able confirm more details but they are provided here in the hope that they will be helpful to others.

Mid-year population estimates for the UK

The 2018 mid-year population estimates for the UK have been linked to the custom lookup created by ONS. A Standard Area Measurement based on those produced by ONS Geography has also been included for each area (calculated in hectares). This allows measures such as population density to be calculated. 

google-mobility-reports-data's People

Contributors

alistair-ons avatar cathyatkinson avatar tomdatasmith avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

google-mobility-reports-data's Issues

Google_Places_Lookup_Table_200417

I mapped the latest Google Mobility data and found that Google have made some minor changes to their Sub-Region names which cause the lookup to boundaries you have provided to miss a few places (e.g. Angus, Bristol and several in Northern Ireland - see list below).

I've revised your look up table (attached) with changes to GPL_NM and now seems to map fine
Google_Places_Lookup_Table_200417 Dan amend.txt

GPL_NM

Angus Council
Antrim and Newtownabbey
Ards and North Down
Armagh City, Banbridge and Craigavon
Bristol City
Derry and Strabane
Fermanagh and Omagh
Mid and East Antrim
Newry, Mourne and Down

image

Non-G20 countries

Hi there,

This is really useful, thanks for doing this. Is there any chance you could provide processed CSVs for all countries, rather than just G20 countries?

Thanks,

Nick

uk-all-trends Missingness

Hi there,

(Great work by the way). I've been working with the above dataset and have noticed that following an unpivot function in R, there is a degree of missingness. The ID oclumns aren't of concern (they are the rows for the UK which have a pattern of NAs) but I wondered about the NAs in the data:

uk-all-trends_missingness

I can see that some of the Google charts in the original pdf stated that data was missing on certain dates, so this may explain the issue. But I wondered if the missing rows had been examined and used against the source data to gain insight into the accuracy of the OCR process?

I've subsetted the data where the data column is NA and will attempt to attach.

Apologies if this work has already been done and fantastic work turning this around so quickly.

uk_all_trends

Hi there,

Will you be maintaining the uk_all_trends csv as new data is published?

Many thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.