jamaps / cltd

Canadian Longitudinal Census Tract Database

Hello there! and welcome to my GitHub page.

I am a Cartographer and Urban Data Scientist based in Toronto.

Currently I work as Lead, Maps & Data Visualization at the School of Cities.

My GitHub activity is a hodgepodge of geographic data analysis/science/cleaning projects (mostly using Python or PostGIS) or developing smaller scale web-based interactive maps and data viz projects (recently with Svelte + D3 + Leaflet/MapLibre).

Check out my Website / Portfolio for a visual library of my maps and some of my other side projects.

Social links

• LinkedIn
• Mastodon

etc.

• Wikimedia Uploads
• OpenStreetMap Contributions
• Google Scholar
• ResearchGate

cltd's Issues

Get and insert data for 1950s, 1960s, 1976, and 2021

Get and insert data for the 1950s, 1960s, 1976, and 2021.

Make sure the CTUIDs are in a common standard - I think this is the case already since these have been retrospectively digitized - but not 100% sure.

Crosswalk validator script / add flags

After outputting a crosswalk table - need to do the following

• If the sum of weights for a source CT sum to 1
• Remove "slivers" (depending on some standard - maybe less than 0.001) - then re-allocate slivers weights to the remaining
• Check again if sum of weights of source CT sum to 1
• Add the appropriate flag (1,2,3,4)

Looks like checking the sum, re-allocating, and adding flags should each be their own function (maybe in a separate file).

Then have a script that calls these for an input crosswalk table, does the above process, and saves as a new file

Crosswalks for 1991 and 1996

• update bf/ea points
• translate points
• then do population-point-based weighting

Interpolation functions

In Python, have simple a module for different areal interpolation functions (Python with SQL)

• population & dasy area
• dasy area
  (should dasy area be integrated? those clips can be really big).

e.g. functions with input parameters

def area(source_ct, target_ct, land_to_clip, output_file_name)
def poparea(source_lower_with_ct_code, target_ct, land_to_clip, output_file_name)

Then call those functions in scripts for each year

interpolation/
---interpolation.py
---2016.py
---2011.py
---etc.

Workflow should be slowly going back in time, starting with 2016

Re-organize repository for updating project

Setup of development space for updating with 2021 and historical (pre-1971) data

Do

• move existing to an "original" or "older" folder
• setup new development space
• update README.md with notice

1951 crosswalk

The first with pure areal apportionment

Use the 1971 built up for clipping

Crosswalk 2006 to 2011 and 2021

1971 boundaries update

The 1971 boundaries are low resolution - if we have the time we can update via the EAs and triangulating with 1966 and 1976

1956 and 1961 Crosswalks

Replicating the areal process from 1951 using 1971 built up for initial clipping

Clean up database

The original database is over 8gb zipped. There's probably a bunch of excess stuff in there that could be removed and generally cleaned up.

Plus also look into updating postgres and PostGIS functions.

Crosswalk 2016 to 2021

Crosswalk 2011 to 2016 (then double check) and 2021

Base this off of 2016

Compare with original work at the end