Notes and comments parser
Reads XML files and saves into database.
Setup
Run npm install
Setup database:
createdb <db_name>
psql <db_name> < scripts/create_tables.sql
The following command adds indexes to the database, making it much more swift to query. You can add them right after the setup of the database, or perform this after the initial data loading (see below) for better performance during the import.
psql <db_name> < scripts/create_indexes.sql
Setup environment variables required for the project:
export OSM_COMMENTS_POSTGRES_URL='postgres://<username>@localhost/osm-comments'
export OSM_COMMENTS_TEST_POSTGRES_URL='postgres://username@localhost/osm-comments-test'
Run
In a node shell:
var notesParser = require('./notes');
notesParser({filename: '/path/to/notes-xml'});
var changesetParser = require('./changesets');
changesetParser({filename: '/path/to/changeset-xml'});
From the terminal:
node index.js <notes|changesets> --filename=/path/to/xml/file
Test
Run npm test
Initial load of changesets
When starting out with an empty database, there is an optimized way to load the initial backlog of changesets. Create an empty folder called csv in the project root and pass the option: initial=true changesetParser. After this command is run, run psql <db_name> < changesets/post_initial.sql to load the CSVs into the database. FIXME: this should be scripted.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent