addressing missing_nodes about epic-osm

[mikelmaron]

Found the bug at https://github.com/rsoden/osm-history/blob/master/import_scripts/pbf_to_mongo.rb#L101

When PbfParser is initiatilized, it parses the first FileBlock. "next" parses the next FileBlock. So what's happening here is that the first FileBlock is skipped. This explains the missing nodes in my case.

I'll commit a fix and make a pull request tomorrow, gotta run.

[jenningsanderson]

Wow. This makes a ton of sense and I'm really glad you caught this, sorry for my absence, but I'm back now and will look for your request tomorrow.

[mikelmaron]

@jenningsanderson just made pull request

Found one other issue. Fixing PbfParser didn't resolve problems with all missing modes. When OSMMongoable constructs a Way (or Relation) geometry, it was selecting Nodes (or Ways) that matched the exact same changeset. But these "component" Nodes (or Ways) won't necessarily have been edited at the same time. Modified the logic to match the most recent changeset.

[mikelmaron]

btw, if I had time right now, would add additional tests to import_pbf_spec.rb, to test under various scenarios that the result in mongo from a test pbf file matched expectations. don't have time though now; another ticket?

[jenningsanderson]

Thanks, @mikelmaron, I just accepted the pull -- I will start another ticket for the import specs -- these will be even more important once we get a system in place to do larger imports.