jar398 / plotter Goto Github PK

• rake traits:sync_resource_metadata - works
• Need schema documentation. repository_id, name, description

• Same command set as for hierarchy deltas
• What to do if not in opendata / no resource?
• How to key the records
• Examples?
  • marine copepods - ready to start - need integrated dwca unpacking
  • GLOBI - awaiting new dumps from Eli

• TBD: hardening & documentation, for Jeremy
• Examples: 0.9 vs. 1.1, and 1.1 vs. 1.2

This is mostly done (e.g. see rake --tasks hierarchy). I'd like to see a scripted way to create a new test instance that contains (a) all Resource nodes for either production or beta, (b) a Page node for each page in the DH, with canonical name and rank, (c) (optional) Vernaculars.

This is not directly required by any outstanding stated need but it would make testing a lot easier, especially for the DH delta feature (#4). If there were a way to load Trait nodes as well (for #3), that would be even better.

Getting a variety of errors here (see jenkins logs). Some may be ephemeral, some may simply need a retry, some might require "refactoring" the code, and some may be intractable.

• Find out what the changes are (Dec 2019 to Aug 2020) with
  • (cd ../eol_website; git diff e4d4ba69 95b17579 -- doc/trait-schema.md) >schema-diff.tmp
• change object_page_id to object_page
• new properties: citation, method, remarks, sample_size,
  name_languagecode, trait_row_count, distinct_page_count ...
• object_for_predicate - do not capture?

These would be for testing. Configured from config.yml probably

traits dumping assumes Terms form a tree. This seems to not be the case.

uri,name,type,parent_uri
http://alaska.fws.gov/,Us Fws Alaska Region,value,https://eol.org/schema/terms/western_north_america
http://alaska.fws.gov/,Us Fws Alaska Region,value,http://www.wikidata.org/entity/Q578170
http://alaska.fws.gov/,Us Fws Alaska Region,value,http://www.geonames.org/5879092

• Name nodes with canonical and status properties, linked to Pages
• identity of a Name is canonical name + page id + supplier ?

Many pages that aren't in the DH show up in trait records. Without page metadata, these are not useful. Either suppress these trait records, or add page records as needed to the traits dump.

Reported by Ray Ma

Loading wikidata vernaculars was timing out ... I think this is fixed now?