Get taxon match categories about gbif-dataset-metrics

[niconoe]

Data extraction module now support this metric. Sample report output:

{
  "8137b32e-f762-11e1-a439-00145eb45e9a": {
    "NUMBER_OF_RECORDS": 756426,
    "BASISOFRECORDS": {
      "UNKNOWN": 6,
      "OBSERVATION": 11660,
      "PRESERVED_SPECIMEN": 744760
    },
    "TAXON_MATCHES": {
      "TAXON_MATCH_HIGHERRANK": 63987,
      "TAXON_MATCH_FUZZY": 17272,
      "TAXON_MATCH_COMPLETE": 607443,
      "TAXON_NOT_PROVIDED": 67724
    }
  }
}

(I took the freedom to make the constants uppercase for the sake of consistency).

[bartaelterman]

@niconoe I think we're missing TAXON_MATCH_NONE for records that have a taxon, but it could not be matched. At least there is a column taxon_match_none in the cartodb table created by @peterdesmet .

[bartaelterman]

test data is written to cartodb. Only taxon_match_none is still missing. All values for that column are set to 0.

[niconoe]

Well, I just had a quick look and it seems the code support it, but that most records that trigger this issue at GBIF have no scientificName at all: http://www.gbif.org/occurrence/search?ISSUE=TAXON_MATCH_NONE

As I blindly implemented Peter's algorithm above, I think we will return TAXON_NOT_PROVIDED for those (unlike GBIF services, this algorithm will put each row in a sigle category... Is that desirable?). And the data extractor (so far) doesn't return TAXON_MATCH_* counters at all if they don't have corresponding record.

Should Peter's algorithm be changed ? Would you like that the report contains TAXON_MATCH_NONE: 0 (instead of nothing) ?

Thx !

[bartaelterman]

Ok, no then everything is fine. The aggregator fills in zeros for tags that are not found, so there is no need to add that to the extractor.