andrewiom / global-pollen-project

The Global Pollen Project is a web application for the digitisation of pollen reference material, and the crowdsourced identification of unknown pollen grains.

My Projects in Ecology and Biodiversity Science

I am a long-term ecologist working at the intersection of evidence-based approaches, long-term biodiversity responses to environmental change, and functional programming.

Tools and Libraries
	Bristlecone is an F# library for model-fitting model-selection (MFMS) of ecological models to observational data. The library was developed for tree ring analysis, but can be used for other time-series modelling problems.	docs
	Ecoset is a simple and flexible typescript app to serve geo-temporal datasets using tools such as GDAL.
	Thalloo (pronounced tol-oo) is a template for geographical mapping and data exploration of systematic maps and systematic reviews. As a Jekyll template, the starter template may be used to make a github pages static site for exploring evidence maps.	Starter template
RProvider (Maintainer)	RProvider is an interop layer for F# to R for scientific computing.	docs

Web Apps
	The Global Pollen Project is a web app for paleoecology. It aids the digitisation of pollen reference material and crowdsourced identification of unidentified pollen grains.	globalpollenproject.org
	The Local Ecological Footprint Tool (LEFT) is a web-based decision support tool which can help businesses minimise the environmental impacts of their activities when they make decisions about how land is used.	left.ox.ac.uk marineleft.ox.ac.uk

global-pollen-project's Issues

Update node dependencies (resolve dependabot issues)

There are 25 open dependabot issues. To resolve, webpack needs updating to version 5, along with its plugins.

Reported problems with digitisation tools

Reported bugs from Oxford:

• Making a new collection didn't seem to be working when I clicked the create button but I ended up with 2 identical collections on my dashboard.
• Making a slide - taxonomic information was slow to come up, boxes are a bit odd -
• Making a slide - text goes white when hovered over and they're a bit in the way but sometimes disappear when you want them!
• when validated text has no spaces between Family Genus Species of accepted name
• at first couldn't get a response from clicking Record Slide. I filled in a couple of optional boxes and was able to get a couple of slides made but wasn't able to consistently get it to make slides. Fixed by correctly returning validation errors from the server
• #47
• Uploading focus image: very long time after clicking Upload image nothing seems to happen; I wasn't able to upload any images.

Others noticed:

• Custom bootstrap radio buttons (recording slide) are no longer clickable
• Saving a calibration doesn't go back to the index page. It stays on the un-updated form
• Validation returns 'Internal error' in json response to dashboard.

Slide taxon not added to master reference collection

For 2x slides currently in the dataset, the projection handler is not adding their confirmed taxon into the master reference collection. This causes the read model to fail when attempting to cache eol data.

Error:

core.api_1         | [Read Model] New event: ("Taxon-b8fbe3ae-63ad-4a32-a9ea-aaf6f6fd6653",
core.api_1         |  EstablishedConnection
core.api_1         |    (TaxonId b8fbe3ae-63ad-4a32-a9ea-aaf6f6fd6653, GbifId 2928970),
core.api_1         |  11/08/2017 14:46:40)
core.api_1         | Establishing connection with GbifId 2928970
core.api_1         | {"EventId":13,"LogLevel":"Error","Category":"Microsoft.AspNetCore.Server.Kestrel","Message":"Connection id \u00220HMFLUSAFO678\u0022, Request id \u00220HMFLUSAFO678:00000002\u0022: An unhandled exception was thrown by the application.","Exception":"System.InvalidOperationException: Read model is corrupt: Could not get read model from Redis: TaxonDetail:b8fbe3ae-63ad-4a32-a9ea-aaf6f6fd6653

Slide recorded event:

core.api_1         |  SlideRecorded
core.api_1         |    { Id =
core.api_1         |       SlideId
core.api_1         |         (CollectionId 891fa5a1-ebfc-4360-83be-69fdf965219b, "154.7.27 - 1")
core.api_1         |      OriginalFamily = "Scrophulariaceae"
core.api_1         |      OriginalGenus = "Verbascum"
core.api_1         |      OriginalSpecies = "phlomoides"
core.api_1         |      OriginalAuthor = "L."
core.api_1         |      Place = Some (Continent Europe)
core.api_1         |      Time = None
core.api_1         |      PrepMethod = Some Acetolysis
core.api_1         |      PrepDate = None
core.api_1         |      Mounting = Some SiliconeOil
core.api_1         |      Taxon =
core.api_1         |       Botanical (TaxonId b8fbe3ae-63ad-4a32-a9ea-aaf6f6fd6653, Unknown, Unknown) },

Hosting Problem

Hi Andrew,

I came across your nice article in MEE and wanted to check out the website. Unfortunately it is currently not reachable. This is probably a problem with your hosting provider rather than the code.
The reported error is:

Error 502 - Bad gateway
Ray ID: 3db82e83ac402372 • 2018-01-11 13:21:53 UTC

I'm trying to connect from Germany.
Looking forward to trying the gpp.
Best regards,
Markus

Uploading focus image: scale doesn't change when zooming

When making a focus image with a microscope calibration, the scale bar on the preview doesn't change value on zoom.

Location widget not working on home page

The location ajax request is returning with a core error arising from within the main core api.

Viewer component scaling not working as expected

The viewer component is not rendering images at the correct scale / position on load. There are also issues when changing images in a gallery; any components (e.g. sliders, scalebars) are not being destroyed leading to multiple instances.

EoL descriptions not reliably caching

There are often missing descriptions for taxa, as these are not getting saved from the EOL requests.

Disqus comments missing

Re-implement disqus comments on unknown grain pages as per live gpp version.

There was an error!: "Value cannot be null. (Parameter 'value')"

Login is not working, with different errors depending on Firefox or other browsers.

On Safari: There was an error!: "Value cannot be null. (Parameter 'value')"

On Firefox:

This page is in Quirks Mode. Page layout may be impacted. For Standards Mode use “”.
[callback]
Loading failed for the <script> with source “https://identity.globalpollenproject.org/cdn-cgi/scripts/7d0fa10a/cloudflare-static/rocket-loader.min.js”. [callback:6:1]
Content Security Policy: The page's settings blocked the loading of a resource at https://identity.globalpollenproject.org/cdn-cgi/scripts/7d0fa10a/cloudflare-static/rocket-loader.min.js ("script-src").
Content Security Policy: The page's settings blocked the loading of a resource at https://identity.globalpollenproject.org/favicon.ico ("default-src"). resource:191:19

Error when password does not contain non-alphanumeric character does not display to user

Currently a non-alphanumeric character is required in passwords. If one is not entered, the user does not see an error but it just returns to the register form.

{"EventId":14,"LogLevel":"Warning","Category":"Microsoft.AspNetCore.Identity.UserManager","Message":"User password validation failed: PasswordRequiresNonAlphanumeric;PasswordRequiresDigit;PasswordRequiresUpper.","State":{"Message":"User password validation failed: PasswordRequiresNonAlphanumeric;PasswordRequiresDigit;PasswordRequiresUpper.","errors":"PasswordRequiresNonAlphanumeric;PasswordRequiresDigit;PasswordRequiresUpper","{OriginalFormat}":"User password validation failed: {errors}."}}

Show taxon history on taxon pages

The taxonomic backbone is dynamic - taxa can be clumped, split and renamed over time.

On the taxon pages, show (a) the initial import time and source (The Plant List 1.1), and any subsequent changes that occur within the backbone.

Published reference slides - focus slider and scale bar only work on first image

When a slide is opened, the first image that loads works well - the focus can be changed and the scale bar changes on zooming. If a different image is selected the scale bar and focus image features no longer work.
If the initial slide is clicked on it will reload with these features broken.

Json deserialisation failing when reading back eventstore

When rebuilding the read model, there has been a regression whereby certain events that contain a tuple with an option type are failing to deserialise. May be related to changes in FSharpLu.Json.

example failure:

{
  "Imported": {
    "Id": {
      "TaxonId": "9f7a2a2a-8a18-4be4-8c4d-f45d88d54246"
    },
    "Group": "Angiosperm",
    "Identity": {
      "Species": [
        {
          "LatinName": "Aster"
        },
        {
          "SpecificEphitet": "subintegerrimus"
        },
        {
          "Scientific": "Trautv."
        }
      ]
    },
    "Parent": {
      "TaxonId": "47597ce4-8a0f-468c-a25e-b6523d24176b"
    },
    "Status": "Accepted",
    "Reference": {
      "Item1": "\"Reisen in Europa, Asien und Afrika 1:\", 161, 1847"
    }
  }
}

Types are at: https://github.com/AndrewIOM/gpp-cqrs/blob/master/src/Services/Core/GlobalPollenProject.Core/Taxonomy.fs

Enable legacy digitise dashboard functions in v2

The legacy dashboard should be retained in the new project structure (v2) while a seperate bolero app is developed to replace this functionality.

The digitise dashboard javascript has been converted to typescript. We should (a) update all api calls to the latest digitise API; and (b) check that digitise functionality still works as expected.

Profile: interactive timeline

Proposed feature
On a user's profile page, list their recent / all activity in terms of digitisation, grain submission, identifications, point scoring.

Neotoma connection is broken

Neotoma have issued breaking changes over their API. We need to update our connection to use their latest API format.

Botanical name tracer not submitting

Clicking the 'Trace' button does not yield any results.

Backbone sometimes ignores authorship when tracing

Example:

There are 2 possible taxa in the GPP's taxonomic backbone:

    Salix lancifolia Andersson (doubtful name)
    Salix lancifolia Döll (doubtful name)

Update taxonomic backbone to World Flora Online

The Plant List 1.1 has been superseded by the WFO. Tasks:

• Update backbone taxa to WFO latest release. Use the latest static release from http://www.worldfloraonline.org/downloadData
• Ensure all read models are updated correctly and automatically to reflect name changes since TPL1.1.
• Change links to the plant list to point to WFO taxon pages (e.g. from GPP taxon pages).

"The Plant List (TPL) was a working list of all known plant species produced by the botanical community in response to Target 1 of the 2002-2010 Global Strategy for Plant Conservation (GSPC). TPL has been static since 2013, but was used as the starting point for the Taxonomic Backbone of the World Flora Online (WFO), and updated information can be found at www.worldfloraonline.org.

WFO is being developed by a consortium of leading botanical institutions worldwide in response to the 2011-2020 GSPC’s updated Target 1: to achieve an online Flora of all known plants by 2020. WFO welcomes feedback from users for improvements to its Taxonomic Backbone which is curated by a growing community of WFO Taxonomic Expert Networks (TENs)."

api: TypeError: Failed to execute 'fetch' on 'Window': Request with GET/HEAD method cannot have body.

I ran into the following problem when I tried the API:

TypeError: Failed to execute 'fetch' on 'Window': Request with GET/HEAD method cannot have body.

When I call the API using python, the result is unexpected even though the access is normal, this is my code： ` import requests import json

base_url = "https://api.globalpollenproject.org"

def backbone():
url = base_url + "/api/v1/backbone/search/"
data = {
"latinName": "Oryza",
"rank": "Genus"
}
headers = {"content-type": "application/json-patch+json"}
res = requests.get(url=url, data=json.dumps(data), headers=headers)
print(res.status_code)
print(res.text)

if name == "main":
backbone()`

Support points and polygons in location info when digitising

Although point and polygon geo types are implemented in the core of GPP, they are not currently accessible when recording a reference slide in the digitisation interface.

Unable to login after cookie expires

There is an issue where - when a cookie expires - the IdentityServer sends the user back to the website without prompting for username / password, but the user remains not logged in.

Improved and up-to-date documentation (and licenses)

The documentation has not been revised since 2017. Review the following sections:

• Accessing data programatically (API) - link to api.globalpollenproject.org
• Citing
• Digitising reference material
• Terms of service and privacy policy
• Whats new - add release 2.0
• Making reference material - make guide
• Remove library, framework
• Basic FAQs

413 request entity too large

Error 413 occurs when uploading a static image in the digitsation dashboard

Appsetting is missing

Hey there. I run docker composer and for the web.ui I get the error 'Appsetting is missing', and it points to View.fs file. Sorry if it's a dumb question, but I'm stuck here (I am also new in docker).