anzsoildata / def-au-scm Goto Github PK

Struggle to understand UoM for Phosphorus buffering capacity (PBC; units of mg/kg/log10 μg/L) and how /if we would need to request a UoM at QUDT

https://raw.githack.com/ANZSoilData/def-au-scma/master/html/09-P/09J1.html

The method reports on two observable properties - The question concerns Phosphorus buffering capacity

In https://github.com/ANZSoilData/def-au-scma/blob/main/rdf/skosplay-scma-001.xlsx
for each line like <http://anzsoil.org/def/au/scma/9A3a>
the skos:definition column should be <https://raw.githack.com/ANZSoilData/def-au-scma/master/html/09-P/09A3a.html>

Required. Dicussion/plan for getting the defs from the online version to this controlled-vocabulary.

Most are on fine earth, some on whole

Needs checking

19C1 and 19D1 are on different fractions of whole soil

relates to w3c/sdw#35

Could we please get Nel to check the analytes that each procedure measures - they are recorded in the excel template for data mapping as a draft.

If we just type in first, then we'll get on to finding ChEBI URIs if they are available

:) thanks!

@ljgregory

Questions relating to the trial of the the observable properties as measured and reported by the scma soil chem methods handbook. Based on https://github.com/CSIRO-enviro-informatics/PUV-ont

The parameters that these methods are a measure of - some are easier to break out following NERC ontology pattern. Some are not as straight forward. Throwing some thoughts/questions up here to help get a better feel of how we may model - yet to get RDF examples up obviously.

The methods up front of book throw curlier questions, should get more repetitive/easier with the other chapters here is hoping....

Replace skos:definition text (which just copies the label) with a hyperlink to a separate web resource.

That would usually be a HTML fragment such as https://raw.githack.com/ANZSoilData/def-au-scma/master/html/9-P/9A1.html , but potentially a PDF or some other definition sheet provided by a lab.

In https://github.com/ANZSoilData/def-au-scma/blob/main/rdf/scma.ttl the concepts in chapter 9, eg http://anzsoil.org/def/au/scma/9A

A question raised was should UoM be part of this vocabulary w their own PIDs
I reviewed the UoM across the chapters.
The recommended units of measure are not straight e.g. mg/kg, kg/ha-1 . They also include what it is a quantity of e.g. mg Cl/kg , mg N/kg
They say what could be the obsv property, and something about the conditions it was observed under
eg 'report as water soluble Cl, mg Cl/kg, on an air-dried basis'
'Report as Organic Matter by LOI (%OM) on an oven dry basis.
So, what would you pull out from the book for the UoM?
To stay true to book, you would have to create for example mg Cl/kg

Or, you could use/link eg QUDT mg/kg. But, I feel this is outside of the book and how they suggest to report results

Should we submit this 2011 version to CSIRO DAP? Or see what the deal is for location of the 1992 defs?

do we want to get rid of the html and just use text in dcterms description field of the x2 concept schemes?

https://demo.vocabs.ardc.edu.au/repository/api/lda/ub/soil-chemical-methods-australasia-scma/scm/resource?uri=http://anzsoil.org/def/au/scm/scma-2011

There was the question can we and should we pull certain elements out of the procedure definitions to be their own concepts.

I reviewed the procedure defs across the procedures - the sub-headers have some similarities across the procedures - they include 'reagents', 'procedure', 'notes' and 'calculations'.

But these are not consistent across the chapters. Some examples -
'reagents/apparatus'
'special apparatus'
'procedure - saline soils', 'procedure non-saline soils'
'calculation - saline soils', 'calculation non-saline soils'
'additional reagents'
'procedure - field', 'procedure - lab'

There are some cases where in the procedure def they cross-ref to a part of another procedure's def - eg a 'procedure' or 'reagents'

Personally I can't see the value/immediate need for pulling these parts out by their sub-headers as separate concepts.

Has anyone seen the demand for this? Are people wanting to re-use these parts to make new protocols/procedures?

They could be pulled out later as separate concepts (e.g. 'reagents', 'procedure', 'notes' and 'calculations') if the need is identified.

Thoughts? @ljgregory

In https://github.com/ANZSoilData/def-au-scm/blob/main/rdf/scm.ttl the namespace has been updated to http://anzsoil.org/def/au/scm/ - /scm/ rather than /scma/ so that the definitions are not tied only to one source (SCMA is the acronym of the 2011 Green Book).

This version of the vocabulary needs to be loaded into RVA, replacing https://vocabs.ardc.edu.au/viewById/634

Then the URI resolution needs to be updated

Required for SCM and SWCM, 2011 and 1996 versions respectively. Compilation underway by chemists. Ensure that if being done for current review that they are applicable to SCM and SWCM, 2011 and 1996, and that there are short descriptions for deprecated and replaced methods.

Notations starting 2-9 need a '0' in front so they sort before the 10-19 methods

I have split up ch9 into separate files, one for each procedure - see https://github.com/ANZSoilData/def-au-scma/tree/main/html/9-P
No editing, just a copy of the relevant lines of HTML from the chapter.

These fragments could be used
(i) as the value for the skos:definition which requires each to be copied over into the TTL file
(ii) stored separately and accessed as a link from skos:definition.

You can get each of these to render OK using raw-githack.
For example this source https://raw.githubusercontent.com/ANZSoilData/def-au-scma/main/html/9-P/9D1.html
renders as https://raw.githack.com/ANZSoilData/def-au-scma/master/html/9-P/9D1.html
I am not (yet) suggesting a definite implementation pattern, just exploring the burden of some options.
These examples could be evaluated to see if there is a (semi-)consistent layout and headings.

Would this be acceptable as at least an initial implementation pattern @peterwilson @ljgregory

to be submitted

@dr-shorthair my understanding is that we don't want the skos:definitions (ie the raw.githack links) from 2011 in the 1992 version. since for example '2A1' in 1992 version may have different definition to '2A1' in 2011

Some methods are a calculation

These methods state that the underlying methods used to generate the one result must be recorded

For example - CEC (eg 15K1), ECEC (15J1) and Ratios (15M1) - are calculations. The multiple methods need to generate a single result need to be recorded

Raising as requiring consideration in model and or implementation in delivering ratio, ECEC and CEC (and some other) results
@abhritchie @ljgregory

https://vocabs.ardc.edu.au/viewById/634

zeros removed from method notation in identifiers

example for http://anzsoil.org/def/au/scma/14E2
that was included in this vocab but superseded in 2011 version by
http://anzsoil.org/def/au/scma/14E2a and http://anzsoil.org/def/au/scma/14E2b

Should we have the defs in for these? if so, the full text only appears to be in hardcopy so may have to be typed.....

Rayment, G. E & Higginson, F. R. (Francis Ross), 1943- (1992). Australian laboratory handbook of soil and water chemical methods. Inkata Press, Port Melbourne

https://trove.nla.gov.au/work/34824443
ISBN 0909605688 :Dewey Number 546.22078Libraries Australia ID 9032227

The following have front/start steps of the methods that proceed them.
They do not have 'report as' at the end of them.
They perhaps should be changed type procedure concept to collections
They are not methods (with 'report as') in either volume of the 'green book' (SWCM or SCMA)
6B4
7A6
7B
7C1
7D
9A
9B
9C
9C2
9E
9I
9J
9K
10B
13D
18F
19B

eg green book ESP can have different green book methods under used to get results, or other underlying methods, such as correction for sodium/corrected sodium value
Same goes for rations 15M1 from memory
And ECEC, CEC
@abigg92

I can see we can probably improve the metadata.
At minimum for now, perhaps we should be saying that it sources from the 1992 version also.

https://trove.nla.gov.au/work/34824443
Rayment, G. E & Higginson, F. R. (Francis Ross), 1943- (1992). Australian laboratory handbook of soil and water chemical methods. Inkata Press, Port Melbourne

ISBN 0909605688 :Dewey Number 546.22078Libraries Australia ID 9032227

required

like http://qudt.org/vocab/unit/MOL-PER-KiloGM

delete terms from SCMA-2011 not found in SWCM-1992

https://github.com/ANZSoilData
rather than the archived
https://github.com/ANZSoilData/aus-vocab
didn't get to changing apologies

vocabs that have either been deprecated from the 2011 version that were in the 1992 version

They are either deprecated and not replaced, which is currently noted in skos:note. Example -
dcterms:identifier "http://anzsoil.org/def/au/scma/1
skos:note "Retired in Soil Chemical Methods Australasia, Rayment G.E. and Lyons D.J. ISBN 9780643101364

or replaced. Example
dcterms:identifier "http://anzsoil.org/def/au/scma/14E2
dcterms:isReplacedBy http://anzsoil.org/def/au/scma/14E2a
dcterms:isReplacedByhttp://anzsoil.org/def/au/scma/14E2b
skos:note "14E2 has been superseded by 14E2a and 14E2b in Soil Chemical Methods Australasia

In both cases, should these also be deprecated?

possibly using -
owl:deprecated a owl:AnnotationProperty ;
rdfs:label "deprecated" ;
rdfs:comment "The annotation property that indicates that a given entity has been deprecated." ;
rdfs:domain rdfs:Resource ;
rdfs:isDefinedBy http://www.w3.org/2002/07/owl# ;
rdfs:range rdfs:Resource .

It does not always make sense of is not easy to specify the analyte a procedure measures

as example -
ratios
calculations (including indices)
indirect methods for example where 'Pressure of CO2' is used as an 'equivalent' for CaCo3 method 19B2 (proxies and indirect measurements are used often in biological soil measures btw)

as https://anzsoil.org.au ?

Noting from email @drshorthair 26042021 for resolution by Megan
Might be worth using a link instead of just the ISBN number.
The Publisher’s page for the book is https://www.publish.csiro.au/book/6418
And this gets a JSON description of the Google books version https://www.googleapis.com/books/v1/volumes?q=isbn:9780643102187
Or we could use the ISBN URN scheme – i.e. these two

URN:ISBN:978-0-643-10218-7
URN:ISBN:978-0-909605-68-1

See https://www.iana.org/assignments/urn-formal/isbn and https://www.myidentifiers.com.au/identify-protect-your-book/isbn/converter-tool

I’m looking here for a more formal/universal notation for ISBN.

Requires post-processing edit to http://anzsoil.org/def/au/scma/

request from @ljgregory

remove from both in IRIs and notations

reviewed structure of for example -
a skos:Collection ;
dcterms:identifier "http://anzsoil.org/def/au/scma/9"^^xsd:anyURI ;
dcterms:source "URN:ISBN:978-0-643-10218-7" ;
ui:topMemberOf http://anzsoil.org/def/au/scma ;
rdfs:label "Phosphorus" ;
skos:definition """<h2 class="h2"><a id="page_147"><a id="chapter09">9

I think that use of skos:Collection is good here. My understanding is that we will be able to relate to other collections and or classes down the track as/if required. Is this correct? For example -

I can see that this is a Collection of green book procedures, and that we may have to expand once we get green book up as is. For example, many providers use procedures outside of the book ('in house' lab methods), or often we only know that a procedure is a 'phosphorus procedure' with the exact method unknown/not given. So, down track we may need a broader Class (?) (and or Collection) of 'phosphorus procedure'

Do we want these pulled out of the procedures for the ANSIS model and or green book vocab?

eg where it says 'report on an air-dry basis'

It may be useful that if observable properties the procedures measured were in this vocabulary that they could be encoded.

Reviewed if could be easily pulled out

It seems not:

• The chapter header tells you the high level obsv prop or substance/species being measured - but not always

• Each chapter has a section before the coded procedures.
  This section sometimes talks about the property being observed at the start, and sometimes not.
  Sometimes, a description of the observable property being measured can be scattered in different spots through the section
  Sometimes, the intro talks about an observable property that is not being measured, before going into what is being measured
  In some cases the section has sub-headers that are the observable property, but there are no definitions with this

• Each individual procedure talks about the property being observed but not always explicitly or in a consistent way between procedures

In conclusion I think the obsv properties each procedure measures can not be pulled out easily.

If there is value in linking the procedures in this collection to the obsv properties they measure, then this would require building on this vocabulary through linking to external obsv property vocabularies and would require expert input/governance. It is outside of first task of representing green book as is

There are notes and references at the end of each chapter that relate to the methods within each chapter.

Thoughts on how best to link in?

Is the place to put them in the skos:definitions of each skos:Collection
eg http://anzsoil.org/def/au/scma/03 ?

If so, should we put the html of the notes and references within the fragment or put the hyperlink in? https://raw.githack.com/ANZSoilData/def-au-scma/master/html/03/03.html

needs reversing

And testing how they can be related. Can this be done in SKOSPlay! using SKOS

This has been updated -
https://github.com/ANZSoilData/def-au-scma/tree/main/rdf

ttl upload to the experimental (sandbox) http://registry.it.csiro.au/sandbox/soil/
Once #3 and #1 are resloved
Regarding #3 definitions - this may take some time to resolve. To keep things moving, I no harm trialling and then just adding defs at later stage?

Review current demos (carbon in ANSIS) and the example for http://anzsoil.org/def/au/scm/10A2 in SCM

requires ability to extend as evolving list of procedures, including, for eg -

Acid sulfate soil methods approved by Nat Acid Sulfate Soil Committee
Pseudo-green book methods
Versions of methods (what is a version vs what is a new/different method)
'method not defined/unknown' ,
lab specific methods

This will require governance and maintenance considerations/consultations

Load SCMA in RVA demo system to test the rendering of the Method descriptions with linked definitions.

there are methods included in the current machine readable from version 1 of the text. URN:ISBN:978-0-909605-68-1

Included were the methods that were not included in version 2 of the text (ie chapter 1 and its procedures), and those that were split out into separate procedures in chapter 2.
For example 05A2 (version 1 text) is replaced in version 2 by
https://raw.githack.com/ANZSoilData/def-au-scma/master/html/05-soluble-Cl/05A2a.html
and
https://raw.githack.com/ANZSoilData/def-au-scma/master/html/05-soluble-Cl/05A2b.html

As far as I know there is no eVersion of version 1, will look into

has not been added to the table used for checking by Nel/data mapping excel table
Quantity kids will be needed for the model - to do

https://demo.vocabs.ardc.edu.au/viewById/904

in 1992 - very small things

https://catalogue.nla.gov.au/Record/685915 was dct:license, changed to dcterms:isFormatOf in excel and .ttl (just direct in text editor so we've not lost our edits required for RVA upload)

skos:editorialNote "The data was converted from a HTML representation to this 'linked-data' format
question - from HTML or hard copy?

edited description and notes -

The only remaining question I have is around title of whole thing
Currently - Soil chemical methods - Australasia (SCMA)
possibilities -
Soil Chemical Methods (Rayment and Higginson 1992; Rayment and Lyons 2011)
Soil Chemical Methods (SWCM 1992; SCMA 2011) (is there acronym book 1992 is well known by?
Soil Chemical Methods Australasia (1992, 2011)
and depending on intention to expand?
Soil Chemical Methods Australia
Soil Chemical Methods Australasia

It all looks good though.
The concepts that were replaced by/replace are all working well

UoM were pulled out for providers mapping table just as text
Need QUDT identifiers
Lodge requests for those that don't exist