eurocris / openaire-cris-validator

A validator tool to assess the compliance of an OAI-PMH endpoint with the OpenAIRE Guidelines for CRIS Managers 1.1 or higher.

OpenAIRE CRIS validator

A tool to assess whether an OAI-PMH endpoint can provide research information complying with the OpenAIRE Guidelines for CRIS Managers versions 1.1 and 1.2. It covers these checks. Please note that releases before 2.0.0 only support version 1.1 of the Guidelines.

This is a command-line Java tool that is organized as a JUnit test suite. You can also run it in your IDE.

Please read below how to build it, run it, explore its internals and give feedback. This is Open Source software, available under the terms of the Apache 2.0 License.

← checking if the software builds and runs on the example files.

Usage

Build

Please make sure you have checked out the guidelines-cris-managers project in a parallel directory. Then do:

mvn clean package

We compile for Java 17 by default, but you can switch to 11 or 1.8 in the POM file.

Run

From command line

java -jar target/openaire-cris-validator-*-jar-with-dependencies.jar {endpoint-url}

From Eclipse

Set up a JUnit launcher for the CRISValidator class. Pass the OAI-PMH endpoint URL as the value of the system property endpoint.to.validate. Add the parallel guidelines-cris-managers project to the classpath of the launcher (in order to access the XML Schemas).

Checking the examples from OpenAIRE Guidelines for CRIS Managers

Use file:samples/ as your endpoint-url.

Diagnostics

The validator copies the responses to the requests it makes into files in the data/ subdirectory.

Internals

CRISValidator is the main validator class. It is the JUnit4 test suite. As it reads the metadata records from the CRIS:

• it does simple checks on the fly (using CheckingIterable); and
• it builds an internal representation: a HashMap of trees that consist of CERIFNodes. The last test, check990_CheckReferentialIntegrityAndFunctionalDependency, works on this internal representation.

OAIPMHEndpoint is an independent implementation of an OAI-PMH 2.0 client in Java. While it uses JAXB to map the OAI-PMH 2.0 markup to Java objects, any metadata payload is opaque to it. For requests that list objects (i.e., ListIdentifiers, ListRecords or ListSets) an Iterable is returned that uses the protocol's resumption token mechanism to fetch successive chunks of objects. This is entirely transparent to the class user.

If the OAI-PMH 2.0 data provider advertises support for a compression, the endpoint client object will use it. CompressionHandlingHttpURLConnectionAdapter is a transparent compression-handling wrapper around an HttpURLConnection.

Feedback

I'll be grateful for your feedback.
Please submit a github issue or email me to [email protected].

License

Copyright 2018–2023 Jan Dvořák  https://orcid.org/0000-0001-8985-152X and other contributors

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

openaire-cris-validator's Issues

Validator fails on ORCID validation

When running the latest version of the validator, I get the error below. For some reason it isn't expecting the <ORCID> here. However, when I look in the specs, I clearly see that <ORCID> is allowed. Also, when looking in the examples I see an <ORCID> in there as well.

See below for the input record that fails in the validator. For obvious reasons I have obviscated the actual data here. However, I have already checked that url given in <ORCID> matches the regular expression listed in the specs.

What I find interesting is that the error message says that it only expects one of four elements (ElectronicAddress, Affiliation, Classification and Link), while the specs contain more elements and don't include the expected element Link.

error message

While validating element Person[@id="PRSxxxxxxx"]: org.xml.sax.SAXParseException; cvc-complex-type.2.4.a: Invalid content was found starting with element 'ORCID'. One of '{"https://www.openaire.eu/cerif-profile/1.1/":ElectronicAddress, "https://www.openaire.eu/cerif-profile/1.1/":Affiliation, "https://www.openaire.eu/cerif-profile/1.1/":Classification, "https://www.openaire.eu/cerif-profile/1.1/":Link}' is expected.

record on which the validator fails

<record>
   <header>
      <identifier>oai:narcis.nl:Persons/PRSxxxxxxx</identifier>
      <datestamp>2018-10-19T08:31:44Z</datestamp>
      <setSpec>openaire_cris_persons</setSpec>
   </header>
   <metadata>
      <Person xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="PRSxxxxxxx">
         <PersonName>
            <FamilyNames>My-Name</FamilyNames>
            <FirstNames>x.y</FirstNames>
         </PersonName>
         <ISNI>some isni</ISNI>
         <ORCID>https://orcid.org/0000-000some-orcid-tail</ORCID>
         <ElectronicAddress>url:http://www.some.website/</ElectronicAddress>
         <Affiliation>
            <OrgUnit id="some org Id" />
         </Affiliation>
      </Person>
   </metadata>
   <about>
      <provenance xmlns="http://www.openarchives.org/OAI/2.0/provenance" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/provenance http://www.openarchives.org/OAI/2.0/provenance.xsd">
         <originDescription harvestDate="2018-10-19T10:31:29Z" altered="true">
            <baseURL>http://tnarcis01.dans.knaw.nl/nodserver-0.0.1/nodbrug</baseURL>
            <identifier>PRSxxxxxxx</identifier>
            <datestamp>2009-06-12</datestamp>
            <metadataNamespace>http://www.onderzoekinformatie.nl/nod/prs</metadataNamespace>
         </originDescription>
      </provenance>
   </about>
</record>

SAXParseException on fetching records

When running the validator on our OpenAIRE endpoint in Narcis, we get a org.xml.sax.SAXParseException on the tests for check400_CheckPersons, check500_CheckOrgUnits and check600_CheckProjects. Listed below is an example input, as well as the error message that is outputed by the validator.

However, if we go back to commit 6fcb5ff (which is the one before you introduced strict mode on the validator), the XML parsing is done successfully. Every commit after, up to the current master, will fail on the aforementioned tests with the same exception.

example input:

<?xml version="1.0" encoding="UTF-8"?>
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
   <responseDate>2018-11-07T14:26:30Z</responseDate>
   <request metadataPrefix="oai_cerif_openaire" verb="ListRecords">http://oai.narcis.nl/cerif</request>
   <ListRecords>
      <record>
         <header>
            <identifier>oai:narcis.nl:OrgUnits/ORG1236019</identifier>
            <datestamp>2018-10-18T07:49:33Z</datestamp>
            <setSpec>openaire_cris_orgunits</setSpec>
         </header>
         <metadata>
            <OrgUnit xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="ORG1236019">
               <Naam xml:lang="nl">test naam</Naam>
               <Naam xml:lang="en">test name</Naam>
               <ElectronicAddress>tel:[phone]</ElectronicAddress>
               <ElectronicAddress>fax:[phone]</ElectronicAddress>
               <ElectronicAddress>mailto:[email protected]</ElectronicAddress>
               <ElectronicAddress>url:http://www.example.com</ElectronicAddress>
            </OrgUnit>
         </metadata>
         <about>
            <provenance xmlns="http://www.openarchives.org/OAI/2.0/provenance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/provenance http://www.openarchives.org/OAI/2.0/provenance.xsd">
               <originDescription harvestDate="2018-10-18T09:49:32Z" altered="true">
                  <baseURL>http://tnarcis01.dans.knaw.nl/nodserver-0.0.1/nodbrug</baseURL>
                  <identifier>ORG1236019</identifier>
                  <datestamp>2003-07-02</datestamp>
                  <metadataNamespace>http://www.onderzoekinformatie.nl/nod/org</metadataNamespace>
               </originDescription>
            </provenance>
         </about>
      </record>
   </ListRecords>
</OAI-PMH>

validator error message:

[org.xml.sax.SAXParseException; lineNumber: 5; columnNumber: 76; cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'provenance'.]

Update JAVA 8 basis to a newer JAVA LTS

Dear,

JAVA 8 with Long-Term-Support (LTS) was generally available in March 2014 and the Premier Support ended this month (March 2022). The extended support is mentioned until December 2030 with *****.

***** The Extended Support uplift fee will be waived for the period June 2019 - July 2022 for Java SE 7. The Extended Support uplift fee will be waived for the period March 2022 - December 2030 for Java SE 8. During this period, you will receive Extended Support as described in the Oracle Technical Support Level sections of the Technical Support Policies.

My intention would be to replace the JAVA 8 version with a newer LTS version (e.g. 11,17), in order to be able to

• to use newer components, like newer TLS support
• to react to security vulnerabilities and their dependencies if necessary
• improvement of performance, would be nice if it happens

The full JAVA SE support roadmap could be found at: https://www.oracle.com/java/technologies/java-se-support-roadmap.html

I'm looking forward to your feedback.

Best, Andreas

IllegalArgumentException: Test class can only have one constructor

OpenAIRE CRIS Validator Version: 2.0.0
Java Version: Temurin-17.0.9+9

There seems to be a problem when running openaire-cris-validator jar:

➜  openaire-cris-validator git:(main) ✗ java -jar target/openaire-cris-validator-*-jar-with-dependencies.jar file:samples/ 

...

JUnit version 4.13.2
.E
Time: 0.003
There was 1 failure:
1) initializationError(org.eurocris.openaire.cris.validator.CRISValidator)
java.lang.IllegalArgumentException: Test class can only have one constructor
        at org.junit.runners.model.TestClass.<init>(TestClass.java:48)
        at org.junit.runners.JUnit4.<init>(JUnit4.java:23)
        at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:10)
        at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
        at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:37)
        at org.junit.runner.Computer.getRunner(Computer.java:50)
        at org.junit.runner.Computer$1.runnerForClass(Computer.java:31)
        at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
        at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:125)
        at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:111)
        at org.junit.runners.Suite.<init>(Suite.java:81)
        at org.junit.runner.Computer$2.<init>(Computer.java:33)
        at org.junit.runner.Computer.getSuite(Computer.java:28)
        at org.junit.runner.Request.classes(Request.java:77)
        at org.junit.runner.JUnitCommandLineParseResult.createRequest(JUnitCommandLineParseResult.java:116)
        at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
        at org.junit.runner.JUnitCore.main(JUnitCore.java:36)
        at org.eurocris.openaire.cris.validator.CRISValidator.main(CRISValidator.java:167)

FAILURES!!!
Tests run: 1,  Failures: 1

Steps To Reproduce

1. mvn clean package
2. java -jar target/openaire-cris-validator-*-jar-with-dependencies.jar file:samples/

Works fine after replacing constructor public CRISValidator( final URL endpointBaseUrl ).

Adapt the OAI identifier validation rules for https://github.com/openaire/guidelines-cris-managers/pull/90

Do not require the type of the entity to be part of the OAI identifier.