nfdi4plants / arc-validate

Home of all the tools and libraries to create and run validation of ARCs

arc-validate

Home of all the tools and libraries to create and run validation of ARCs:

• ARCExpect (docs 📓) - easy to use and understand APIs to create and execute validation cases.
• ARCValidationPackages (docs 📓) - package for installing, updating, and executing ARC validation packages.
• arc-validate/ (docs 📓) - CLI tool that offers commands for validating ARCs and managing validation packages.

Docker container

This repository provides a docker container that has the arc-validate tool pre-installed for using it in DataHUB-CI jobs.

Use the containers tagged with main for production use.

Project aim

Validation of ARCs based on:

• ARCTokenization: Structural ontologies for file formats (for parsing/tokenizing files): INVMSO, STDMSO, ASSMSO
• OBO.NET:
  • parsing ontologies, generation of OBO graphs based on ontology term relation
  • code genearation of ontology modules with accessible terms
• ARCGraph: Graph representation of file content based on structural ontologies via OBO graph
• Graph-based completion of File content (missing cells -> empty tokens) via ARCGraph
• ARCExpect: Expecto-like API for creating validation cases
• Validation Packages: API for installing and executing additional validation packages

Project layout

flowchart TD

ControlledVocabulary("<b>ControlledVocabulary:</b><br>Data model for CVs")
ARCTokenization("<b>ARCTokenization:</b><br>Tokenization of ARCs into CVs")
OBO.NET("<b>OBO.NET:</b><br>OBO Ontology data model and parsing")
ARCGraph("<b>ARCGraph:</b><br>Graph based on structural ontologies")
ARCExpect("<b>ARCExpect:</b><br>Expecto-like API for validation")
ARCValidationPackages("<b>ARCValidationPackages:</b><br>API for additional validation packages")
arc-validate("<b>arc-validate:</b><br>validation CLI tool")

arc-validate --depends on--> ARCExpect
arc-validate --depends on--> ARCValidationPackages
ARCTokenization --depends on--> ControlledVocabulary
ARCTokenization --depends on--> OBO.NET
ARCExpect --depends on--> ARCGraph
ARCExpect --depends on--> ARCTokenization
ARCGraph --depends on--> ARCTokenization
ARCGraph --depends on--> OBO.NET

arc-validate's Issues

Refactor validate API from `arc-validate` as library function on `ARCExpect`

We need that functionality as part of a package that can be referenced in validation package scripts, therefore it cannot stay in the tool exlcusively.

Code is here:

arc-validate/src/arc-validate/APIs/ValidateAPI.fs

Lines 17 to 91 in 1cf4d0a

	let validate (verbose: bool) (args: ParseResults<ValidateArgs>)=
	let root =
	args.TryGetResult(ARC_Directory)
	|> Option.defaultValue (System.Environment.GetEnvironmentVariable("ARC_PATH")) // default to ARC_PATH if argument is not provided
	|> fun s -> if System.String.IsNullOrWhiteSpace(s) then System.Environment.CurrentDirectory else s // default to ./ if ARC_PATH is not set
	|> Path.GetFullPath
	|> fun p -> p.Replace("\\","/")
	|> fun p -> if not (p.EndsWith("/")) then p + "/" else p // ensure path ends with a slash
	let outPath =
	args.TryGetResult(Out_Directory)
	|> Option.defaultValue root
	let hasInvFile = File.Exists(Path.Combine(root, "isa.investigation.xlsx"))
	let investigationTokens =
	if hasInvFile then
	Investigation.parseMetadataSheetFromFile (Path.Combine(root, "isa.investigation.xlsx"))
	|> List.filter (fun p ->
	Param.getValueAsTerm p <> Terms.StructuralTerms.metadataSectionKey // filter these out to get only value-holding cells
	)
	else
	[]
	/// these tests MUST pass for an ARC to be considered for publishing
	let criticalTests =
	testList "Critical" [
	TestGeneration.Critical.ARC.FileSystem.generateARCFileSystemTests root
	TestGeneration.Critical.ARC.ISA.generateISATests investigationTokens
	]
	let criticalTestResults =
	criticalTests
	|> performTest
	if criticalTestResults.failed |> List.isEmpty && criticalTestResults.errored |> List.isEmpty then // if no critical tests failed or errored
	/// these tests SHOULD pass for an ARC to be considered of high quality
	let nonCriticalTests =
	testList "Non-critical" [
	TestGeneration.NonCritical.ARC.ISA.generateISATests investigationTokens
	]
	let nonCriticalTestResults =
	nonCriticalTests
	|> performTest
	let combinedTestResults =
	[criticalTestResults; nonCriticalTestResults]
	|> combineTestRunSummaries // aggregate critical and non-critical test results
	let badge =
	combinedTestResults
	|> BadgeCreation.createSuccessBadge "ARC quality"
	badge.WriteBadge(Path.Combine(outPath, "arc-quality.svg"))
	combinedTestResults
	|> writeJUnitSummary verbose (Path.Combine(outPath, "arc-validate-results.xml")) // write the combined result to a single file
	ExitCode.Success // critical tests passed, non-critical tests have been performed. Success!
	else // one or more critical tests failed or errored.
	let badge =
	criticalTestResults
	|> BadgeCreation.createCriticalFailBadge "ARC quality"
	badge.WriteBadge(Path.Combine(outPath, "arc-quality.svg"))
	criticalTestResults
	|> Expecto.writeJUnitSummary verbose (Path.Combine(outPath, "arc-validate-results.xml"))
	ExitCode.CriticalTestFailure

[Feature Request] ORCID validation should also be able to treat URLs

Atm., only ORCID numbers are allowed in validation. Expand this to also incorporate URLs to the ORCID website.

Also, try to detect (and ignore) leading and trailing whitespaces.

Rename ArcValidation -> ARCValidation

Let's be consistent with ARCtrl and ARCTokenization, especially since this project has no nuget package released so far.

Add Issue templates for this repo

Would be nice to have them. Maybe when some time is left...

Email address must be in correct format

Atm., the pipeline only looks for persons with present email addresses but since Invenio cannot parse emails in incorrect formats (e.g. "myName[at]server.domain") or text that is no email at all, we must implement a check for valid email format (i.e. "@.")

CC: @Zerskk

Pipeline fails although excessive isa metadata is provided

To add this specific case here as well.

This public ARC constantly fails the validation pipelie as no xml file ist written.
It contains excessive isa metadata and was already closer inspected by @omaus
It was found that the ORICD entries were not causative as I assumed but parsing one cell from the isa.investigation.xlsx file failed which did not cause a problem in the ARCPrototype which only had a different value there.

Maybe @omaus can specify this here as it might be causative for others as well when the validaton pipeline fails.

[BUG] Missing email address is not always correctly recognized in validation

[Feature Request] SWATE Integration

ORCID may be missing but if it exists it MUST be valid

Make ORCID great again! valid if present.
Invenio is able to parse ORCIDs only if they are valid. But Invenio also allows ORCID fields to be empty. Thus, IF an ORCID is given, it MUST be valid.

CC: @Zerskk

Reduce git tests temporarily

As there are some currently issues on gitlab CI with hooks etc., let's ignore most of the tests for now and only test if the .git folder is present

Replace CvContainer-dependent parsing with mere CvParam parsing of ARC metadata

Atm., metadata files are parsed via ArcGraphModel.IOs parsing functions. Yet, we use those that return ICvBase lists consisting of CvParams and CvParams that are already aggregated into CvContainers.
These aggregations hinder flat CvParam evaluation and overcomplicate the information extraction much more than necessary. Therefore, it may be best to use shortly implemented parsing functions that only return mere CvParams.

Add API project to consume and run validation packages

A hard requirement for v2 is the consumption and execution of validation packages from the validation package index.

Basically, it has to perform 2 functions:

• Download a package by querying the package index and keeping it up to date.
• Execute the package. packages are just F# scripts, so we should be fine by just calling dotnet fsi programmatically.

This API should then subsequently added as commands for the cli tool.

[Support] Pipeline not generating any output

This is more of a support request than an actual Issue, but I hope you can point me into the right direction.
I'm interested in incorporating the arc-validate pipeline into the arc-manager and re-use this for the Plantmicrobe project. Therefore I wanted to test your pipeline locally, but I'm facing some issues.

I pulled the latest docker image from the repo and started the docker container locally. Then I copied this data inside the container and executed the arc-validate validate command on it. But no xml file or badge is generated and there's also no error message. In an earlier version, where I had to build the image from source it worked this way and I was able to generate a badge for the same example data.

Add `ARCExpect.forall`

signature:

// performs action (e.g. ARCExpect.isValid) for each CvParam with the given term
// on class ByTerm:
static member forall (
    term: CvTerm,
    action: CvParam -> unit,
    cvp: seq<CvParam>) = ...

reminder @omaus

Move ArcGraph functionality to ARCTokenization

The functionality to use ARCs as a graph representation is very useful not only in Validation contexts. Therefore, it is adviced to move it to a more fitting library like ARCTokenization.

See also nfdi4plants/ARCTokenization#34

Move OboGraph to FsObo(Parser)

OboGraph is a nice module that allows for creating FGraphs from OboOntologies. It's too nice to not have this in FsObo(Parser). Move it from here to there.

See also CSBiology/OBO.NET#17

[BUG] Title as critical validation case is missing

Title is required as of nfdi4plants/archigator-backend#5 and not present though expected!
Implement ASAP!

Add description as critical validation case

Description is required as of nfdi4plants/archigator-backend#5

Remove double namespace `ARCExpect.ARCExpect`

[BUG] cli tool cannot find test package on index

This should be resolved by adding top level API tests for the ARCValidationPackages package, which should mirror closely what the cli tests do, but in a more observable manner

No validation result XML file is generated if the isa.*.xlsx is missing

While testing corner cases of the ARC validation pipeline following error occured. The test was done with a repository containing only a README.md (to test what happens if people create empty repos before pushing an ARC into it). This shows the output of both the arc-validate tool as well as the create-badge.py script:

+ arc-validate
Internal Error:                         
The option value was None (Parameter 'option')
"   at Microsoft.FSharp.Core.OptionModule.GetValue[T](FSharpOption`1 option) in D:\a\_work\1\s\src\FSharp.Core\option.fs:line 13
   at ArcVal[e-mail](ICvBase x)
   at Microsoft.FSharp.Collections.SeqModule.Exists[T](FSharpFunc`2 predicate, IEnumerable`1 source) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 641
   at Microsoft.FSharp.Collections.SeqModule.Exists[T](FSharpFunc`2 predicate, IEnumerable`1 source) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 641
   at ArcVal[e-mail](CvContainer cv) in /opt/arc-validate/src/ArcValidation/InformationExtraction.fs:line 60
   at Microsoft.FSharp.Collections.Internal.IEnumerator.next@246[T](FSharpFunc`2 f, IEnumerator`1 e, FSharpRef`1 started, Unit unitVar0) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 248
   at [e-mail]() in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 250
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 966
   at ArcValidation.TestGeneration.Critical.Arc.ISA.generateISATests(ArcConfig arcConfig) in /opt/arc-validate/src/ArcValidation/TestGeneration/Critical/ArcISA.fs:line 22
   at ARCValidate.main(String[] argv) in /opt/arc-validate/src/arc-validate/Program.fs:line 29"
$ /opt/arc-validate/create-badge.py
Traceback (most recent call last):
  File "/opt/arc-validate/create-badge.py", line 9, in <module>
    xml = JUnitXml.fromfile(xml_path)
  File "/usr/local/lib/python3.9/dist-packages/junitparser/junitparser.py", line 751, in fromfile
    tree = etree.parse(filepath)  # nosec
  File "/usr/lib/python3.9/xml/etree/ElementTree.py", line 1229, in parse
    tree.parse(source, parser)
  File "/usr/lib/python3.9/xml/etree/ElementTree.py", line 569, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: './arc-validate-results.xml'
$ exit 0
Uploading artifacts for successful job 00:00
Uploading artifacts...
WARNING: arc-validate-results.xml: no matching files. Ensure that the artifact path is relative to the working directory (/builds/doniparthi1/Facultative-CAM-in-Talinum) 
WARNING: arc-quality.svg: no matching files. Ensure that the artifact path is relative to the working directory (/builds/doniparthi1/Facultative-CAM-in-Talinum) 
ERROR: No files to upload

Sounds like just an uncaught expection? The tool should always generate the arc-validate-results.xml in order for the create-badge.py script to be able to create a 0/x badge.

Add package bulk install and validation

install and validate against multiple packages in bulk might be a nice addition

Distribute APIs as nuget packages

Once the transition to the new API is done and stable, we should release them as nuget packages.

Checking 16-digit ORCID identifier

@omaus, @kMutagene

the Orcid identifier must correspond to the following scheme: xxxx-xxxx-xxxx-xxxx. Otherwise, no valid record can be created in Invenio. This must be checked, otherwise Archigator will create a "broken" Invenio record.

The current example is the ARCInvenioTesting repository, where "orcid1", "orcid2" etc. are entered as identifiers in the isa file. The creation of an Invenio record fails here.

[Feature Request] Run docker container tests on a test ARC in this repo

[Feature Request] Regex function for character limits

We need a function that returns a Regex pattern for lower and upper character limits of a given string.
(This is needed in the validation of some information that PRIDE validation package needs)

Add installation of packages based on a `validation_packages.yml` file

this way we can offload yaml parsing in CI jobs to the tool

Distribute packages binary and dotnet tool alongside the container

In the same vein as #38 , we should distribute the arc-validate cli tool v2 as binary and dotnet tool in addition to the container.

[BUG] User Comment cannot be resolved

When building an ARCGraph with a token list containing at least one User Comment, the following occurs:

System.Collections.Generic.KeyNotFoundException: The given key 'User Comment' was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at ARCExpect.ARCGraph.isPartOfHeader(IParam header, Dictionary`2 ontoGraph, IParam ip) in C:\Repos\nfdi4plants\arc-validate\src\ARCExpect\ARCGraph.fs:line 353
   at [email protected](IParam ip) in C:\Repos\nfdi4plants\arc-validate\src\ARCExpect\ARCGraph.fs:line 383
   at Microsoft.FSharp.Collections.SeqModule.Exists[T](FSharpFunc`2 predicate, IEnumerable`1 source) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 671
   at [email protected](Int32 i, Tuple`2 tupledArg) in C:\Repos\nfdi4plants\arc-validate\src\ARCExpect\ARCGraph.fs:line 383
   at [e-mail](b& curr) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 150
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 113
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 1003
   at ARCExpect.ARCGraph.constructIntermediateMetadataSubgraph(Dictionary`2 ontoGraph, IEnumerable`1 ips) in C:\Repos\nfdi4plants\arc-validate\src\ARCExpect\ARCGraph.fs:line 440
   at [email protected](IEnumerable`1 ips)
   at [e-mail](b& curr) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 130
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 113
   at [e-mail](b& curr) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 130
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 113
   at [e-mail](b& curr) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 130
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 113
   at [e-mail](b& curr) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 130
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 113
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeOuter@319[T,TResult](ConcatEnumerator`2 x, Unit unitVar0) in D:\a\_work\1\s\src\FSharp.Core\seqcore.fs:line 320
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeOuter@319[T,TResult](ConcatEnumerator`2 x, Unit unitVar0) in D:\a\_work\1\s\src\FSharp.Core\seqcore.fs:line 320
   at [e-mail](b& curr) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 130
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System.Collections.IEnumerator.MoveNext() in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 113
   at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc`2 action, IEnumerable`1 source) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 631
   at <StartupCode$FSI_0010>.$FSI_0010.main@() in C:\Repos\nfdi4plants\arc-validate\playgrounds\qcPackage_prototypes\pride_prototype_v0.1.0.fsx:line 53
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
Stopped due to error

Problem probably lies in isPartOfHeader function which cannot find the User Comment term in the given structural ontology.

Add Affiliation as critical test

Every InvestigationContact must have an Affiliation in order to have a publishable ARC.
An implementation of a critical test that checks for a signed Affiliation field of every InvestigationContact is urgent.

[Feature] Add basic validation tests, covering ARC specification

While additional tests shall be provided via ARCValidationPackages, the core library shall provide basic validation tests that cover the ARC specification.

Add OBO file for error classes

We want to standardize our error messages. To do so, an OBO-based ontology would be useful. This ontology should inhabit all errors, structured into classes, that we use and link them via relations.

Wrap Github api json extraction into result to catch api key problems

Add ORCID as non-critical validation case

Every InvestigationContact should have an ORCID in order to have a higher quality ARC.
An implementation of a non-critical test that checks for a signed ORCID comment field of every InvestigationContact is indicated.

Semantic test name incorrect for git hooks folder content

Name duplication in ISA critical tests

Rename `ARCExpect.test` CE -> `ARCExpect.validationCase` or `ARCExpect.validate`

the confusion of "tests" with "unit tests" while we are actually talking about validation cases is way to prominent. this helps addressing that.

Default path for storing global ARC config

ArcCommander currently uses $appdata$/DataPLANT

See source code:

    let tryGetGlobalConfigPath () =
        let log = Logging.createLogger "IniDataTryGetGlobalConfigPathLog"
        // most of this part only remains for legacy reasons. Config file should not be downloaded and placed by the user (as before) but installed by the ArcCommander itself.
        let getFolderPath specialFolder inOwnFolder inCompanyFolder newName = 
            Environment.GetFolderPath(specialFolder, Environment.SpecialFolderOption.DoNotVerify)
            |> fun x -> 
                if inOwnFolder then 
                    Path.Combine(x, "ArcCommander", "config") 
                elif inCompanyFolder && (not newName) then
                    Path.Combine(x, "DataPLANT", "ArcCommander", "config")
                elif inCompanyFolder && newName then
                    Path.Combine(x, "DataPLANT", "ArcCommander", "ArcCommander.config")
                else 
                    Path.Combine(x, "config")

[Feature Request] Metadata sheet transform into graphs

For easy accession of metadata sheet information, the parsed CvParams should be constructed into a graph that represents ARC structure.

Make tests more modular

Currently person is tested as one module either returning success or a general error for the complete person.

In this case we loose the information, which field of person is faulty.

We should change this behaviour to test all fields individually. Such as person.validateName and person.validateEmail.

Format error messages, do not show exception text

Error output that ends up in the jsunit file is too verbose. It contains the full exception message with stacktrace. If the goal is to provide helpful insights on what is wrong in the arc, I suggest cleaning up the error messages.

Here is an example on how this gets displayed (rendered junit file on gitlab):

• Talking about an "entity" here is weird.

• There is way too much text

• Ideally, the error message should be short and simple, e.g.: Folder was not present: "./.git/hooks"

• As a future improvement: maybe replace the . signifying the arc root with the path supplied via ARC_PATH

• Format error messages

• Do not show exception text

Refactor fillTokens function from ValidationPackage POC into ARCExpect

Not sure about the naming though. Reminder @omaus , this is the section:

arc-validate/playgrounds/qcPackage_prototypes/invenio_prototype_v0.1.0.fsx

Lines 83 to 93 in bf19955

	let fillTokenList onto tokens =
	let ontoGraph = OboGraph.ontologyToFGraphByName onto
	let ipsAdded = ARCGraph.addMissingTerms onto tokens
	let partitionedIps = Seq.groupWhen (ARCGraph.isHeader ontoGraph) ipsAdded
	let partitionallyFilledIps = partitionedIps |> Seq.map (ARCGraph.addMissingTermsInGroup ontoGraph)
	let groupedIps = partitionallyFilledIps |> Seq.map ARCGraph.groupTerms
	let matchedIps = groupedIps |> Seq.map (ARCGraph.matchTerms onto)
	let subgraphs = Seq.map (ARCGraph.constructIntermediateMetadataSubgraph ontoGraph) matchedIps
	let filledSubgraphs = Seq.map (fst >> ARCGraph.addEmptyIpsToNodeData) subgraphs
	let splitSubgraphs = Seq.map ARCGraph.splitMetadataSubgraph filledSubgraphs
	Seq.map ARCGraph.metadataSubgraphToList splitSubgraphs

Create badges with AnyBadge.NET

https://github.com/kMutagene/AnyBadge.NET

This eliminates the need for python in the container.

Additionally, since we create the badge in the same environment as the tests are run, we have way more fine-grained control over the information available at badge-creation.

Add Person.Email as critical test

This will be needed for our data publication workflow, since all authors will get confirmation emails.

Provide an easy-to-use top-level API to writing custom test packages

This is needed for people being able to author validation packages in https://github.com/nfdi4plants/arc-validate-packages.

Also related: #38

ideally, a validation package would look something like this:

#r "nuget: ARCValidation"
open ARCValidation

//<create a list of validation tests created via the API>
//eg:
let validationTests = [
    ValidationTest.create(<API logic here>)
    ValidationTest.create(<API logic here>)
]

//<end the script with an API function that applies the tests to an arc root path provided via cli args>
// This function would for example take a list of tests and apply it to tzhe first cli arg, which is checked to be a path
ARCValidation.runValidationTestsWithCLIArgs validationTests 

Note that a top-level API like this can also be used for the internal default tests that will always be performed by the tool, e.g. structural metadata file validation

[bug] Collection of (current) bugs with validation pipeline

I'm currently working on a few ARCs that keep running into failing CQC pipelines due to divergent handling of ISA structure / naming / referencing between ARC commander (v0.5), ARCitect (v≤0.07, based on ARCCtrl).

Collecting here. Feel free to move to more relevant repo / discussion.

• .arc (not being added by ARCitect) "message": "Actual entity is not present: ./.arc"
• registering ISA files in investigation
  • ARCitect: Study File Name | studies/<studyName>/isa.study.xlsx
  • ARC commander: Study Assay File Name | <assayName>/isa.assay.xlsx
• name of isa.assay.xlsx and isa.study.xlsx metadata sheets
  • ARCitect: isa_study; isa_assay
  • ARC commander: Assay, Study

1. CQC currently based off latest ARC commander configuration and hence failing for all ARCitect ARCs
2. ARC commander (obviously) fails on ARCitect-created ARCs (e.g. arc export, arc a list, arc a register)

Improve critical/non-critical validation case distinction

We should reflect this difference on library level, e.g. Perform.ValidationPipeline should receive two separate validation lists etc.

validation error

the command:
bash arc-validate.sh -p /arc/cmqtl_val1_arc -v inside the docker container gives me this error message

+ arc-validate -p /arc/cmqtl_val1_arc -v
Internal Error:                         
The option value was None (Parameter 'option')
"   at Microsoft.FSharp.Core.OptionModule.GetValue[T](FSharpOption`1 option) in D:\a\_work\1\s\src\FSharp.Core\option.fs:line 13
   at ArcVal[e-mail](ICvBase x)
   at Microsoft.FSharp.Collections.SeqModule.Exists[T](FSharpFunc`2 predicate, IEnumerable`1 source) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 641
   at Microsoft.FSharp.Collections.SeqModule.Exists[T](FSharpFunc`2 predicate, IEnumerable`1 source) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 641
   at ArcVal[e-mail](CvContainer cv) in /opt/arc-validate/src/ArcValidation/InformationExtraction.fs:line 60
   at Microsoft.FSharp.Collections.Internal.IEnumerator.next@246[T](FSharpFunc`2 f, IEnumerator`1 e, FSharpRef`1 started, Unit unitVar0) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 248
   at [e-mail]() in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 250
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source) in D:\a\_work\1\s\src\FSharp.Core\seq.fs:line 966
   at ArcValidation.TestGeneration.Critical.Arc.ISA.generateISATests(ArcConfig arcConfig) in /opt/arc-validate/src/ArcValidation/TestGeneration/Critical/ArcISA.fs:line 22
   at ARCValidate.main(String[] argv) in /opt/arc-validate/src/arc-validate/Program.fs:line 29"

[BUG] arc-validation fails when adding a person to a study

arc-validation in the HUB and locally fails whenever I am adding a person to a study as shown below using the ARC Commander v0.5.0.. arc-validate-results.xml is not written in these cases but arc-valiation is smooth without adding a person to the study.

STUDY CONTACTS
Study Person Last Name	Schrader
Study Person First Name	Andrea
Study Person Mid Initials
Study Person Email	[email protected]
Study Person Phone
Study Person Fax
Study Person Address	50674 Cologne, Germany
Study Person Affiliation	Data Science and Management
Study Person Roles	contact person; data-curator; leads investigation; designs experiments; supervisor; performs experiments; analyses data
Study Person Roles Term Accession Number	;;;;;;
Study Person Roles Term Source REF	SCORO; SCORO; SCORO; SCORO; SCORO; SCORO; SCORO
Comment[]	0000-0002-3879-7057

Cleanup V2 branch

Atm., V2 branch (which will be the basis for the next big Validation remake) is filled with outdated and deprecated functionality. Time to get rid of that.