hakaiinstitute / hakai-ckan-records-checks

Hakai CKAN Records Checks

This repository contains a set of tests that can be executed on each CKAN record available on the Hakai Institute CKAN catalogue. These tests are designed specifically for the Hakai Institute CKAN catalogue.

The result is summarized via the github page: https://hakaiinstitute.github.io/hakai-ckan-records-checks/

Purpose

The purpose of this repository is to ensure the quality and integrity of the CKAN records in the Hakai Institute CKAN catalogue. By running these tests, we can identify any issues or inconsistencies in the data and take appropriate actions to resolve them.

Getting Started

To get started with running the tests, follow these steps:

1. Clone this repository to your local machine.
2. Install the necessary dependencies (ideally within an environment). We recommend to use poetry:

   pip install -e .

3. Execute the tests:

   hakai_ckan_records_checks -c https://catalogue.hakai.org --cache
   

Test Categories

The tests in this repository are organized into the following categories:

1. Data completeness: These tests check if all the required fields in the CKAN records are populated.
2. Data consistency: These tests verify the consistency of data across different CKAN records.
3. Data accuracy: These tests validate the accuracy of specific data fields in the CKAN records.
4. Data format: These tests ensure that the data in the CKAN records adhere to the specified format.

Contributing

Contributions to this repository are welcome. If you encounter any issues or have suggestions for improvements, please open an issue or submit a pull request.

License

This repository is licensed under the MIT License.

hakai-ckan-records-checks's Issues

addon to the ckan check

To make the tool useful, we would need to implement the following features:

• Add workflow_dispatch to rebuild the page on demand manually
• Add excel output download
• Add dataset row id and sort datasets by submission date
• Split warning and errors columns
• Implement couple tests to reflect checks from the https://github.com/cioos-siooc/metadata-entry-form/blob/main/firebase-functions/functions/dataset-name.md
• review if contact has a ror id, if *@hakai.org, make a warning
• review version element to match general recommended standards

HAKAI CKAN records review tool

The present repo is a tool for summarizing the different records present within Hakai CKAN and review each of those records and detect issues related to each individual records.

This tool will be useful to test the extend of some issues and help fix future requirements in historical records.

A first draft of the report is made available here:
https://hakaiinstitute.github.io/hakai-ckan-records-checks/

@Br-Johnson @timvdstap Let me know your thoughts or what would need to be review.

Tests applied to each records are done here:

hakai-ckan-records-checks/hakai_ckan_records_checks/hakai.py

Line 19 in 398dd23

def test_record_requirements(record) -> pd.DataFrame:

Hakai CKAN Checker tests?

The Hakai CKAN checker is now running live. The only thing stopping it from being useful is proper checks.

The tests implemented are presented here (see every lines starting with _test( :
https://github.com/HakaiInstitute/hakai-ckan-records-checks/blob/main/hakai_ckan_records_checks/hakai.py

The main question what should we look for?