tw4l / metsflask Goto Github PK

Replace custom code in METSFlask/parsemets.py with Artefactual's mets-reader-writer where possible.

mets-reader-writer API documented here: https://mets-reader-writer.readthedocs.io/en/latest/

Move from one single model containing pickled metslist to multi-table schema with appropriate foreign key contstraints (likely models: METSFile, DCMetadata, FSEntry, PREMISEvent).

Moving to a better-designed and normalized multi-table structure will enable exposing a useful REST API and, ultimately, better possibilities for visualization and reporting.

e.g. pie charts using D3.js

Currently, METSFlask returns a 500 Internal Server Error if a user (1) tries to upload a file that is not a valid Archivematica METS file, or (2) the application runs into some other error during upload.

Add a user-friendly page at minimum for case 1, so that users know the issue is with the file they are uploading.

• Modified dates? (verify that these are untouched by ingest into Archivematica)

After launch I am getting:
OperationalError

sqlalchemy.exc.OperationalError no such table METS.

I think I have followed the Readme carefully, is there something I have missed?

Flask, SQLAlchemy, sqlalchemy-migrate

• Which formats to start with?
• Define values to be extracted for each tool
• Possibility: rather than hard-coding values, loop through each found value and save both key and value to file_data (in list?)

Regexp to create PRONOM links doesn't work with results of Fido identifications

See Archivematica script parse_mets_to_db.py

Sample PREMIS output:

<mets:digiprovMD ID="digiprovMD_85">
      <mets:mdWrap MDTYPE="PREMIS:EVENT">
        <mets:xmlData>
          <premis:event xmlns:premis="info:lc/xmlns/premis-v2" xsi:schemaLocation="info:lc/xmlns/premis-v2 http://www.loc.gov/standards/premis/v2/premis-v2-2.xsd" version="2.2">
            <premis:eventIdentifier>
              <premis:eventIdentifierType>UUID</premis:eventIdentifierType>
              <premis:eventIdentifierValue>4f6c2b6d-4d25-4552-b42c-b44c3e2e3714</premis:eventIdentifierValue>
            </premis:eventIdentifier>
            <premis:eventType>ingestion</premis:eventType>
            <premis:eventDateTime>2017-01-13T04:00:41+00:00</premis:eventDateTime>
            <premis:eventDetail></premis:eventDetail>
            <premis:eventOutcomeInformation>
              <premis:eventOutcome></premis:eventOutcome>
              <premis:eventOutcomeDetail>
                <premis:eventOutcomeDetailNote></premis:eventOutcomeDetailNote>
              </premis:eventOutcomeDetail>
            </premis:eventOutcomeInformation>
            <premis:linkingAgentIdentifier>
              <premis:linkingAgentIdentifierType>Archivematica user pk</premis:linkingAgentIdentifierType>
              <premis:linkingAgentIdentifierValue>6</premis:linkingAgentIdentifierValue>
            </premis:linkingAgentIdentifier>
            <premis:linkingAgentIdentifier>
              <premis:linkingAgentIdentifierType>preservation system</premis:linkingAgentIdentifierType>
              <premis:linkingAgentIdentifierValue>Archivematica-1.5</premis:linkingAgentIdentifierValue>
            </premis:linkingAgentIdentifier>
            <premis:linkingAgentIdentifier>
              <premis:linkingAgentIdentifierType>repository code</premis:linkingAgentIdentifierType>
              <premis:linkingAgentIdentifierValue>canadian-centre-architecture</premis:linkingAgentIdentifierValue>
            </premis:linkingAgentIdentifier>
          </premis:event>
        </mets:xmlData>
      </mets:mdWrap>
    </mets:digiprovMD>

At the top of the details page, it could be useful to have an option to view and/or download the original XML file for comparison, or if the user doesn't have the original file immediately accessible.

Hi Tessa,

Getting a lot of importing errors when trying to create db or run.py - possibly due to Python 3.10 dist-packages being different than 3.6? Or at mine are in a different location since I upgraded from 3.6. I tried moving some of the flask and migrate packages over to the new location (/usr/local/lib/python3.10/dist-packages), but seem to still be having issues so I'm not certain.

Running BitCurator 4.4

Thanks!
Jarad

• Calculate number of files in mets_to_list_of_dicts
• Add int value to database
• Display on AIP page