discoverygarden / islandora_solution_pack_manuscript

Allows users to create and view Manuscripts. Including the upload of TEI and XSLT and CSS documents. Users will be able to view transformed manuscript TEI (via the upload XSLT) side by side with the image(s) of the manuscript (via the Open Seadragon viewer). Users will also be able to browse Manuscripts via Box / Folder hierarchies as defined by their inventory record.

Manuscript Solution Pack

Introduction

Allows users to create and view Manuscripts. Including the upload of TEI and XSLT and CSS documents. Users will be able to view transformed manuscript TEI (via the upload XSLT) side by side with the image(s) of the manuscript (via the Open Seadragon viewer). Users will also be able to browse Manuscripts via Box / Folder hierarchies as defined by their record in an associated finding aid.

The Connecticut Digital Archive has put together an informative user documentation guide that outlines the functionality of this module.

Key Features

• Ability to upload TEI
• Ability to upload XSLT (1.0) / CSS
• Ability to associate XSLT / CSS with a Finding Aid Object
• Rendering of TEI as HTML (XSLT)
• Open Sea Dragon Viewer display of Manuscript image Content
• Side by side view of Transformed TEI and Open Sea Dragon Viewer
• Navigation of manuscripts by box / folder meta-data (SOLR driven)

Requirements

This module requires the following modules/libraries:

• Islandora
• Islandora Paged Content
• jstree

This module has the following recommended (optional) modules/libraries:

• Islandora Large Image Solution Pack
• Islandora OCR
• Islandora OpenSeadragon
• Islandora SOLR Search
• Islandora Paged TEI Seadragon Viewer

Installation

Install as usual, see this for further information.

Download/clone the jsTree library to sites/all/libraries/jstree. The module has been tested with the 3.0.0 release of jsTree.

Troubleshooting/Issues

Having problems or solved a problem? Check out the Islandora google groups for a solution.

• Islandora Group
• Islandora Dev Group

FAQ

Q. What elements are necessary in finding aid EAD metadata?

A. Components (c, c01, c02, c03, etc) MUST have id attributes unique to the given XML document in order to reliably produce links and relationships. Components MUST have a level as one of:

• series
• subseries
• file

Additionally, as of writing, the only supported types of containers inside of components are boxes and folders. Folder entries MAY be associatited to boxes using the parent attribute, to target the id given to a box. Alternatively, associations will be made by iterating containers and producing a new association for each "box" encountered.

A minimal example of the structure we require:

<ead xmlns="urn:isbn:1-931666-22-9">
  <eadheader>
    <eadid>example-id</eadid>
    <filedesc>
      <titlestmt>
        <titleproper>Example Collection</titleproper>
      </titlestmt>
    </filedesc>
  </eadheader>
  <archdesc level="collection">
    <did>
      <unittitle>Example Collection</unittitle>
    </did>
    <dsc>
      <c01 id="alpha" level="series">
        <!--
          "bravo" makes use of the "parent" attribute to associate a folder
          with a box.
        -->
        <did>
          <unittitle>Alpha</unittitle>
        </did>
        <c02 id="bravo" level="file">
          <did>
            <unittitle>Bravo</unittitle>
            <container id="container-one" type="box">1</container>
            <container parent="container-one" type="folder">1</container>
          </did>
        </c02>
        <!--
          "charlie" relates containers by associating boxes and folders as they
          occur in document order.
        -->
        <c02 id="charlie" level="file">
          <did>
            <unittitle>Charlie</unittitle>
            <container type="boxes">2-3</container>
            <container type="box">4</container>
            <container type="folder">1</container>
            <container type="box">5</container>
            <container type="folders">1-7</container>
          </did>
        </c02>
      </c01>
    </dsc>
  </archdesc>
</ead>

In bravo, we have one logical container:

• folder 1 from box 1

In charlie, we have three logical containers:

• boxes 2 to 3
• folder 1 from box 4
• folders 1 to 7 from box 5

Do note that the code tries not to make any assumptions about the numbering of boxes or folders. Folders could either be numbered sequentially across boxes (in which case specifying a range of folders could make sense when specifying a range of boxes) or specific to a box. Additionally, pluralization of types is largely ignored.

Maintainers/Sponsors

Current maintainers:

• discoverygarden

Development

If you would like to contribute to this module, please check out our helpful Documentation for Developers info, as well as our Developers section on the Islandora.ca site.

License

GPLv3

islandora_solution_pack_manuscript's Issues

during ingest, makes OCR regardless of whether or not islandora_ocr is installed

The config page /admin/islandora/solution_pack_config/manuscript setting for "OCR datastreams (OCR, HOCR). Requires Islandora OCR module" does not care whether or not the islandora_ocr module is actually installed... if that checkbox is checked, the OCR derivative will be created during ingest.
I would think that the code here should really be checking more than the value of just that "OCR datastreams" checkbox" with "if (module_exists('islandora_ocr')) { ... do the OCR

This is pretty much the same as the islandora_solution_pack_book bug I submitted here: https://jira.duraspace.org/browse/ISLANDORA-2036

Enhancement: Expose headers in semantic tags

The archival description is properly a definition list:

islandora_solution_pack_manuscript/transforms/ead_to_html.xslt

Lines 79 to 81 in c0fee29

	<dl>
	<xsl:copy-of select="$contents"/>
	</dl>

But is handled primarily with p tags:

islandora_solution_pack_manuscript/transforms/ead_to_html.xslt

Lines 89 to 91 in c0fee29

	<p>
	<xsl:apply-templates select="."/>
	</p>

And with dt and dd tags at the container level.

We can semantically tag the top level archival description as well, including exposing headings for sections like bibliography and controlled access.

Translation of EAD content

The EAD to HTML transform supports passing strings as parameters for rendering content not present in the EAD itself.

islandora_solution_pack_manuscript/theme/theme.inc

Line 97 in c0fee29

"container_string" => t('Containers'),

This content is passed through the t() function, but that translates it to the current language setting. It would be more appropriate to translate the content to the language in which the EAD is written (available in /eadheader/profdesc/langusage/language).

Some fool proposed extending this base usage with new strings, despite being aware of the core problem, so this issue is his admission of guilt.

The id used in islandora-manuscript-manuscript.tpl.php causes bookreader to not be able to use fullscreen toggle

I have traced this down to the islandora_internet_archive_bookreader/js/islandora_book_reader.js code where it is setting the elements to 100% height, but the template used for manuscripts uses a different id. The bookreader expects the wrapping id to be "book-viewer". See https://github.com/Islandora/islandora_internet_archive_bookreader/blob/7.x/js/islandora_book_reader.js#L505 (referenced "$('div#book-viewer')" this way in several places).

To easily address this, I changed my template to simply use the id of book-viewer even though this is being used for a manuscript because I feel that the javascript code should not have to distinguish between multiple paged-content types.

I could make a pull request with this change in it if needed -- please let me know.

Enhancement: render fourth level containers

Container rendering currently stops at the third level:

islandora_solution_pack_manuscript/transforms/ead_to_html.xslt

Line 48 in c0fee29

<xsl:template match="ead:c | ead:c01 | ead:c02 | ead:c03">

Additional levels could be helpful, such as a four-level:

• Subgroup
• Series
• Subseries
• File

Truncate display of long titles in Finding Aid

Long container titles can exceed the width of the fieldset legend, wrapping outside of the fieldset itself.

The preferred truncation is probably in the center of the text, which is why this may not be a simple theming issue. E.g.:
"SERIES II: J. Adams, S. Adams, J. Bartlett, C. Braxton, C. Carroll, S. Chase, A. Clark, G. Clymer, et. al. to George III, A Declaration. 4 July 1776"
might become
"SERIES II: J. Adams, S. Adams, J. Bartlett, C. Braxton, ..., G. Clymer, et. al. to George III, A Declaration. 4 July 1776"

Enhancement: allow for alternate EAD linking to digitized objects

Links to digital objects from the EAD currently depend on the configuration of search keys:
https://github.com/discoverygarden/islandora_solution_pack_manuscript/blob/7.x/README.md#q-what-elements-are-necessary-in-finding-aid-ead-metadata

With the child objects' metadata referencing the box and folder.

Another possible implementation could be to use the xlink:href from DAO entries for linking within the Finding Aid display.

For example, the href might point to a link resolver, a DOI, or just a URI or unique ID. An additional setting could be added for an optional prefix to apply to the xlink, e.g. "https://doi.org/" or "/islandora/object/".

Container list block should not display if no structure was found

In cases where a finding aid presents only file level components (or more generally, does not match the allowed levels in the treebuilder), the container list block should not render as an empty entity.

In this case, the block should detect the empty treebuilder and skip rendering entirely.

Error in Islandora 7.x-1.10

I'm a volunteer testing the 7.x-1.10 modules of Islandora. We use the Manuscript Solution Pack extensively here and have done so successfully for a long time on 7.x-1.8.

Unfortunately, in 7.x-1.10 I'm get this "debug" message about the islandora_openseadragon_viewer.

User deprecated function: The parameter 'fedora_object' in theme('islandora_openseadragon_viewer') has been deprecated. As of 7.x-1.10, please update your code before the next release. in template_preprocess_islandora_openseadragon_viewer() (line 19 of /var/www/drupal/sites/all/modules/islandora_openseadragon/theme/theme.inc).

User deprecated function: The parameter 'uri' in theme('islandora_openseadragon_viewer') has been deprecated. As of 7.x-1.10, please update your code before the next release. in template_preprocess_islandora_openseadragon_viewer() (line 25 of /var/www/drupal/sites/all/modules/islandora_openseadragon/theme/theme.inc).

Also, The page navigation arrows do not move between pages anymore. I'm wondering if this issue is related to the "debug" error.

Are these two issues going to be addressed in the code of the Manuscript Solution Pack any time soon?

These issues are blocking us from updating our Islandora modules at this time and we do want to do so eventually.

Thanks,
Peter MacDonald

Readme file - possible documentation source

Under the Configuration heading, suggest linking to (or pulling content from) this slideshare from CT Digital Archive: http://www.slideshare.net/CTDigitalArchive/how-to-use-the-manuscript-content-model

Install Results in Missing Objects

Possibly a noob/user error:

Ran from .../all/modules

git clone https://github.com/discoverygarden/islandora_solution_pack_manuscript.git
drush --yes en islandora_manuscript

The following extensions will be enabled: islandora_manuscript
Do you really want to continue? (y/n): y
WD islandora: Failed to ingest object: islandora:findingAidCModelcode: [error]
401message: Unauthorized
WD php: RepositoryException: Unauthorized in [error]
RepositoryConnection->parseFedoraExceptions() (line 229 of
/var/www/drupal/sites/all/libraries/tuque/RepositoryConnection.php).
Cannot modify header information - headers already sent by (output started [warning]
at /usr/share/php/drush/includes/output.inc:37) bootstrap.inc:1232
RepositoryException: Unauthorized in RepositoryConnection->parseFedoraExceptions() (line 229 of /var/www/drupal/sites/all/libraries/tuque/RepositoryConnection.php).
Drush command terminated abnormally due to an unrecoverable error. [error]

Within the GUI .../admin/islandora/solution_pack_config/solution_packs

Object status: Some objects are missing and must be installed. See objects list for details.

• Finding Aid Content Model islandora:findingAidCModel Missing
• Manuscript Content Model islandora:manuscriptCModel Missing
• Manuscript Page Content Model islandora:manuscriptPageCModel Missing
• Transform Content Model islandora:transformCModel Missing
• Manuscript Collection islandora:manuscriptCollection Missing
• Transform Collection islandora:transformCollection Missing
• Default Transform islandora:defaultTransform Missing

The container list block shouldn't hardcode the object path

The jstreebuilder hardcodes the reference to the object's URL when building the tree:

islandora_solution_pack_manuscript/includes/jstreebuilder.inc

Line 224 in c0fee29

$to_return['a_attr']['href'] = url("islandora/object/{$this->object->id}", array('fragment' => $to_return['id']));

But this block will be rendered on the page displaying that object, and the page could be arbitrarily aliased.

If aliased, these links will redirect to the /islandora/object/path unnecessarily, rather than creating a self-referential anchor, as intended.

Enhancement: locally cache EAD XSLT results for performance

Large finding aids can initiate many inline PHP calls and can be a performance hit. An optional local cache of the XSLT results per Drupal site could improve performance.

It is tempting to make an HTML Fedora derivative of the EAD datastream and serve that, but this misses the possibility of serving the same object from multiple Islandora frontends (and thus multiple base urls), and misses the changes which might come from checking at compile time for the existence of related objects in the links.

Render the processing information note

An early decision chose to exclude the processinfo tag from display, but this display may be desired.

islandora_solution_pack_manuscript/transforms/ead_to_html.xslt

Line 27 in c0fee29

<xsl:template match="ead:processinfo"/>

Odd installation/de-installation behavior

If I try to install the manuscript solution pack from scratch I get an unauthorized 401 error. If I disable the module, uninstall it, and then enable it again, I get the error again. If I disable the module but don't uninstall it and then enable it, I don't get the error. But, the solution pack's required objects aren't installed. If I go into the Web interface (admin/islandora/solution_pack_config/solution_packs) and install them, they install fine.

Here is the command line process:

vagrant@islandora:/var/www/drupal$ drush en islandora_manuscript
The following extensions will be enabled: islandora_manuscript
Do you really want to continue? (y/n): y
WD islandora: Failed to ingest object: islandora:findingAidCModelcode: 401message: Unauthorized                                                                   [error]
WD php: RepositoryException: Unauthorized in RepositoryConnection->parseFedoraExceptions() (line 229 of                                                           [error]
/var/www/drupal/sites/all/libraries/tuque/RepositoryConnection.php).
Cannot modify header information - headers already sent by (output started at /usr/share/php/drush/includes/output.inc:37) bootstrap.inc:1232                     [warning]
RepositoryException: Unauthorized in RepositoryConnection->parseFedoraExceptions() (line 229 of /var/www/drupal/sites/all/libraries/tuque/RepositoryConnection.php).
Drush command terminated abnormally due to an unrecoverable error.                                                                                                [error]
vagrant@islandora:/var/www/drupal$ drush dis islandora_manuscript
The following extensions will be disabled: islandora_manuscript
Do you really want to continue? (y/n): y
islandora_manuscript was disabled successfully.                                                                                                                   [ok]
vagrant@islandora:/var/www/drupal$ drush pm-uninstall islandora_manuscript
The following modules will be uninstalled: islandora_manuscript
Do you really want to continue? (y/n): y
islandora_manuscript was successfully uninstalled.                                                                                                                [ok]
vagrant@islandora:/var/www/drupal$ drush en islandora_manuscript
The following extensions will be enabled: islandora_manuscript
Do you really want to continue? (y/n): y
WD islandora: Failed to ingest object: islandora:findingAidCModelcode: 401message: Unauthorized                                                                   [error]
WD php: RepositoryException: Unauthorized in RepositoryConnection->parseFedoraExceptions() (line 229 of                                                           [error]
/var/www/drupal/sites/all/libraries/tuque/RepositoryConnection.php).
Cannot modify header information - headers already sent by (output started at /usr/share/php/drush/includes/output.inc:37) bootstrap.inc:1232                     [warning]
RepositoryException: Unauthorized in RepositoryConnection->parseFedoraExceptions() (line 229 of /var/www/drupal/sites/all/libraries/tuque/RepositoryConnection.php).
Drush command terminated abnormally due to an unrecoverable error.                                                                                                [error]
vagrant@islandora:/var/www/drupal$ drush dis islandora_manuscript
The following extensions will be disabled: islandora_manuscript
Do you really want to continue? (y/n): y
islandora_manuscript was disabled successfully.                                                                                                                   [ok]
vagrant@islandora:/var/www/drupal$ drush en islandora_manuscript
The following extensions will be enabled: islandora_manuscript
Do you really want to continue? (y/n): y
islandora_manuscript was enabled successfully.                                                                                                                    [ok]

After the solution pack objects have been installed, if I go back in from the command line and disable the module, uninstall it, and then enable it again, I get:

islandora_manuscript was enabled successfully.                                                                                                                    [ok]
islandora_manuscript: Did not install Finding Aid Content Model. The object already exists and is up-to-date.                                                     [warning]
islandora_manuscript: Did not install Manuscript Content Model. The object already exists and is up-to-date.                                                      [warning]
islandora_manuscript: Did not install Manuscript Page Content Model. The object already exists and is up-to-date.                                                 [warning]
islandora_manuscript: Did not install Transform Content Model. The object already exists and is up-to-date.                                                       [warning]
islandora_manuscript: Did not install Manuscript Collection. The object already exists and is up-to-date.                                                         [warning]
islandora_manuscript: Did not install Transform Collection. The object already exists and is up-to-date.                                                          [warning]
islandora_manuscript: Did not install Default Transform. The object already exists and is up-to-date.                                                             [warning]

Shouldn't those be uninstalled when the module is uninstalled?

Is all this the expected behavior? Things are fine when it's been installed and the solution pack objects have been loaded through the Web UI -- the module acts the way you'd expect it, but the installation and de-installation process, and its errors, don't seem right to me. Fwiw, I'm testing it on the Islandora Vagrant machine.

My install script is:

JSTREE_VERSION=3.0.0
DRUPAL_HOME=/var/www/drupal

# Download the manuscript solution pack and its module dependencies
cd $DRUPAL_HOME/sites/all/modules
git clone https://github.com/discoverygarden/islandora_solution_pack_manuscript.git
git clone https://github.com/discoverygarden/islandora_paged_tei_seadragon.git
git clone https://github.com/discoverygarden/islandora_rest.git

# Install JSTree, a library dependency of the manuscript solution pack
wget -O /tmp/jstree-$JSTREE_VERSION.zip https://github.com/vakata/jstree/archive/$JSTREE_VERSION.zip
unzip -d /tmp /tmp/jstree-$JSTREE_VERSION.zip
mkdir $DRUPAL_HOME/sites/all/libraries/jstree
mv /tmp/jstree-$JSTREE_VERSION/dist $DRUPAL_HOME/sites/all/libraries/jstree

# Install the manuscript solution pack module
drush -y en islandora_manuscript

Tagging @kjallen on this since he said he'd seen this behavior too.

Display metadata

We've noticed that no metadata is displayed below the object (TEI and images) for manuscript. Would it be possible to add this functionality?

Thanks.

Enhancement: treat EAD lists as lists

EAD lists like the bibliography, controlled access, and generic lists can be represented in ul tags.

New Manuscript objects should have "mixed materials" as the default value.

According to how I interpret the MODS standards, the typeOfResource for manuscript objects should most-likely be "mixed materials" (see: https://www.loc.gov/standards/mods/userguide/typeofresource.html).

The default_value used in the XML Forms code for new manuscript objects is set to "still image" (see: https://github.com/discoverygarden/islandora_solution_pack_manuscript/blob/7.x/data/forms/islandora_manuscript_form_mods.xml#L260).

I will make a pull request within github for this simple change after creating this issue.

Container list block does not consider children of unittitle

The Container List block draws the text() directly from unittitle:

islandora_solution_pack_manuscript/includes/jstreebuilder.inc

Lines 87 to 88 in c0fee29

	$title = $this->xpath->evaluate('normalize-space(string(ead:did/ead:unittitle/text()))', $element);
	$date = $this->xpath->evaluate('normalize-space(string(ead:did/ead:unitdate/text()))', $element);

but, unittitle may contain all text in its children. E.g.:

<did>
  <unittitle>
    <title render="italic">My Publication Name</title>
  </unittitle>
  <unitid>12.</unitid>
</did>

A quick fix is to pull text nodes from descendant-or-self.

Refactor EAD to HTML processing in to its own include file

Currently, the rendering of EAD as HTML is contained within the theme.inc.

The introduction of caching for the XSLT transform (#69) demands that we be able to render the EAD to HTML independent of display oriented functions, like drupal_add_js().

This also better separates the functional scope, and will make merges for fixes to #70, #68, #67, and #66 cleaner.

jquery source no longer present in jstree install

Release 3.0.0 of jsTree from the README is quite ancient:

Download/clone the jsTree library to sites/all/libraries/jstree. The module has been tested with the 3.0.0 release of jsTree.

Current releases of jsTree no longer distribute jQuery in the dist folder, as required here:

islandora_solution_pack_manuscript/includes/blocks.inc

Lines 30 to 32 in c0fee29

	"$library_path/dist/libs/jquery.js" => array(
	'group' => JS_LIBRARY,
	),

Distribution was last present in 3.0.4 and dropped in 3.0.5.

If islandora_ocr module is disabled, it still attempts to process OCR for manuscript

UPDATE: Closing this issue - it pertains to the batch code, not the solution pack code.

This is with the drush script that is used with islandora_batch... (even if the drush_islandora_manuscript_batch_preprocess was called outside of islandora_batch it could probably also be a problem).

When the islandora_ocr module is disabled or uninstalled, and if the islandora_solution_pack_manuscript configured to generate_ocr (or actually configured to "do_not_generate_ocr" == FALSE), the code will prepare a batch that will actually run OCR.

This causes a fatal error during ingest:

Enhancement: supply unit id in the container list block

Since the container series is generally structured, it would be helpful to supply the unitid of the did in addition to the unittitle and unitdate in the tree representation.

IAB fullscreen does not work

The Internet Archive Bookreader full-screen reading option does not function with this the rendering of Manuscripts.

The Islandora Internet Archive Bookreader module presumes that the fullscreen button will act on a div with the id of "book-viewer":
https://github.com/Islandora/islandora_internet_archive_bookreader/blob/691ba69a22a93e199f56288295efe7ca0e22ee46/js/islandora_book_reader.js#L836-L843

But the theming of the manuscript solution pack's manuscript view names this div as "manuscript-viewer":

islandora_solution_pack_manuscript/islandora_manuscript.module

Lines 182 to 184 in c0fee29

	'viewer_wrapper_attributes' => array(
	'id' => 'manuscript-viewer',
	),