buda-base / buda-iiif-presentation

iiif presentation (collections and manifests) generation

Vagrant scripts for BUDA platform instanciation

The base platform is built using Vagrant and VirtualBox:

1. Install Vagrant and VirtualBox.
2. Download or git clone this repository.
3. cd into the unzipped directory or git clone
4. install VirtualBox guest additions with vagrant plugin install vagrant-vbguest
5. run vagrant up to summon a local instance

Or for an AWS EC2 instance:

1. install the vbguest plugin: vagrant plugin install vagrant-vbguest
2. and run the command: vagrant up or rename Vagrantfile.aws to Vagrantfile and run vagrant up --provider=aws

This will grind awhile installing all the dependencies of the BUDA platform.

Once the initial install has completed the command: vagrant ssh will connect to the instance where development, customization of the environment and so on can be performed as for any headless server.

Similarly, the jena-fuseki server will be listening on:

http://localhost:13180/fuseki

Lds-pdi application is accessible at :

http://localhost:13280/

(see https://github.com/buda-base/lds-pdi/blob/master/README.md for details about using this rest services)

The command: vagrant halt will shut the instance down. After halting (or suspending the instance) a further: vagrant up will simply boot the instance without further downloads, and vagrant destroy will completely remove the instance.

If running an AWS instance, after provisioning access the instance via ssh -p 15345 and delete Port 22 from /etc/ssh/sshd_config and sudo systemctl restart sshd. This will further secure the instance from attacks on port 22.

buda-iiif-presentation's Issues

wio: collections should use the wv: manifests

instead of the v: with a range

outline of a volume in a manifest

No iiif viewer currently handles collections of collections, which is what would be needed for our outlines to be displayed in the viewer directly for a whole work. In the meantime, I think it would be good to have a per-volume outline in each manifest, by having different ranges. This seems to be something iiif viewers can display and it would make a nice improvement.

Technically, I think it's best to have these "outlined manifests" in another "namespace" (I'm talking about the first part of the identifier, like i:, etc.), and there should be a few manual tests beforehand to make sure iiif viewers are able to display that.

wv: not working for external manifests

like http://iiifpres.bdrc.io/2.1.1/wv:bdr:WIA_BCA/manifest

fix empty collections

empty collections are useless and should return 404, example:

http://presentation.bdrc.io/2.1.1/collection/wio:bdr:W23423_0035

small: add collection viewingHint

According to https://iiif.io/api/presentation/2.1/#viewinghint , our collections should probably have viewingHint: multi-part

redirecto to external manifest when available

there should be a way to detect that:

• a volume doesn't have any image available
• a volume has an external manifest and there should be a redirection

better choosing of format according to color depth

while converting tif into png makes sense for bitonal images, it's not very good for color images. In this case jpg is much better. We currently don't have any way to tell if a tiff is color or bitonal from the manifest, but we should do that. Example of work having this problem: http://library-dev.bdrc.io/show/bdr:W1KG4941 (front page)

bug in some image lists

the manifest

http://iiifpres.bdrc.io/2.1.1/v:bdr:V1PD96684_I1PD106657/manifest

contains links to image filenames like I1PD1066530003.jpg which are wrong (they should start with I1PD106657), the volume image list in the data is correct so this looks like a bug in this code

500 error (urgent)

see http://iiifpres.bdrc.io/2.1.1/collection/wio:bdr:W22084

performance issues with per-volume outline generation

The SPARQL request to get the outline for a volume is too slow, we need to find other ways to create it. Knowing that an outline for a volume is most often asked with the outline of the other volumes, it would probably be a good idea to fetch the outline for an entire work and then select the nodes relevant to the volume programmatically.

handle auth for fairuse works

when requesting a manifest for a fair use work, the auth token should be read and if the user doesn't have access to the full book, the manifest should be adjusted accordingly

collections of root level works should also contain the manifests

for instance http://presentation.bdrc.io/2.1.1/collection/wio:bdr:W23703

improve error message for works which dimensions.json didn't reach S3 yet

500 error on external manifests with vo: prefix

see

http://iiifpres.bdrc.io/2.1.1/vo:bdr:VEAP676-1-14/manifest

Wrong url for zip rendering

in the following, 2d id should be http://iiif.bdrc.io/download/ zip /v:bdr:V22084_I0888::1-588

rendering: [
{
@id: "http://iiif.bdrc.io/download/pdf/v:bdr:V22084_I0888::1-588",
label: "Download as PDF",
format: "application/pdf"
},
{
@id: "http://iiif.bdrc.io/download/pdf/v:bdr:V22084_I0888::1-588",
label: "Download as ZIP",
format: "application/zip"
}

],

implement wv: scheme

The wv: schema expressed in this doc should be implemented (and have nice labels like "worklabel in volume volumeNumber"). Manifests using this schema should be used by collections using the wio: schema.

some manifests only have a couple of images, for no reason

http://presentation.bdrc.io/2.1.1/v:bdr:V22084_I0889/manifest has only two images while purl.bdrc.io/resource/V22084_I0889.ttl indicates the correct image list and the dimensions.json file are correct... after rebooting the presentation server everything worked fine... that's weird

500 error

http://presentation.bdrc.io/2.1.1/v:bdr:V3CN2648_I3CN2658/manifest

option not to display the first two pages if present

(to be implemented after next import, when data will be fixed)

answer to canvas URLs

Although it's not mandatory, it would make sense to answer to canvas URLs, for instance

http://presentation.bdrc.io/2.1.1/v:bdr:V23703_I1421::4-82/canvas/4

aws credentials

re: buda-base/buda-base#7 we should be able to have a way to configure the aws credentials in a file in /etc/buda/iiifpres/(it could in a properties file)

use mirador trick to lazy load collections (for performance purposes)

see

ProjectMirador/mirador#1490 (closed, probably won't be reopened)

ProjectMirador/mirador#1481 (closed, but might be reopened... doesn't solve the main problem though, which is the loading of manifests, not thumbnails)

`wi:` prefix doesn't work anymore

http://iiifpres.bdrc.io/2.1.1/collection/wi:bdr:W22084 return an error, it shouldn't

inconsistency between manifest and actual image list

manifest uses wrong first image index in http://library.bdrc.io/show/bdr:W22083 :

• actual first image is 38040001.tif / http://purl.bdrc.io/query/IIIFPres_volumeInfo?R_RES=bdr%3AV22083_I3804 / http://iiif.bdrc.io/image/v2/bdr:V22083_I3804::38040001.tif/full/full/0/default.jpg
• whereas manifest gives 37540010.tif / http://iiif.bdrc.io/image/v2/bdr:V22083_I3804::37540010.tif/full/full/0/default.jpg which is not found on S3 server : Works/fe/W22083/images/W22083-3804/37540010.tif The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey;

add status, license and provider string

the manifests and collection building processes should always have access to status, license and provider string, which are not in all models

indicate protected images with the IIIF auth API

Protected images should be indicated using the IIIF Auth API

list manifests in root work even when it has parts

example

http://presentation.bdrc.io/2.1.1/collection/wio:bdr:W0CJ001_P001_P001

should have the list of volumes, not just sub collections

500 error on wio: collections

example : http://presentation.bdrc.io/2.1.1/collection/wio:bdr:W22084

wrong number of images in manifest

Manifest http://presentation.bdrc.io/2.1.1/v:bdr:V15098_I2167::35-44/manifest has only one image instead of 10.

exception when a wrong identifier is used

example

tbrc-x pages change

In the manifest

http://iiifpres.bdrc.io/2.1.1/v:bdr:V1NLM7_I1NLM7_001/manifest

the first page should be p1, not tbrc-1, this messes with the library code

link to PDF / ZIP

There is a property (rendering) that could be used to link the PDF from the manifest, it should be added (just to manifests, not to collections)

500 error

on http://presentation.bdrc.io/2.1.1/collection/wio:bdr:W27401_0003

add labels to subcollections

In the wio case (ex), it would be useful to have actual labels in subcollections

500 error

inspect http://iiifpres.bdrc.io/2.1.1/vo:bdr:V22084_I0988/manifest

wio should answer the volume list on wio: identifiers when a work doesn't have an outline

I'm not sure what's the best way to do that but it seems very reasonable

bug in page range parsing

see

http://presentation.bdrc.io/2.1.1/v:bdr:V22084_I0889::-605/manifest

missing bo-x-ewts tags

in http://presentation.bdrc.io/2.1.1/collection/wio:bdr:W19997_0538 for instance

change formatHints into preferredFormats

see buda-base/public-digital-library#64

(for later) dnt-policy

See

https://www.eff.org/dnt-policy

the goal is to mitigate things like buda-base/public-digital-library#44

it would probably be useful in the iiif server too, I'm not sure we can implement it in the pdl though, as we'll embed some Google maps scripts

null pointer exception when getting collection bdr:WEAP676-14-20

http://iiifpres.bdrc.io/2.1.1/collection/wio:bdr:WEAP676-14-20

change canvas url

Currently canvas url depend on the identifier that was passed to generate the manifest, example:

http://presentation.bdrc.io/2.1.1/v:bdr:V23703_I1421::4-82/canvas/4

but some other identifiers (such as v:bdr:V23703_I1421::4-81 would use the exact same cavas. So only parts of the identifier should be used. This would allow annotations to be targetted at one "independent" canvas. The new URLs could be:

http://presentation.bdrc.io/2.1.1/v:bdr:V23703_I1421/canvas/4

or

http://presentation.bdrc.io/2.1.1/v:bdr:V23703_I1421/canvas/14210004.tif

(although the latter is a bit strange because it's not at all a .tif file)

choose compliance level 1 for big images

For works that have big cover images (ex: W1KG4941), it would be better to advertise level 1 in the manifest instead of level 0, so that the viewer requests a smaller image.

fix 500 error

in http://library.bdrc.io/show/bdr:WZHBCA

language preference implementation

see this doc, to be written. The idea is to perform the language selection and transliteration on the server so that the manifests display nicely in any viewer that passes the header. Note that issues should probably be open in viewers to use a custom Accept-Language header.

see:

• UniversalViewer/universalviewer#631 (comment)
• ProjectMirador/mirador#1969

recursive index in universal viewer

when opening the Universal Viewer gallery in http://library.bdrc.io/show/bdr:W22704_4185 and clicking on "index" in the content panel on the left, it shows recursively other resource ids among which itself:

these come from a "collections" array in the manifest http://presentation.bdrc.io/2.1.1/collection/wio:bdr:W22704_4185 :

vertical error image for vertical works

Many copyrighted works are recent editions with a more vertical layout. The current error image is more pecha-like. It would actually be easy to know if the previous image is vertical or horizontal and adjust the error image accordingly

ex: http://library-dev.bdrc.io/show/bdr:W1PD96682

add link to root work record

Manifests should have a link to the work record they correspond to

"image number greater than total number of images" error

http://library.bdrc.io/show/bdr:W1PD96684_0206 cannot retrieve manifest because workLocation page numbers seem inconsistent: http://presentation.bdrc.io/2.1.1/v:bdr:V1PD96684_I1PD106659::534-552/manifest

in same Work but another Volume it is fine: http://library.bdrc.io/show/bdr:W1PD96684_0608

handle client side caching

see various issues on lds-pdi