sul-dlss / exhibits

Stanford University Libraries online exhibits showcase

SUL Spotlight Exhibit

The project's master branch provides a template Spotlight application with SUL branding and functionality.

Configuration

Exhibits need to provide the following configuration files:

• config/database.yml - Standard Rails database configuration
• config/honeybadger.yml - Honeybadger.io exception reporting configuration
• config/blacklight.yml - Blacklight solr configuration
• config/initializers/secret_token.rb - Rails secret token

Reindexing content

A Rake task is provided to (re)index content into the Solr index. It uses the configured sets in config/exhibit.yml.

$ bin/rake spotlight:index

Development

Requirements

• Redis (for running background jobs with Sidekiq)

See projectblacklight/spotlight for additional requirements.

Install dependencies, set up the databases and run migrations:

$ bundle
$ bin/rake db:setup

You can spin up the Rails server, solr_wrapper, and populate the Solr index using this command:

$ REMOTE_USER="[email protected]" bin/rails server

When prompted to create an admin user, the email should match the email provided in REMOTE_USER. This will allow you to bypass authentication.

Testing

Run RuboCop and tests:

$ bin/rake

If you'd like to run a single test, you must seed the index first:

bin/rails spotlight:seed

Tip: if you receive the error message ERROR: Core 'blacklight-core' already exists! you have an instance of Solr running elsewhere. Clean out your data with solr_wrapper clean or search for rogue instances with ps aux | grep solr.

Deploying

You must be on VPN to deploy the worker machine. Then deploy as usual using Capistrano:

$ cap stage deploy

exhibits's Issues

Add blacklight range facet gem for the year facet

SALT indexing non-standard needs

what does SALT need in spotlight to have parity with old SALT?

would profiling the MODS help w this?

Use sul-styles gem for common styling, iconography, and CSS variables

Add schema.org attributes

Blacklight now supports basic schema.org attributes for search results. We should add these attributes to our blacklight configuration as appropriate:

https://github.com/projectblacklight/blacklight/wiki/Configuration---Results-View#schemaorg-configuration

SALT: Create seed pages for Feigenbaum

• Home page
  
• Browse categories (@ggeisler to provide?)
• About pages

diagram relating exhibits/spotlight/indexing repos

Each sul-dlss git repo spotlight uses, with short description of what is in it, and links between repo showing how they relate (annotate links, too?)

• blacklight
• spotlight
• exhibits_requests
• sul_exhibits_template
• spotlight-dor-resources
• harvestdor-indexer (and its dependencies?)
• harvestdor

SALT: Create redirect rules from saltworks to Feigenbaum exhibit, PURL, and/or stacks

• determine the URL patterns we want to map
• check with ops team about how they'd like to handle this. I know for "vanity urls", we've piled on new vhosts to sulwebapp* (via the www_proxy_service), but we should confirm that this is a reasonable approach here. (SAR-3011)
• create appropriate apache RedirectMatch rules for our URL patterns

maps-of-africa branch problems

• Add contact email ([email protected]) on Settings page, Save page → exception

It shouldn't be an exception, I think, but I'm not surprised. I don't think we ever built out a way to add users before they have accounts

• Click edit for the default Browse All Exhibit Items browse category → exception

Bad data?

ActionView::Template::Error (undefined method `[]' for nil:NilClass):
    4:   <%= bootstrap_form_for [@search.exhibit, @search], style: :horizontal, right: "col-sm-5", data: {:'autocomplete-url'=> spotlight.autocomplete_exhibit_catalog_index_path(@search.exhibit, format: "json")}, html: {id: 'edit-search'} do |f| %>
    5:     <div class="row">
    6:       <%= f.text_field :title %>
    7:       <%= f.text_field :featured_item_id, value: presenter(@search.featured_item).document_heading, data: {:"featured-item-typeahead" => true, :id_field => "[data-featured-item-id]" }, id: "featured-item-title" %>
    8:       <%= f.hidden_field :featured_item_id, data: {:"featured-item-id" => true} %>
    9:       <%= f.text_area :short_description %>
   10:       <%= f.text_area :long_description %>
  blacklight (5.4.0.rc1) lib/blacklight/document_presenter.rb:22:in `document_heading'
  /home/lyberadmin/exhibits/maps-of-africa/shared/bundle/ruby/2.1.0/bundler/gems/spotlight-0c20cb4ee925/app/views/spotlight/searches/edit.html.erb:7:in `block in __home_lyberadmin_exhibits_maps_of_africa_shared_bundle_ruby_______bundler_gems_spotlight__c__cb_ee____app_views_spotlight_searches_edit_html_erb___2387820890112593466_35623880'
  actionpack (4.0.4) lib/action_view/helpers/capture_helper.rb:38:in `block in capture'
  actionpack (4.0.4) lib/action_view/helpers/capture_helper.rb:200:in `with_output_buffer'
  actionpack (4.0.4) lib/action_view/helpers/capture_helper.rb:38:in `capture'

• Create new About page, click Save in the page title slide out thingy → exception

I can't reproduce this. I didn't make any change to address it but I'm about to add an a bout page.

NoMethodError (undefined method `label_or_default' for #<ActiveRecord::AssociationRelation []>):
  activerecord (4.0.4) lib/active_record/relation/delegation.rb:121:in `method_missing'
  activerecord (4.0.4) lib/active_record/relation/delegation.rb:68:in `method_missing'
  /home/lyberadmin/exhibits/maps-of-africa/shared/bundle/ruby/2.1.0/bundler/gems/spotlight-0c20cb4ee925/app/controllers/spotlight/about_pages_controller.rb:23:in `attach_breadcrumbs'
  activesupport (4.0.4) lib/active_support/callbacks.rb:438:in `block in _run__938318687475247527__process_action__callbacks'
  activesupport (4.0.4) lib/active_support/callbacks.rb:212:in `block in _conditional_callback_around_88'

• Create Feature page, publish and save → saves fine, but main menu is not updated to show feature page title
• Same for Browse category, publishing doesn't show Browse menu item
• Slideshow results view → same problem we saw last week; modal displays but is non-functional

I know what this is. Forgot to run the -gallery generator again.

• Map view → nothing shows up on map

The actual data doesn't have the geocoding, and I don't think we should fake it in -prod. Supposedly they're adding that data to records somehow.

• If you construct a search just using a facet value, and then remove that value using the "x" in the facet sidebar, page refreshes to the Blacklight homepage

Certainly not unique to this deploy, @ggeisler should specify the behavior in a new ticket, I guess.

• We'll probably need to hide the collection record items (Maps of Africa and Oscar I. Norwich Collection) in some way (I guess we could just make them private with the Items checkbox if it isn't feasible to not include the collection objects in the exhibit)

Let's make them private. Maybe there's a way we could add a default filter to strip them out, but this doesn't seem like it's a blocker, right?

SALT indexing: extracted_entities datastream data

So it appears that some of the SALT index data comes from an 'extracted_entities' datastream in the SALT fedora objects.

Where does this information come from, and is this information in the MODS for the items being ingested into DOR? @peetucket

Add Google Analytics Tracking

SALT indexing: document type facet

can we get this or something equivalent via mods genre values? (not sure - probably want to look at all the genre values once SALT is accessioned?)

e.g. from http://purl.stanford.edu/vw282gv1740.mods

    <genre authority="aat" valueURI="http://vocab.getty.edu/aat/300028579">manuscripts for publication</genre>

from http://purl.stanford.edu/qy593db7842.mods

    <mods:genre authority="aat" valueURI="http://vocab.getty.edu/aat/300026392">interviews</mods:genre>

Add favico

Hide "embed" view on curator's metadata page.

Create an exhibit with Jarndyce objects

SALT indexing: facet value parity: year/date, author, organization

We believe the SALTworks "Year" facet is akin to the spotlight "Year" facet but we should double check the algorithm once we have all the data accessioned and indexed.

Steps:

• all the SALT data accessioned into DOR (then purl) as MODS (waiting for projectblacklight/spotlight#1181)
• index all the SALT data via spotlight
• look at facet counts and values for "Year" in saltworks and spotlight. look at result sets when particular year is selected.

we should also look at author and organization facet values and make sure things look ok

• look at facet counts and values for "Author" in saltworks and spotlight. look at result sets when particular year is selected.
• look at facet counts and values for "Organization" in saltworks and spotlight. look at result sets when particular year is selected.

Document major discrepancies for project team to review, and present to stakeholders as necessary.

testing solr configs in single repo

Add is_it_working/about_page gems

Users should not need to log in before being assigned roles

SALT indexing: physicalLocation (was: series, box, folder)

In existing Saltworks, series, subseries, box and folder are separate, independent facets:

This information is represented in DOR mods as a single string in a physicalLocation element that has no attributes (there may be multiple physicalLocation strings):

http://purl.stanford.edu/jz524jt5425.mods

  <location>
    <physicalLocation>Call Number: SC0340, Accession: 1986-052, Box: 15, Folder: 2</physicalLocation>
  </location>

http://purl.stanford.edu/dk023cx0427.mods

  <location>
    <physicalLocation>Call Number: SC0340, Accession: 1986-052, Box: 47, Folder: 20</physicalLocation>
  </location>

http://purl.stanford.edu/gw545by8468.mods

  <location>
    <physicalLocation>Call Number: SC0340, Accession 2005-101</physicalLocation>
  </location>

The Series facet value is the number after "Accession" -- Saltworks shows 2 series values for the entire collection.

The Box and Folder facet values are sometimes missing, but are also in the same string, when present.

SALT indexing: facet parity

Facet Parity issues:

Facets present but perhaps not equal:

• Author: prob ok - is this only "as creator/contributor?"
• Organization: prob ok - is this only "as creator/contributor"?
• Year / Date: (see #49)
• Document Type: (see #48 )

Facets not avail via current spotlight:

• Tagged by Donor (see #50) - MODS element: note displayLabel="Donor tags"; needs to be single valued but repeated MODS field; @peetucket will change template for un-ingested item
• possible broader usage; single string in location/physicalLocation @ndushay will talk to Laura for more examples:
  • Series (see #51)
  • Box (see #51)
  • Folder (see #51)
• Subseries: this appears to be a different animal. (see #52)

spotlight facets to suppress?

• Topic (same 3 topics for all items - actually, there are other topics, too ...)
• Resource Type (this is searchworks resource type)

Note: facets are displayed when looking at a single document in saltworks: we do not intend to continue this behavior in spotlight.

Responsive header problem at the smallest screen sizes

Merge spotlight-dor-resources gem into this application

(Related to #100)

To simplify our local exhibit deployment, we should merge the spotlight-dor-resources gem into this application. We can also simplify some of the namespacing when we move classes around.

These engine settings can moving into the config settings.yml:

https://github.com/sul-dlss/spotlight-dor-resources/blob/master/lib/spotlight/dor/resources/engine.rb#L10-L13

Logging into production Feigenbaum produces exception

For webauth user petucket:

TypeError (no implicit conversion of nil into Array):
app/models/user.rb:40:in &' app/models/user.rb:40:increate_roles_from_workgroups'
app/models/user.rb:28:in `webauth_groups='

https://github.com/sul-dlss/sul-exhibits-template/blob/master/app/models/user.rb#L40

guard against nil?

also see https://github.com/sul-dlss/sul-exhibits-template/blob/b72fb7616a462e8a0cabc03da7d73aa14501df2b/config/initializers/devise.rb#L3-L13

Update project dependencies to the latest and greatest

Add customized mailer text for add new exhibits administrator (super-admin)

@jkeck Here's suggested text for our customized version of the email sent out when adding a new super-admin. In the `someone_invited_you' text, make the 'SUL online exhibits' portion a link to https://exhibits.stanford.edu if you can.

exhibits_admin_invitation_mailer:
  invitation_instructions:
    hello: "Hello %{email}!"
    someone_invited_you: "The SUL Exhibits Administrator has invited you to join the administrative group that manages the SUL online exhibits. You can accept this invitation by clicking the link below."
    accept: "Accept invitation"
    ignore_html: "If you don't want to accept the invitation, please ignore this email. You won't be made a SUL online exhibits administrator until you access the link above."

Search results (gallery view) should not render a blank image tag when no thumbnail is present

Search results:

When viewing search results for objects with no image assets, there are broken thumbnails shown on search result views (e.g. https://exhibits.stanford.edu/feigenbaum/default-exhibit/browse/sound-recordings)

Initial implementation could be as simple as a blank or generic placeholder thumb if none is found; or could be more elaborate if we know the media type (audio/video/text) and could do some media specific generic icon

Detail Page:

Seems to be ok with current embedding widget.

Add rake task to deploy all site branches

Add http_logger and report solr request timing to graphite

add coveralls

(and the badge to the README)

Split donor tags into multiple nodes based on commas

• get list of objects that have already been accessioned
• run remediation script to split tags and update contentMetadata
• adjust MODs template
• talk to Ben to update pre-assembly YAML config and MODs template for objects not yet accessioned

Add blacklight hierarchy gem for series/box/folder

Adjust site thumbnail cropping to produce a square image

Add a site thumbnail-specific jcrop_options method to jcrop_help.rb so that site thumbnails are cropped at a square aspect ratio (they are currently cropped using the same method used for other thumbnails, like the browse categories, and we don't want to change their aspect ratio)

Enhance design for adding new or existing items to an exhibit

We want a UI that offers the curator a choice of ways to add objects to an exhibit. We have several examples of importing existing resources (from flickr, IA, etc), but have no way to surface these to the curator.

Update the text label for "All fields" in fielded search drop-down to "Everything"

This clarifies that the default is to search everything, including full text if available.

add badges to sul-exhibits readme

travis, gemnasium, coveralls

SALT indexing: folder name (SALT subseries) facet

saltworks has a Subseries facet:

Create a spotlight indexing stack using discovery-indexer

SALT indexing: tagged by donor facet

This information is a MODS note field with displayLabel of "Donor tags".

The original template to get from the SALT data in spreadsheet format to MODS put all the values in a single note field, comma separated. For indexing, it is far more desirable to have each value in a separate note field.

Pre-requisite steps:

• accessioning stopped
• template changed to make a separate note field for each value (thanks @peetucket ) for uningested items
• @blalbrit will update the template used for ingest
• @blalbrit will start accessioning the rest of the salt data
• remediate multivalued note fields (#44) - IN PROGRESS NOW

Then

• note field with displayLabel "Donor tags" will get indexed into a field suitable for faceting
• spotlight exhibit will be configured to include this field as a facet.

SALT indexing: display location parity

Show separate display fields for box, folder and series on item detail page.
These will be the same as the facetable values.

Customize Spotlight sidebar with SUL-specific links

After projectblacklight/spotlight#1327 is completed, we should update the exhibits home sidebar that was added through that work to append the same sidebar links we currently show:

<ul class="nav list-unstyled">
    <li>
      <a href="http://searchworks.stanford.edu/catalog?f%5Bcollection_type%5D%5B%5D=Digital+Collection">Digital collections in SearchWorks</a>
    </li>
    <li>
      <a href="http://library.stanford.edu/collections">Older collection exhibits</a>
    </li>
    <li>
      <a href="http://library.stanford.edu/spc">Special Collections &amp; University Archives</a>
    </li>
    <li>
      <a href="http://library.stanford.edu/#middle-tab-24">Giving to SUL</a>
    </li>
  </ul>

indexing: profile MODS to assess indexing needs

in order to do a good job parsing particular strings in MODS elements, or to make sure we know a collection's value range in a MODS element, profile the information using the sul-dlss/mods-profiling-indexer.

Pre-req for #51 and possible #52, #50 and #48

Merge exhibits_requests application into this application

During this series of sprints, one common reaction has been against the number of gems + applications that make up the exhibits stack. I suggest we merge the exhibits_requests application (functionality.. not the code) and spotlight-dor-resources gem into this repository.

I believe this is an artifact of how spotlight was initially developed, coupled with a desire to keep our spotlight instance fairly close to vanilla spotlight. There may also be an element of (unrealized) flexibility in this architecture, allowing widely different exhibits to share common code.

In addition to trying to simplify the architecture, I think we'll get a couple other benefits:

• shorter release cycles for new exhibit functionality (easier to deploy; don't need to cut releases of all components; easier to run the stack locally)
• less custom components (ripping out custom capistrano deploy; can use our choice of background job processors for exhibits)
• faster exhibit creation using multi-tenancy in Spotlight instead of multi-tenant Apache
• more easily load-balanced application
• "cleaner" urls for exhibits (by removing the "default-exhibit" slug)

Downsides are:

• less flexibility to respond to exhibit-specific needs (which, again, have yet to really come up)
• more coupling between exhibits (hopefully mitigated by load-balancing the application)
• time to migrate exhibits_requests functionality into this application (hopefully just minimal, controller-level changes needed, hopefully quickly regained by faster time to deploy)
• migration time from current architecture to a multitenant one (for one, I think we'll want to improve the import/export tooling)

This change would also let us contribute, upstream, some of our "super admin" behaviors and exhibit creation workflows.

• Add exhibits index to spotlight
• Add sitemap generation to spotlight
• Update sul-exhibits-template application to be multitenant
• Add requests dashboard to sul-exhibits-template
• deploy to exhibits-prod-b with shared data going to the shared NFS mount
• Migrate existing collections into multitenant application and reindex
• Add redirects from "default-exhibit" urls to new urls
• Point exhibits.stanford.edu at exhibits-prod-b
• Decommission exhibits-prod and rebuild as exhibits-prod-a
• Rebuild exhibits-prod-b and add it to the pool

Extract superadmin workgroup name to configuration

https://github.com/sul-dlss/sul-exhibits-template/blob/master/app/models/user.rb#L22

Add OCR indexing for Jarndyce objects

Expose MODS XML as a Blacklight document export format

capistrano prod deploy: deploy to all prod boxes with single command

Ideally, we could type

cap production deploy

and, with user prompts, be deploying to all prod boxes at once, rather than to each one individually.

SALT indexing: searching parity

Steps:

• all the SALT data accessioned into DOR (then purl) as MODS (waiting for projectblacklight/spotlight#1181)
• index all the SALT data via spotlight
• perform searches in saltworks and spotlight and determine if parity is achieved. (this could even be coded w rspec-solr)

Add reference links to exhibit invitation email

When we have a more formal policy around creating SUL exhibits and some SUL-specific exhibit documentation online, we should enhance the invitation email users get when they are invited to work on an exhibit (#144) with a few links at the end of the email so users unfamiliar with exhibits have something to reference before making the decision to accept the invitation. These might be:

• Name and email of contact person if they have questions (e.g., Tony N or equivalent)
• Link to SUL exhibits landing page
• Link to SUL exhibits policy

Exhibit setup should create the default exhibit

Current bug causes multiple exhibits to be created when initially set up.

Identify "orphaned" content on thumper drives and accession as a blog

There is some unidentified content on the thumper drives with no druids that is not in SALT and has not been accessioned into SDR. If it not too time consuming, locate it, zip it, and then accession as one dark object for preservation only.