Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Export/import fails with IIIF harvested item #2983

Open
mephillips-durham opened this issue Oct 17, 2023 · 0 comments · May be fixed by #3197
Open

Export/import fails with IIIF harvested item #2983

mephillips-durham opened this issue Oct 17, 2023 · 0 comments · May be fixed by #3197
Assignees
@dnoneill

Comments

@mephillips-durham
Copy link
Contributor

It seems that the JSON created when exporting a complete exhibition fails to link the metadata object in solr_document_sidecars with the IIIF manifest in the "resources" array further down the file.

Steps to reproduce:

  1. Create a new exhibition, giving it a name and a slug.
  2. Go to "Items" and "Add items" and choose the "IIIF URL" option.
  3. Paste in a manifest URL to import a single item.
  4. In the General section, use the "Export data" facility to produce the JSON file.

Example of export.
matthew-small-export.json

  1. Then go to another Spotlight instance and create a new blank exhibition.
  2. In the General section of the admin interface, use "Import data" to import the JSON file.

The indexing fails with an error "undefined method `downcase' for nil:NilClass" with details:

{
  "backtrace": "/home/vagrant/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/spotlight-9c0ae85b1417/app/models/spotlight/resources/iiif_manifest.rb:111:in `map'\n/home/vagrant/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/spotlight-9c0ae85b1417/app/models/spotlight/resources/iiif_manifest.rb:111:in `missing_keys'\n/home/vagrant/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/spotlight-9c0ae85b1417/app/models/spotlight/resources/iiif_manifest.rb:104:in `create_sidecars_for'\n/home/vagrant/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/spotlight-9c0ae85b1417/app/models/spotlight/resources/iiif_manifest.rb:94:in `manifest_metadata'\n/home/vagrant/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/spotlight-9c0ae85b1417/app/models/spotlight/resources/iiif_manifest.rb:86:in `add_metadata'",
  "resource_id": null
}

In the JSON file there is a solr_document_sidecars array containing a single object. It contains the metadata for the object:

  "solr_document_sidecars": [
    {
      "public": true,
      "data": {
        "readonly_published_tesim": [
          "1800 - 1900; late Qing dynasty"
        ],
        "readonly_dimensions_tesim": [
          "diameter 57 mm; circumference 185 mm"
        ],
        "readonly_description_tesim": [
          "Ivory ball of 'Devil's Work', contains nine layers including the surface layer. The inner balls are decorated with openwork star patterns, while the outer ball is decorated with three four-claw dragons amidst swirling clouds. Made for export."
        ]
      },
      "created_at": "2023-10-17T15:57:31.664Z",
      "updated_at": "2023-10-17T15:57:31.811Z",
      "document_id": "156ca8ee331bda9ff474bac5a529e9e9",
      "document_type": "SolrDocument",
      "resource_id": 298,
      "resource_type": null,
      "index_status": {
      },
      "tag_list": [

      ]
    }
  ],

Further down the file there is "resources":

  "resources": [
    {
      "url": "https://iiif.durham.ac.uk/manifests/trifle/museum/object/19/60/19606/manifest",
      "data": {
      },
      "indexed_at": null,
      "created_at": "2023-10-17T15:57:28.631Z",
      "updated_at": "2023-10-17T15:57:28.631Z",
      "metadata": null,
      "index_status": null,
      "type": "Spotlight::Resources::IiifHarvester"
    }
  ],

Note that the solr_document_sidecars contains two values that could be used as unique IDs, document_id and resource_id:

      "document_id": "156ca8ee331bda9ff474bac5a529e9e9",
      "document_type": "SolrDocument",
      "resource_id": 298,
      "resource_type": null,

The "resources" object contains the unique IIIF manifest, but no other unique values that could link it to the main metadata object.

The document in Solr itself looks like this, and has all the information you would expect in a single object:

{
      "spotlight_resource_id_ssim":["gid://dur-spotlight/Spotlight::Resources::IiifHarvester/298"],
      "spotlight_resource_type_ssim":["spotlight/resources/iiif_harvesters"],
      "id":"156ca8ee331bda9ff474bac5a529e9e9",
      "exhibit_matthew-small_public_bsi":true,
      "exhibit_matthew-small_readonly_published_tesim":["1800 - 1900; late Qing dynasty"],
      "exhibit_matthew-small_readonly_dimensions_tesim":["diameter 57 mm; circumference 185 mm"],
      "exhibit_matthew-small_readonly_description_tesim":["Ivory ball of 'Devil's Work', contains nine layers including the surface layer. The inner balls are decorated with openwork star patterns, while the outer ball is decorated with three four-claw dragons amidst swirling clouds. Made for export."],
      "spotlight_exhibit_slug_matthew-small_bsi":true,
      "spotlight_exhibit_slugs_ssim":["matthew-small"],
      "full_title_tesim":["Ball (Ivory carving, decorated with carve-in-relief and openwork design.) 牙雕穿花龙纹鬼工球; 鬼工球; Guigong qiu - DUROM.2001.97"],
      "full_image_url_ssm":["https://iiif.durham.ac.uk/iiif/trifle/8d/1b/ae/8d1bae24c15004a0c63945f53e64ed82/a3b5372b7d7e3613653abdf906ef9f51.jp2/full/full/0/default.jpg"],
      "iiif_manifest_url_ssi":"https://iiif.durham.ac.uk/manifests/trifle/museum/object/19/60/19606/manifest",
      "content_metadata_image_iiif_info_ssm":["https://iiif.durham.ac.uk/iiif/trifle/8d/1b/ae/8d1bae24c15004a0c63945f53e64ed82/a3b5372b7d7e3613653abdf906ef9f51.jp2/info.json","https://iiif.durham.ac.uk/iiif/trifle/8d/1b/ae/8d1bae24c15004a0c63945f53e64ed82/baa4e79d9883a9035c304243da10d86a.jp2/info.json","https://iiif.durham.ac.uk/iiif/trifle/8d/1b/ae/8d1bae24c15004a0c63945f53e64ed82/76e9457a3ce6656950e0c4f13c374c42.jp2/info.json"],
      "readonly_published_tesim":["1800 - 1900; late Qing dynasty"],
      "readonly_dimensions_tesim":["diameter 57 mm; circumference 185 mm"],
      "readonly_description_tesim":["Ivory ball of 'Devil's Work', contains nine layers including the surface layer. The inner balls are decorated with openwork star patterns, while the outer ball is decorated with three four-claw dragons amidst swirling clouds. Made for export."],
      "_version_":1780018841720455168,
      "timestamp":"2023-10-17T15:57:32.067Z",
      "score":2.9375608
    }

It looks very much like the Export process does not export sufficient information to allow the item to be recreated properly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dnoneill dnoneill

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix for import using :label instead of 'label' for some custom fields projectblacklight/spotlight
2 participants
@mephillips-durham @dnoneill