IIIF Text Image Linking Survey
*A survey of providers exposing text in their IIIF manifests conducted April 29, 2022 based on manifests included in Johannes Baiter's (@jbaiter)'s Mirador Text Overlay code
The IIIF Presentation API lists several ways to refer users to textual content, including rendering elements, seeAlso blocks, and annotation on canvases. But how are content providers actually implementing these properties to expose human-transcribed text, ALTO OCR, or other formats?
Sample manifest: https://iiif.europeana.eu/presentation/9200396/BibliographicResource_3000118436165/manifest Presentation standard: 2
Manifests link to text by using annotations referenced by otherContent elements
{
"@type":"sc:Canvas",
"@id":"https://iiif.europeana.eu/presentation/9200396/BibliographicResource_3000118436165/canvas/p7",
"label":"p. 7",
"height":2048,
"width":1256,
"attribution":"Journal historique et littéraire - 1788-09-01 - https://www.europeana.eu/item/9200396/BibliographicResource_3000118436165. National Library of Luxembourg. Public Domain Mark - http://creativecommons.org/publicdomain/mark/1.0/",
"images":[
{
"@type":"oa:Annotation",
"motivation":"sc:painting",
"resource":{
"@type":"dctypes:Image",
"@id":"https://iiif.europeana.eu/image/QNYVL2Z2FVHRGMK2UNXIP4DUUOTOVA2ND3WIPJQF6V23SO2CJ5UA/presentation_images/ee0edfa0-0220-11e6-a696-fa163e2dd531/node-3/image/BNL/Journal_historique_et_littéraire/1788/09/01/00007/full/full/0/default.jpg",
"format":"image/jpeg",
"service":{
"@context":"http://iiif.io/api/image/2/context.json",
"@id":"https://iiif.europeana.eu/image/QNYVL2Z2FVHRGMK2UNXIP4DUUOTOVA2ND3WIPJQF6V23SO2CJ5UA/presentation_images/ee0edfa0-0220-11e6-a696-fa163e2dd531/node-3/image/BNL/Journal_historique_et_littéraire/1788/09/01/00007",
"profile":"http://iiif.io/api/image/2/level1.json"
}
},
"on":"https://iiif.europeana.eu/presentation/9200396/BibliographicResource_3000118436165/canvas/p7"
}
],
"otherContent":[
"https://iiif.europeana.eu/presentation/9200396/BibliographicResource_3000118436165/annopage/7"
]
}All canvases contain an otherContent element; some links return 404 codes (presumably for canvases containing no text).
Example AnnotationList: https://iiif.europeana.eu/presentation/9200396/BibliographicResource_3000118436165/annopage/7
{
"@context":[
"http://iiif.io/api/presentation/2/context.json",
"https://www.europeana.eu/schemas/context/edm.jsonld"
],
"@id":"https://iiif.europeana.eu/presentation/9200396/BibliographicResource_3000118436165/annopage/7",
"@type":"sc:AnnotationList",
"resources":[
{
"@id":"https://data.europeana.eu/annotation/9200396/BibliographicResource_3000118436165/bd9038eb741a7e45d403a3195c084cfe",
"@type":"oa:Annotation",
"motivation":"sc:painting",
"dcType":"Page",
"resource":{
"@id":"https://www.europeana.eu/api/fulltext/9200396/BibliographicResource_3000118436165/efb162f0e4524fa766ae20e4f533af57"
},
"on":[
"https://iiif.europeana.eu/image/QNYVL2Z2FVHRGMK2UNXIP4DUUOTOVA2ND3WIPJQF6V23SO2CJ5UA/presentation_images/ee0edfa0-0220-11e6-a696-fa163e2dd531/node-3/image/BNL/Journal_historique_et_littéraire/1788/09/01/00007/full/full/0/default.jpg"
]
},
{
"@id":"https://data.europeana.eu/annotation/9200396/BibliographicResource_3000118436165/82bb059565169f02e407eb4582d3a60d",
"@type":"oa:Annotation",
"motivation":"sc:painting",
"dcType":"Word",
"resource":{
"@id":"https://www.europeana.eu/api/fulltext/9200396/BibliographicResource_3000118436165/efb162f0e4524fa766ae20e4f533af57#char=0,1"
},
"on":[
"https://iiif.europeana.eu/image/QNYVL2Z2FVHRGMK2UNXIP4DUUOTOVA2ND3WIPJQF6V23SO2CJ5UA/presentation_images/ee0edfa0-0220-11e6-a696-fa163e2dd531/node-3/image/BNL/Journal_historique_et_littéraire/1788/09/01/00007/full/full/0/default.jpg#xywh=957,301,19,32"
]
},
{
"@id":"https://data.europeana.eu/annotation/9200396/BibliographicResource_3000118436165/d12aee09cc98462464a6b1567da884a6",
"@type":"oa:Annotation",
"motivation":"sc:painting",
"dcType":"Line",
"resource":{
"@id":"https://www.europeana.eu/api/fulltext/9200396/BibliographicResource_3000118436165/efb162f0e4524fa766ae20e4f533af57#char=0,1"
},
"on":[
"https://iiif.europeana.eu/image/QNYVL2Z2FVHRGMK2UNXIP4DUUOTOVA2ND3WIPJQF6V23SO2CJ5UA/presentation_images/ee0edfa0-0220-11e6-a696-fa163e2dd531/node-3/image/BNL/Journal_historique_et_littéraire/1788/09/01/00007/full/full/0/default.jpg#xywh=957,301,19,32"
]
},
{
"@id":"https://data.europeana.eu/annotation/9200396/BibliographicResource_3000118436165/3b45229ea81c1a146d741803f5d4cd24",
"@type":"oa:Annotation",
"motivation":"sc:painting",
"dcType":"Block",
"resource":{
"@id":"https://www.europeana.eu/api/fulltext/9200396/BibliographicResource_3000118436165/efb162f0e4524fa766ae20e4f533af57#char=0,1"
},
"on":[
"https://iiif.europeana.eu/image/QNYVL2Z2FVHRGMK2UNXIP4DUUOTOVA2ND3WIPJQF6V23SO2CJ5UA/presentation_images/ee0edfa0-0220-11e6-a696-fa163e2dd531/node-3/image/BNL/Journal_historique_et_littéraire/1788/09/01/00007/full/full/0/default.jpg#xywh=951,288,32,52"
]
},
{
"@id":"https://data.europeana.eu/annotation/9200396/BibliographicResource_3000118436165/968a28a5129ddb1ad069dd36a34d9ab0",
"@type":"oa:Annotation",
"motivation":"sc:painting",
"dcType":"Word",
"resource":{
"@id":"https://www.europeana.eu/api/fulltext/9200396/BibliographicResource_3000118436165/efb162f0e4524fa766ae20e4f533af57#char=2,4"
},
"on":[
"https://iiif.europeana.eu/image/QNYVL2Z2FVHRGMK2UNXIP4DUUOTOVA2ND3WIPJQF6V23SO2CJ5UA/presentation_images/ee0edfa0-0220-11e6-a696-fa163e2dd531/node-3/image/BNL/Journal_historique_et_littéraire/1788/09/01/00007/full/full/0/default.jpg#xywh=483,305,22,22"
]
},
{
"@id":"https://data.europeana.eu/annotation/9200396/BibliographicResource_3000118436165/bfdbfc56744f554d183c3c42e18dadc9",
"@type":"oa:Annotation",
"motivation":"sc:painting",
"dcType":"Word",
"resource":{
"@id":"https://www.europeana.eu/api/fulltext/9200396/BibliographicResource_3000118436165/efb162f0e4524fa766ae20e4f533af57#char=5,14"
},
"on":[
"https://iiif.europeana.eu/image/QNYVL2Z2FVHRGMK2UNXIP4DUUOTOVA2ND3WIPJQF6V23SO2CJ5UA/presentation_images/ee0edfa0-0220-11e6-a696-fa163e2dd531/node-3/image/BNL/Journal_historique_et_littéraire/1788/09/01/00007/full/full/0/default.jpg#xywh=526,297,149,37"
]
},
...
{
"@id":"https://data.europeana.eu/annotation/9200396/BibliographicResource_3000118436165/5e4cb0df7cc71ad293d33361bca7b21b",
"@type":"oa:Annotation",
"motivation":"sc:painting",
"dcType":"Block",
"resource":{
"@id":"https://www.europeana.eu/api/fulltext/9200396/BibliographicResource_3000118436165/efb162f0e4524fa766ae20e4f533af57#char=1584,1586"
},
"on":[
"https://iiif.europeana.eu/image/QNYVL2Z2FVHRGMK2UNXIP4DUUOTOVA2ND3WIPJQF6V23SO2CJ5UA/presentation_images/ee0edfa0-0220-11e6-a696-fa163e2dd531/node-3/image/BNL/Journal_historique_et_littéraire/1788/09/01/00007/full/full/0/default.jpg#xywh=276,813,31,27"
]
}
]
}The AnnotationList for a canvas contains a resources element with references to text at all levels of text granularity. This is indicated by a dcType element with values of Word, Line, Block, and Page. All text contents are external.
{
"@context":"https://www.europeana.eu/schemas/context/edm.jsonld",
"id":"https://www.europeana.eu/api/fulltext/9200396/BibliographicResource_3000118436165/efb162f0e4524fa766ae20e4f533af57",
"type":"FullTextResource",
"edmRights":"http://creativecommons.org/publicdomain/mark/1.0/",
"language":"fr",
"value":"i\nï, Septembre 1788.\n» rencontra, furent témoins de fes tranf-» ports & entendirent fes fermens. Mais le » comte de Murray étoit devenu l'objet n d’une haine fi générale , que perfonne ?» n’entreprit ni de calmer la rage du jennC si Hamilton, ni d’en prévenir l’elfet. Lé » 23 Janvier 1570, les députés d’Elifabetli « avoient reçu à Linlithgow une audience » de Murray, pour traiter de l’échange de «s Marie avec le duc de Northumbérland. s» Murray fe croyoit au moment d'un triom-31 phe complet. Il falloit pour fortir de la ville, pafl’er par une rue, au haut de la-« quelle étoit une galerie de bois, oit Ha-» milton l’attendoit. Il l’apperçut qui ve-m noit lentement à cheval ; le pafl’age étoit » refferré & rempli de peuple , il lui tira » un coup de moufquet qui l’étendit mort. 11 Hamilton ne fut point pourfuivi ; on lui 31 laiffa tout le tems de la retraite , & il n pafla aifément en France. Marie donna » des larmes à la mort de fon perfécuteur, « & demanda pour lui, à Dieu, miféricorde « & clémence. La douleur d’Elifabeth fut si différente ; elles’écrioit qu’elle avoir perdu « le meilleur ami qu'elle eût au monde, & » le plus dévoué à fes intérêts. Bien loin » de rendre la reine d’Ecofle au peuple qui si la redemandoit, Elifabeth envoya des »i troupes pour réprimer leur zele, & ce9 » troupes marquèrent leur route par une 11 défolation générale, en détruifant, par le n fer & par le feu, cinquante châteaux S? » trois cens villages. >1 Après l’infairfie dont le bâtard Murray s’eft couvert , il n’y en a pas qui égale celle qui a rendu Elifabeth odieufe & mé‘ A 4\n11"
}All resource URIs referenced in the AnnotationList return identical contents, regardless of dcType value specified in the annotation. In order to identify the word-level or line-level portion of text referenced in each annotation, clients must parse the URI fragment of the textual resource, much as they would need to parse the fragment of a canvas garget to identify the rectangle of a canvas on which to display the text.
Not all content providers are using linking properties, and of those which are, not all are using them consistently. (Note that no effort was made to evaluate which version of the Presentation API was implemented at each provider.)
- Qatar Digital Library
- Harvard University Library
- National Library of Wales
George Washington, lettre au comte Louis-Marie de Noaillesau comte de Noailles, Princeton,
"related" : "http://gallica.bnf.fr/ark:/12148/btv1b105243418",
Target is the Gallica page viewer for this item.
Francis Bolton Elwell Scrapbook: 1917-1919.
"related": {
"@id": "https://digital.library.villanova.edu/Item/vudl:482844",
"format": "text/html"
},
Target is the Villanova Digital Library viewer with the item loaded.
"related": {
"@id": "https://wellcomelibrary.org/item/b21051951",
"format": "text/html"
},
Target is Wellcome's instance of the Universal Viewer with this item loaded
Added 2017-07-17 Joseph Farington, 1747–1821, British, Caernarvon Castle, ca. 1780, Oil on canvas, Yale Center for British Art, B1976.7.118, Paintings and Sculpture
"related":{
"@id":"http://collections.britishart.yale.edu/vufind/Record/1667702",
"label":"catalog entry at the Yale Center for British Art",
"format":"text/html"
},
Target is a web page for the catalog record, featuring a thumbnail and links to viewers.
http://iiif.harvardartmuseums.org/manifests/object/299843
"rendering":{
"label":"Full record view",
"@id":"http://harvardartmuseums.org/collections/object/299843",
"format":"text/html"
},
Target is the catalog record webpage and viewer for this item.
Travels of Cyrus At the default sequence level
"rendering" : [
{
"@id" : "https://deriv.nls.uk/dcn23/1188/7314/118873148.23.pdf",
"format" : "application/pdf",
"label" : "Download as PDF"
}
],
Target is a direct link to a PDF of the scanned images, with a text layer supporting search-within on OCR.
The Re-Conquest of Ireland, A. D. 1895.
"rendering": {
"@id": "https://digital.ucd.ie/get/ivrla:2622/content",
"format": "application/pdf",
"label": "Download as PDF"
},
Target is a direct link to a PDF of the scanned images, with no text layer.
Francis Bolton Elwell Scrapbook: 1917-1919. This is a member of the default sequence:
"rendering": [],
This is a member on each canvas:
"rendering": [
{
"@id": "https://digital.library.villanova.edu/files/vudl:482873/MASTER",
"format": "image/tiff",
"label": "Original source file - 148.38 MB"
},
{
"@id": "https://digital.library.villanova.edu/files/vudl:482873/OCR-DIRTY",
"format": "text/plain",
"label": "Raw OCR Data"
},
{
"@id": "https://digital.library.villanova.edu/files/vudl:482873/MASTER-MD",
"format": "application/xml",
"label": "Technical Metadata"
}
],
- Target 1 is a direct link to a TIFF of the page image.
- Target 2 is uncorrected, plain-text OCR output for this page.
- Target 3 is a "FITS Output" XML file containing information about image size, sampling/scanning parameters, tools used, etc.
Note that this is a member of the default sequence, not the manifest
"rendering": [
{
"@id": "https://dlcs.io/pdf/wellcome/pdf-item/b21051951/0",
"format": "application/pdf",
"label": "Download as PDF"
},
{
"@id": "https://wellcomelibrary.org/service/fulltext/b21051951/0?raw=true",
"format": "text/plain",
"label": "Download raw text"
}
],
- Target 1 is a direct link to a PDF of the entire work, apparently with a text layer only on the cover page, but not on the work contents itself.
- Target 2 is a plaintext file containing OCR output for the entire work, without page or line-break formatting.
George Washington, lettre au comte Louis-Marie de Noaillesau comte de Noailles, Princeton,
"seeAlso" : [ "http://oai.bnf.fr/oai2/OAIHandler?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai:bnf.fr:gallica/ark:/12148/btv1b105243418" ],
Target is an XML response to an OAI-PMH query for this item.
Added 2017-07-29 MS. Barocci 197
"seeAlso": "http://digital.bodleian.ox.ac.uk/inquire/p/eba7435d-673e-449b-8a8f-bd644ee88ea4",
Target is the catalog entry web page, containing the item in an embedded viewer.
"seeAlso":[
{
"@id":"https://opacplus.bsb-muenchen.de/title/BV037443140"
}
],
Target is a web page for the catalog record for this item, with links to viewers.
Bulle, Mus\u00e9e gru\u00e9rien, 383
"seeAlso":"http://www.e-codices.unifr.ch/en/list/one/mgb/0383",
Target is a web page for the catalog record for this item, including a thumbnail, link to a viewer, list of annotations, IIIF manifest icon, and reference images.
"seeAlso": {
"@id": "https://purl.stanford.edu/rh991sr7828.mods",
"format": "application/mods+xml"
},
Target is an XML MODS record.
"seeAlso" : "http://digi.ub.uni-heidelberg.de/diglit/codheidnf21_2/mets",
Target is an XML METS record.
The Re-Conquest of Ireland, A. D. 1895.
"seeAlso": [
{
"@id": "https://digital.ucd.ie/view/ivrla:2622.xml",
"format": "text/xml",
"profile": "http://www.loc.gov/mods/v3",
"label": "MODS metadata describing this object"
},
{
"@id": "https://digital.ucd.ie/view/ivrla:2622.n3",
"format": "text/rdf+n3",
"label": "RDF n3 serialisation of metadata describing this object"
},
{
"@id": "https://data.ucd.ie/api/edm/v1/ivrla:2622",
"format": "application/x.europeana-edm+xml",
"label": "EDM (Europeana Data Model) RDF metadata",
"profile": "http://www.europeana.eu/schemas/edm/"
},
{
"@id": "https://digital.ucd.ie/view/ivrla:2622.rdf",
"format": "application/rdf+xml",
"label": "RDF-XML metadata describing this object"
} ],
- Target 1 is XML metadata in MODS.
- Target 2 is item metadata in RDF N3.
- Target 3 is item metadata in EDM RDF.
- Target 4 is XML metadata in RDF.
"seeAlso": [
{
"@id": "https://wellcomelibrary.org/data/b21051951.json",
"format": "application/json",
"profile": "http://wellcomelibrary.org/profiles/res"
},
{
"@id": "https://wellcomelibrary.org/resource/schemaorg/b21051951",
"format": "application/ld+json",
"profile": "http://iiif.io/community/profiles/discovery/schema"
},
{
"@id": "https://wellcomelibrary.org/resource/dublincore/b21051951",
"format": "application/ld+json",
"profile": "http://iiif.io/community/profiles/discovery/dc"
}
],
- Target 1 is item metadata in a Wellcome-specific JSON format.
- Target 2 is item metadata in Schema.org JSON-LD.
- Target 3 is item metadata in Dublin Core JSON-LD.
Bulle, Mus\u00e9e gru\u00e9rien, 383
"service":[
{
"@context":"https://www.w3.org/TR/webmention/",
"@id":"http://www.e-codices.unifr.ch/webmention/receive",
"profile":"http://w3.org/TR/webmention",
"label":"e-codices Webmention Service"
}
],
Target is a REST endpoint for posting webmentions
"service": [
{
"@context": "http://wellcomelibrary.org/ld/iiif-ext/0/context.json",
"@id": "https://wellcomelibrary.org/iiif/b21051951-0/access-control-hints-service",
"profile": "http://wellcomelibrary.org/ld/iiif-ext/access-control-hints",
"accessHint": "open"
},
{
"@context": "http://iiif.io/api/search/0/context.json",
"@id": "https://wellcomelibrary.org/annoservices/search/b21051951",
"profile": "http://iiif.io/api/search/0/search",
"label": "Search within this manifest",
"service": {
"@id": "https://wellcomelibrary.org/annoservices/autocomplete/b21051951",
"profile": "http://iiif.io/api/search/0/autocomplete",
"label": "Get suggested words in this manifest"
}
},
{
"@context": "http://universalviewer.io/context.json",
"@id": "http://wellcomelibrary.org/service/trackingLabels/b21051951",
"profile": "http://universalviewer.io/tracking-extensions-profile",
"trackingLabel": "Format: monograph, Institution: Francis A. Countway Library of Medicine, Identifier: b21051951, Digicode: digmhl(francis), Collection code: n/a"
}
],
- Target 1 returned a 404 error
- Target 2 is a IIIF Search API response
- Service embedded within target 2 returned a server exception.
- Target 3 returned a 404 error.
Added 2017-07-17 Joseph Farington, 1747–1821, British, Caernarvon Castle, ca. 1780, Oil on canvas, Yale Center for British Art, B1976.7.118, Paintings and Sculpture
"seeAlso":[
{
"@id":"http://collection.britishart.yale.edu/id/data/object/340",
"format":"text/rdf+n3"
},
{
"@id":"https://manifests.britishart.yale.edu/xml/340.xml",
"format":"text/xml"
}
],
After discussion on the IIIF Community Call on 2017-07-19, I surveyed how viewers present the linking properties covered by this survey. To my knowledge, only Mirador, the Universal Viewer, the Bodleian Manifest Editor, and FromThePage are clients which consume IIIF manifests, although individual customizations of clients may present links differently, as with the Harry Ransom Center's change to place related links above metadata in their installation of Mirador.
The Manifest Editor prompts users to change related and seeAlso if they are present in the manifest, but it is unclear whether this functionality is fully supported. It does not expose rendering in the sequence metadata editor.
FromThePage does nothing at all with the four linking properties surveyed.
Mirador presents seeAlso and related links under the metadata block in the information pane.
No support for rendering or service (at least the Search API) was visible.
The Universal Viewer does not appear to display related properties.
The rendering property for a sequence is displayed on the pop-up a user sees when they click on the Download icon:
No support for the rendering at the canvas level was apparent.
The service property is supported for the Search API:
