Translation Alignments: Port translation alignment endpoints from Flask app to ATLAS

[jacobwegner]

Models translation alignments in ATLAS and provides hooks to import alignment data and query TextAlignmentChunk data by reference.

Each TextAlignmentChunk has both start and end Line FKs as well as a contains property that can be used to retrieve all TextPart instances within the range.

importers.alignments handles importing the data from Homeric Iliad and Odyssey aligned at the sentence level to the A. T. Murray Translation.

When retrieving TextAlignmentChunk instances, the actual alignment data is stored in an items JSON field with the following format:

[
  [
    // the Greek line refeference, e.g. 1.1-1.7 or 1.8
    "<bookPosition>.<linePosition>"
    // the Greek text content
    "<content>",
    // a valid value enumerated by the `LINE_KIND_<foo>` constants 
    // that indicates if the line is a new line, is continued by 
    // the next line, or is a continuation of the previous line
    "<continuation-data>"
  ],
  [
    // the English line refeference, e.g. 1.1-1.7 or 1.8
    "<bookPosition>.<linePosition>"
    // the English text content
    "<content>",
    // the value enumerated as LINE_KIND_UNKNOWN (since the)
    // Greek lines are being aligned to the English sentences
    null
  ],
]

[jacobwegner]

This is being ported from the examples provided at:

scaife-viewer/readhomer#25

and implemented within:

https://github.com/scaife-viewer/readhomer/blob/d9f8b5cc36af2c7fa99f2d703aa3d9a46b99317a/src/homer/api.js#L21

[jacobwegner]

I've pushed, deployed and loaded all alignment data. Still working out the best way to support a "reference" based query for alignment chunks like was done in scaife-viewer/readhomer#25

[jtauber]

Still working out the best way to support a "reference" based query for alignment chunks

I think this is an example of a more general pattern where you want to retrieve As but you want to specify the range of As in terms of some other referencing scheme based on Bs.

Other examples might include, "give me the sentences in John 3.1–3.11" or give me the Beowulf fitts for lines 1000–1500.

I think the basic algorithm is: (1) work out the (first) fitt containing line 1000; (2) work out the (last) fitt containing line 1500; (3) use those two fitts as the "new" range returned. We might want to annotate the results with the fact it was actually lines 1000 to 1500 that were requests (and the two chunking systems aren't perfectly aligned.

Obviously that means we need to index to support (1) and (2).

Anyway, IMO all this applies to translation units containing other referencing schemes too.

[jacobwegner]

@jtauber I agree; my original comment could have been clearer in that I was trying to figure out how to support a reference-based query in GraphQL / graphene-django.

I'd already set up a contains relation and have begun implementing a custom filter in
42cecdc:

Sample query

{
  alignmentChunks(version_Urn: "urn:cts:greekLit:tlg0012.tlg001.perseus-grc2", reference: "1.1-1.6",) {
    edges {
      cursor
      node {
        id
        citation
        items
      }
    }
  }
}

Will still want to work out the "annotation" approach for GraphQL as well. The Flask-based API had a metadata entry for reference_resolved:

https://readhomer-dev-api.herokuapp.com/urn:cts:greekLit:tlg0012.tlg001.perseus-grc2/alignment/eng/1.1-1.6/

[jacobwegner]

I've updated this branch based on some things we've worked on with KITAB's ATLAS server.

LineReferenceFilterMixin is used to implement a "range" query (rather than relying on a contains relation).