add rml:languageMap

[VladimirAlexiev]

This issue is about http://rml.io/spec.html but I couldn't find a repo for that spec. If this is the wrong repo for this issue, please let me know and I'll move it.

There is no standard way to take the lang tag of a literal from a data field.
Back in Sep 2013 I proposed an extension rrx:languageColumn that does this (and uses the standard rr:language as fallback in case of missing value).

• implemented in perl: https://metacpan.org/pod/release/TOBYINK/RDF-RDB2RDF-0.008/lib/RDF/RDB2RDF/R2RML.pm#Language-Extension
• implemented in db2triples: https://github.com/antidot/db2triples/issues/13
• implemented in Ultrawrap
• under consideration in carml: multiReference with "multilanguage" carml/carml#39
• used in Getty LOD: http://dipp.math.bas.bg/images/2018/019-050_32_11-iDiPP2018-34.pdf
• in tutorial by Christophe Debruyne and Ordnance Survey Ireland: https://image.slidesharecdn.com/seminar-4-171104171711/95/uplift-generating-rdf-datasets-from-nonrdf-data-with-r2rml-39-638.jpg?cb=1509977153
• in Trinity College course: https://www.scss.tcd.ie/~mcglink/lectures/CS7012/tutorials/r2rml/tutorial_r2rml_csv_slides.pptx

Please add this feature to RML.
It uses a namespace rrx: http://purl.org/r2rml-ext/ but if you want, we can re-host it under the RML namespace to increase standardization/uptake.

Thanks!

[andimou]

@VladimirAlexiev thanks for the recommendation!

we encountered the same issue long time ago and we approached it by introducing the rml:languageMap property, following the same pattern as for subjectMap, predicateMap, objectMap.
However we never documented it in the spec. Note to myself: update the spec.

The rml:languageMap expects an rr:TermMap in its range so all options, namely rr:template, rr:constant, rr:column, rml:reference are available. This way, rr:language acts as a short version of rml:languageMap when the rml:languageMap is combined with rr:constant, in the same way that rr:predicate acts as the shortcut of rr:predicateMap if the latter is combined with rr:constant.

See this test case https://github.com/RMLio/RML-Processor/tree/05266b836857f3c528db7a134fa0b84559a34432/src/test/resources/exampleLanguageMap

As a Term Map is combined with FnO.io for transformations, it is also possible to modify the value from the original data, e.g. if the original data value is "engish", a function can be used to change it to a valid language tag, e.g. "en".

As far as I see, rrx:languageColumn covers only references and it will also be difficult to have any modifications.

[VladimirAlexiev]

Agree, this sounds better.
Please also add some test cases. Cheers!

[gedro]

Hi @andimou,

I've tried to use the test cases you mentioned above. In my opinion they shouldn't work because of rml:reference "Localization[@Culture]" part. This is a valid xPath for elements but the language value is in the Culture attribute.
I also tried to replace it with: rml:reference "Localization/@Culture" but then I got the same language for all the literals. (I think this could be due to a bug)
Could you please help me how to use properly the rml:languageMap with xPath rml:referenceFormulation?

[VladimirAlexiev]

• The spec still doesn't have rml:languageTag, it has only the constant rr:language https://rml.io/specs/rml/#language-tag
• The spec should also say that rr:language is used as fallback for rows where the designated language value is null
• How about supporting this in YARRML? This almost works in Matey https://rml.io/yarrrml/matey/#edit but not quite:
  YARRML:

prefixes:
  ex: "http://example.com/"
mappings:
  person:
    sources:
      - ['persons.json~jsonpath', '$.persons[*]']
    s: http://example.com/person/$(firstname)
    po:
      - [a, foaf:Person]
      - [ex:lastName, $(lastname), $(lang)~lang]
      - [ex:score, $(num), $(dt)]

Data:

{
    "persons": [
        {
            "firstname": "John",
            "lastname": "Doe",
            "lang": "en",
            "num": 3,
            "dt": "xsd:integer",
        },
        {
            "firstname": "Jane",
            "lastname": "Smith",
            "lang": "fr",
            "num": "3.14",
            "dt": "xsd:decimal",
        },
    ]
}

RML:

map:om_2 a rr:ObjectMap;
    rml:reference "lastname";
    rr:termType rr:Literal;
    rr:language "$(lang)".  ## close but not quite
map:om_3 a rr:ObjectMap;
    rml:reference "num";
    rr:termType rr:Literal;
    rr:datatype <$(dt)>.    ## See next issue