Home

Useful links

• LOFAR Long-Term Archive (LTA)

  • web interface
  • web service
  • Python API (Astro-WISE)

• List of papers referencing LOFAR data (via the SAO/NASA Astrophysics Data System)

• Physics/astronomy-related data standards, controlled vocabs, ontologies etc.

  • IVOA: Docs & standards | Semantics Working Group
  • IVOA vocabularies: Description | Download
    • UCD (Unified Content Descriptors vocabulary for describing astronomical data quantities)
    • IVOAT Thesaurus
    • AAkeys
  • Ontology of Astronomical Object Types
  • Astronomy/Science ontologies
  • Ontology of units of Measure (OM)
  • QUDT Ontologies for units of measure, quantity kind, dimensions and data types
  • Flexible Image Transport System (FITS) data format
  • LOFAR MeasurementSet

• example catalogs:

  • Sloan digital sky survey (SDSS): http://www.sdss.org
  • Mikulski Archive for Space Telescopes (MAST): https://archive.stsci.edu/
  • Herschel Science Archive: http://archives.esac.esa.int/hsa/whsa/

• FAIR/Liked Data tools, standards and services:

  • FAIR Data Guiding Principles
  • Linked Data approach
  • W3C RDB2RDF recommendations
  • Virtuoso Universal Server (incl. R2RML processor)
  • FAIR Data Point (example instance)
  • Oracle RDF Graph
  • grlc WebAPI
  • OpenRefine
  • Protege
  • FAIRsharing.org
  • EUDAT B2x services
    • B2HANDLE, ePIC PID service at SURFsara
    • B2SAFE (-FAIR)
  • Zenodo
  • FAIR Metrics (ALL)
    • FAIR Metrics Evaluation Results

TODO

1. Get to know your data(sets) and use case.

• Which data are relevant or subject of this use case? (e.g. see LOFAR DBView)
  • The terms "Epoch, Spectral Window, Pointing, FWHM, Duration" were identified by Astronomy expert Hanno Spreeuw as relevant. In Ontology, columns of the MS were identified as relevant. This lead to the selection of "Observation", "Pointing" and "SubArrayPointing" as the 3 DB tables to analyse.
• How do you access the (meta)data using which protocols?
• Which data can (not) be referred to by persistent IDs?
• In what formats are the data distributed?
• How are the data structured or organized? (e.g. catalog->dataset->distribution for use of the FAIR Data Point metadata service)
• Is this structure machine-readable? Suggestion: Install the (Python) FDP and try to fill in the config file.

2. Make the (meta)data available in a machine-readable and semantically interoperable format(s).

• Which (domain-specific) vocabularies and/or ontologies are relevant for this use case? (e.g. DCMI, DCAT, VoID, IVOA Ontology)
• Which terms (or URIs/IRIs) can be used to describe (identify) the data and associated metadata?
• Transform the relational data into RDF graph ("semantification") and deploy using Linked Data services

Complementary Ideas

1. Improve the Search UI

• Create a bol.com styled faceted search
• Use an Oracle search interface
  • Application Express: https://apex.oracle.com/en/
  • Secure Enterprise Search: http://www.oracle.com/technetwork/search/oses/ses-twp-10-1-133183.pdf (page 10)
• ElasticSearch: https://www.elastic.co/products/elasticsearch + https://github.com/appbaseio/dejavu
• Apache Lucene + Solr: http://lucene.apache.org/solr/

2. Apply IVOA standards and tools

• The Virtual Observatory (VO) is the vision that astronomical datasets and other resources should work as a seamless whole. Many projects and data centres worldwide are working towards this goal. The International Virtual Observatory Alliance (IVOA) is an organisation that debates and agrees the technical standards that are needed to make the VO possible.
• Publishing Data into the VO: http://wiki.ivoa.net/twiki/bin/view/IVOA/PublishingInTheVO
• European VO software: http://www.euro-vo.org/?q=science/software
• TAP Service (the "Virtuoso" of Astronomy): http://saada.unistra.fr/taphandle/
• Search tool #1: http://svo.cab.inta-csic.es/docs/index.php?pagename=Projects/Astronomical_Data_Centres/Publishing including ConeSearch
• Search tool #2: https://github.com/csiro-rds/casda_vo_tools including ConeSearch, TAP etc.
• Search tool #3: http://ia2.oats.inaf.it/vo-services/vo-dance including ConeSearch, TAP etc.

Ontology-based Database Access

See Ontology/schema mapping page.

References

Wilkinson et al. (2018) A design framework and exemplar metrics for FAIRness, Scientific Data, 5. doi: 10.25504/FAIRsharing.WWI10U

Wilkinson et al. (2016) The FAIR Guiding Principles for scientific data management and stewardship, Scientific Data, 3. doi: 10.1038/sdata.2016.18

Wilkinson et al. (2017) Interoperability and FAIRness through a Novel Combination of Web Technologies. PeerJ Computer Science, 3. doi: 10.7717/peerj-cs.110

Mons et al. (2018) Cloudy, increasingly FAIR; revisiting the FAIR Data guiding principles for the European Open Science Cloud. Information Services and Use, 37, 49-56. doi: 10.3233/ISU-170824