269 dcat us writer optional fields (#282)

* Add DCAT-US Writer

Migrating changes from another branch.
Unit tests and writers for most required fields
and some if-applicable fileds.

* Add DCAT-US.md

* Remove Whitespace

* New Access Level Unit Test

Fix accessLevel writer.

* Add Unit Tests Access Level

New tests for public and restricted public.

* New Unit Test For License

* New Unit Test Spatial

* Remove Comment Headers

* New Temporal Unit Test

* Remove Comment Header

* Add DCAT-US Tests to Rakefile

* Update MiniTest to Minitest

* New Distribution Unit Test

* Complete Temporal Unit Test & Writer

* Update Rights Unit Test & Writer

* Remove Sample Files

* New Issued Field

Writer, unit test, and test file.
Enabled issue field in main dcat_us writer.

* New Theme Field

Writer, unit test, and test file.
Enabled theme in main dcat_us writer.

* New Landing Page Field

Writer, unit test, and test file.
Enabled landingPage in main writer.

* New IsPartOf Field

Writer, unit test, and test file.
Enabled isPartOf in main writer.

* Update DCAT-US.md

* New Described By Field

Writer, unit test, and test file.
Enabled describedBy in the main writer.

* New References Field

Writer, unit test, and test file.
Enabled references field in main writer.

* New System of Records Field

Writer, unit test, and test file.
Enabled systemOfRecords field in main writer.

DCAT-US.md

@@ -62,6 +62,6 @@
 | Collection | dcat:isPartOf | for each associatedResource[0, n].initiativeType = "collection" and associatedResource.associationType = "collectiveTitle" | associatedResource.resourceCitation[0].uri |
 | System of Records | dcat:systemOfRecords | [*Add code "sorn" to DS_InitiativeTypeCode*] <br> for each associatedResource[0, n].initiativeType = "sorn"   | associatedResource.resourceCitation[0].uri |
 | Primary IT Investment | dcat:primaryITInvestmentUII | | [*Links data to an IT investment identifier relative to Exhibit 53 docs, community valuation should be determined*] |
-| Data Dictionary | dcat:describedBy | if dataDictionary.dictionaryIncludedWithResource IS NOT TRUE and citation[0].onlineResource[0].uri exists | dataDictionary.citation[0].onlineResource[0].uri |
+| Data Dictionary | dcat:describedBy | if dataDictionary.dictionaryIncludedWithResource IS NOT TRUE and citation.onlineResource[0].uri exists | dataDictionary.citation.onlineResource[0].uri |
 | Data Dictionary Type | dcat:describedByType | | [*For simplicity, leave blank implying html page, community decision needed whether to support other format types using mime type and in the form of "application/pdf"*]|
 | Data Standard | dcat:conformsTo | | [*Currently not able to identify the schema standard the data conforms to, though this has been proposed. Should this be built and there is community decision to support it, then it can be mapped*] |

lib/adiwg/mdtranslator/writers/dcat_us/sections/dcat_us_described_by.rb

@@ -0,0 +1,30 @@
+require 'jbuilder'
+
+module ADIWG
+   module Mdtranslator
+      module Writers
+         module Dcat_us
+            module DescribedBy
+
+               def self.build(intObj)
+                  dataDictionaries = intObj[:dataDictionaries]
+
+                  dataDictionaries.each do |dataDictionary|
+                    if !dataDictionary[:dictionaryIncludedWithResource]
+                      onlineResources = dataDictionary[:citation][:onlineResources]
+                      onlineResources.each do |resource|
+                        if resource[:olResURI]
+                          return resource[:olResURI]
+                        end
+                      end
+                    end
+                  end
+
+                  return nil
+                end                
+
+            end
+         end
+      end
+   end
+end

lib/adiwg/mdtranslator/writers/dcat_us/sections/dcat_us_is_part_of.rb

@@ -0,0 +1,27 @@
+require 'jbuilder'
+
+module ADIWG
+   module Mdtranslator
+      module Writers
+         module Dcat_us
+            module IsPartOf
+
+               def self.build(intObj)
+                  associatedResources = intObj.dig(:metadata, :associatedResources)
+
+                  associatedResources.each do |resource|
+                    next unless resource[:initiativeType] == "collection" && resource[:associationType] == "collectiveTitle"
+
+                    onlineResources = resource.dig(:resourceCitation, :onlineResources) || []
+                    uri = onlineResources.find { |onlineResource| onlineResource[:olResURI] }&.dig(:olResURI)
+                    return uri if uri
+                  end
+
+                  nil
+                end                
+
+            end
+         end
+      end
+   end
+end

lib/adiwg/mdtranslator/writers/dcat_us/sections/dcat_us_issued.rb

@@ -0,0 +1,19 @@
+require 'jbuilder'
+
+module ADIWG
+   module Mdtranslator
+      module Writers
+         module Dcat_us
+            module Issued
+
+               def self.build(intObj)
+                  dates = intObj[:metadata][:resourceInfo][:citation][:dates].map { |obj| obj[:date] }
+                  earliest_date = dates.min
+                  return earliest_date
+                end
+
+            end
+         end
+      end
+   end
+end

lib/adiwg/mdtranslator/writers/dcat_us/sections/dcat_us_landing_page.rb

@@ -0,0 +1,27 @@
+require 'jbuilder'
+
+module ADIWG
+   module Mdtranslator
+      module Writers
+         module Dcat_us
+            module LandingPage
+
+               def self.build(intObj)
+                  onlineResources = intObj.dig(:metadata, :resourceInfo, :citation, :onlineResources)
+
+                  if onlineResources
+                    onlineResources.each do |resource|
+                      if resource.dig(:olResFunction) == 'landingPage'
+                        return resource.dig(:olResURI)
+                      end
+                    end
+                  end
+
+                  return nil
+                end                
+
+            end
+         end
+      end
+   end
+end

lib/adiwg/mdtranslator/writers/dcat_us/sections/dcat_us_references.rb

@@ -0,0 +1,50 @@
+require 'jbuilder'
+
+module ADIWG
+   module Mdtranslator
+      module Writers
+         module Dcat_us
+            module References
+
+               def self.build(intObj)
+                  metadata = intObj.dig(:metadata)
+                  associated_resources = metadata&.dig(:associatedResources)
+                  additional_documents = metadata&.dig(:additionalDocuments)
+                  uris = []
+
+                  if associated_resources
+                    associated_resources.each do |associated|
+                      citation = associated.dig(:resourceCitation)
+                      online_resources = citation&.dig(:onlineResources)
+                      next unless online_resources
+
+                      online_resources.each do |online|
+                        uri = online.dig(:olResURI)
+                        uris << uri if uri
+                      end
+                    end
+                  end
+
+                  if additional_documents
+                    additional_documents.each do |additional|
+                      citations = additional&.dig(:citation) || []
+                      citations.each do |citation|
+                        online_resources = citation&.dig(:onlineResources)
+                        next unless online_resources
+
+                        online_resources.each do |online|
+                          uri = online.dig(:olResURI)
+                          uris << uri if uri
+                        end
+                      end
+                    end
+                  end
+
+                  uris.join(',')
+               end
+
+            end
+         end
+      end
+   end
+end

lib/adiwg/mdtranslator/writers/dcat_us/sections/dcat_us_system_of_records.rb

@@ -0,0 +1,28 @@
+require 'jbuilder'
+
+module ADIWG
+   module Mdtranslator
+      module Writers
+         module Dcat_us
+            module SystemOfRecords
+
+               def self.build(intObj)
+                  associatedResources = intObj.dig(:metadata, :associatedResources)
+
+                  return nil if associatedResources.nil?
+
+                  associatedResources.each do |resource|
+                    if resource[:initiativeType] == 'sorn'
+                      onlineResources = resource.dig(:resourceCitation, :onlineResources)
+                      return onlineResources.first[:olResURI] if onlineResources&.first&.has_key?(:olResURI)
+                    end
+                  end
+
+                  return nil
+               end
+
+            end
+         end
+      end
+   end
+end

lib/adiwg/mdtranslator/writers/dcat_us/sections/dcat_us_theme.rb

@@ -0,0 +1,28 @@
+require 'jbuilder'
+
+module ADIWG
+   module Mdtranslator
+      module Writers
+         module Dcat_us
+            module Theme
+
+               def self.build(intObj)
+                  resourceInfo = intObj[:metadata][:resourceInfo]
+                  keywords_str = []
+
+                  resourceInfo[:keywords].each do |keyword_group|
+                    if keyword_group[:thesaurus][:title] == "ISO Topic Categories"
+                      keyword_group[:keywords].each do |keyword_obj|
+                        keywords_str << keyword_obj[:keyword]
+                      end
+                    end
+                  end
+
+                  return keywords_str.join(" ")
+               end                
+
+            end
+         end
+      end
+   end
+end

test/writers/dcat_us/tc_dcat_us_described_by.rb

@@ -0,0 +1,24 @@
+require 'minitest/autorun'
+require 'json'
+require 'adiwg-mdtranslator'
+require_relative 'dcat_us_test_parent'
+
+class TestWriterDcatUsDescribedBy < TestWriterDcatUsParent
+
+   # get input JSON for test
+   @@jsonIn = TestWriterDcatUsParent.getJson('describedBy.json')
+
+   def test_described_by
+      metadata = ADIWG::Mdtranslator.translate(
+         file: @@jsonIn, reader: 'mdJson', validate: 'normal',
+         writer: 'dcat_us', showAllTags: false)
+
+      hJsonOut = JSON.parse(metadata[:writerOutput])
+      got = hJsonOut['dcat:describedBy']
+
+      expect = 'http://adiwg.org/described/by'
+
+      assert_equal expect, got
+   end
+
+end

test/writers/dcat_us/tc_dcat_us_is_part_of.rb

@@ -0,0 +1,24 @@
+require 'minitest/autorun'
+require 'json'
+require 'adiwg-mdtranslator'
+require_relative 'dcat_us_test_parent'
+
+class TestWriterDcatUsIsPartOf < TestWriterDcatUsParent
+
+   # get input JSON for test
+   @@jsonIn = TestWriterDcatUsParent.getJson('isPartOf.json')
+
+   def test_isPartOf
+      metadata = ADIWG::Mdtranslator.translate(
+         file: @@jsonIn, reader: 'mdJson', validate: 'normal',
+         writer: 'dcat_us', showAllTags: false)
+
+      hJsonOut = JSON.parse(metadata[:writerOutput])
+      got = hJsonOut['dcat:isPartOf']
+
+      expect = 'http://ISO.uri/adiwg/0'
+
+      assert_equal expect, got
+   end
+
+end

test/writers/dcat_us/tc_dcat_us_issued.rb

@@ -0,0 +1,24 @@
+require 'minitest/autorun'
+require 'json'
+require 'adiwg-mdtranslator'
+require_relative 'dcat_us_test_parent'
+
+class TestWriterDcatUsIssued < TestWriterDcatUsParent
+
+   # get input JSON for test
+   @@jsonIn = TestWriterDcatUsParent.getJson('issued.json')
+
+   def test_issued
+      metadata = ADIWG::Mdtranslator.translate(
+         file: @@jsonIn, reader: 'mdJson', validate: 'normal',
+         writer: 'dcat_us', showAllTags: false)
+
+      hJsonOut = JSON.parse(metadata[:writerOutput])
+      got = hJsonOut['dcat:issued']
+
+      expect = '2017-06-13T00:00:00+00:00'
+
+      assert_equal expect, got
+   end
+
+end

test/writers/dcat_us/tc_dcat_us_landing_page.rb

@@ -0,0 +1,24 @@
+require 'minitest/autorun'
+require 'json'
+require 'adiwg-mdtranslator'
+require_relative 'dcat_us_test_parent'
+
+class TestWriterDcatUsLandingPage < TestWriterDcatUsParent
+
+   # get input JSON for test
+   @@jsonIn = TestWriterDcatUsParent.getJson('landingPage.json')
+
+   def test_landingPage
+      metadata = ADIWG::Mdtranslator.translate(
+         file: @@jsonIn, reader: 'mdJson', validate: 'normal',
+         writer: 'dcat_us', showAllTags: false)
+
+      hJsonOut = JSON.parse(metadata[:writerOutput])
+      got = hJsonOut['dcat:landingPage']
+
+      expect = 'http://ISO.uri/adiwg/0'
+
+      assert_equal expect, got
+   end
+
+end

test/writers/dcat_us/tc_dcat_us_references.rb

@@ -0,0 +1,24 @@
+require 'minitest/autorun'
+require 'json'
+require 'adiwg-mdtranslator'
+require_relative 'dcat_us_test_parent'
+
+class TestWriterDcatUsReferences < TestWriterDcatUsParent
+
+   # get input JSON for test
+   @@jsonIn = TestWriterDcatUsParent.getJson('references.json')
+
+   def test_references
+      metadata = ADIWG::Mdtranslator.translate(
+         file: @@jsonIn, reader: 'mdJson', validate: 'normal',
+         writer: 'dcat_us', showAllTags: false)
+
+      hJsonOut = JSON.parse(metadata[:writerOutput])
+      got = hJsonOut['dcat:references']
+
+      expect = 'http://ISO.uri/adiwg/0,http://ISO.uri/adiwg/1,http://ISO.uri/adiwg/2,http://ISO.uri/adiwg/3,http://ISO.uri/adiwg/4,http://ISO.uri/adiwg/5,http://ISO.uri/adiwg/6,http://ISO.uri/adiwg/7,http://ISO.uri/adiwg/8,http://ISO.uri/adiwg/9,http://ISO.uri/adiwg/10,http://ISO.uri/adiwg/11'
+
+      assert_equal expect, got
+   end
+
+end

test/writers/dcat_us/tc_dcat_us_system_of_records.rb

@@ -0,0 +1,24 @@
+require 'minitest/autorun'
+require 'json'
+require 'adiwg-mdtranslator'
+require_relative 'dcat_us_test_parent'
+
+class TestWriterDcatUsSystemOfRecords < TestWriterDcatUsParent
+
+   # get input JSON for test
+   @@jsonIn = TestWriterDcatUsParent.getJson('systemOfRecords.json')
+
+   def test_system_of_records
+      metadata = ADIWG::Mdtranslator.translate(
+         file: @@jsonIn, reader: 'mdJson', validate: 'normal',
+         writer: 'dcat_us', showAllTags: false)
+
+      hJsonOut = JSON.parse(metadata[:writerOutput])
+      got = hJsonOut['dcat:systemOfRecords']
+
+      expect = 'http://ISO.uri/adiwg/0'
+
+      assert_equal expect, got
+   end
+
+end

test/writers/dcat_us/tc_dcat_us_theme.rb

@@ -0,0 +1,24 @@
+require 'minitest/autorun'
+require 'json'
+require 'adiwg-mdtranslator'
+require_relative 'dcat_us_test_parent'
+
+class TestWriterDcatUsTheme < TestWriterDcatUsParent
+
+   # get input JSON for test
+   @@jsonIn = TestWriterDcatUsParent.getJson('theme.json')
+
+   def test_theme
+      metadata = ADIWG::Mdtranslator.translate(
+         file: @@jsonIn, reader: 'mdJson', validate: 'normal',
+         writer: 'dcat_us', showAllTags: false)
+
+      hJsonOut = JSON.parse(metadata[:writerOutput])
+      got = hJsonOut['dcat:theme']
+
+      expect = 'inlandWaters location climatologyMeteorologyAtmosphere'
+
+      assert_equal expect, got
+   end
+
+end