KMS-545: Streamlined download of legacy data to use new KMS endpoint.

[cgokey]

Overview

What is the feature?

The focus of this ticket is to streamline the creation of RDF from a single source, legacy JSON, and no longer use legacy XML. The old code would use both legacy json and xml and now KMS provides a new endpoint that includes all data needed to build RDF from a single source.

What is the Solution?

Updated code to no longer call legacy xml and look in legacy json for all fields when converting to RDF.
Also updated the scripts to use the new endpoint for creating test data, such as published and draft.

What areas of the application does this impact?

syncConceptData has changed to only download the json data and use that for creating the RDF.

The scripts have changed to no longer download individual concepts, but rather the entire json batch which can be used for processing and converting to rdf.

Testing

Test synch data.
Run syncData, i.e., startup offline mode with:
IS_OFFLINE=true SYNC_API_ENDPOINT=https://gcmd.sit.earthdata.nasa.gov SHOULD_SYNC=true RDF4J_SERVICE_URL=http://localhost:8080 serverless offline start
Then try calling the syncData handler.
curl -X POST http://localhost:4001/dev/sync-concept-data -H "Content-Type: application/json" -d '{"version": "published", "versionType": "published"}'

Test scripts that download and create RDF
(Note you'll need to be on VPN, as it is accessing SIT)
npm run download-data (will download the json data for published and draft)
npm run create-rdf-files (will convert the files to RDF)
Check data directory for created files.

Checklist

• I have added automated tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings

[mandyparson]

In both skos:definition and in changeNote, the text is coming in with '. Will need to format text in both places. See 027286de-800d-4141-b17b-6df71fbef30c in concepts_draft

[cgokey]

In both skos:definition and in changeNote, the text is coming in with '. Will need to format text in both places. See 027286de-800d-4141-b17b-6df71fbef30c in concepts_draft

I think this is just your browser showing it, try: curl "http://localhost:4001/dev/concept/027286de-800d-4141-b17b-6df71fbef30c?version=draft"

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.79%. Comparing base (bc49b89) to head (cdba58b).

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #39      +/-   ##
==========================================
+ Coverage   99.74%   99.79%   +0.04%     
==========================================
  Files         120      120              
  Lines        1998     1986      -12     
  Branches      476      471       -5     
==========================================
- Hits         1993     1982      -11     
+ Misses          5        4       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.