Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switchover to using Registry API for generating context products json #1036

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

jordanpadams
Copy link
Member

🗒️ Summary

Query Registry API to generate the context products JSON file used for context product validation

⚙️ Test Data and/or Report

$ validate -u

PDS Validate Tool Report

Configuration:
   Version     3.6.0-SNAPSHOT
   Date        2024-10-17T21:50:32Z

Parameters:
   Targets                      []
   Severity Level               WARNING
   Recurse Directories          true
   File Filters Used            [*.xml, *.XML]
   Data Content Validation      on
   Product Level Validation     on
   Max Errors                   100000
   Registered Contexts File     /Users/jpadams/proj/pds/pdsen/workspace/validate/validate-3.6.0-SNAPSHOT/resources/registered_context_products.json



Product Level Validation Results

  PASS: /Users/jpadams/proj/pds/pdsen/workspace/validate/validate-3.6.0-SNAPSHOT/resources/registered_context_products.json
    Begin Fragment: file:/Users/jpadams/proj/pds/pdsen/workspace/validate/validate-3.6.0-SNAPSHOT/resources/registered_context_products.json
      INFO  [info.validation.general]   Successfully updated registered context products config file from PDS Search API.
      INFO  [info.validation.general]   4590 registered context products found.
    End Fragment: file:/Users/jpadams/proj/pds/pdsen/workspace/validate/validate-3.6.0-SNAPSHOT/resources/registered_context_products.json
        1 product validation(s) completed

Summary:

  1 product(s)
  0 error(s)
  0 warning(s)

  Product Validation Summary:
    1          product(s) passed
    0          product(s) failed
    0          product(s) skipped
    1          product(s) total

  Referential Integrity Check Summary:
    0          check(s) passed
    0          check(s) failed
    0          check(s) skipped
    0          check(s) total


End of Report
Completed execution in 38713 ms

# testing running validate against a product
PDS Validate Tool Report

Configuration:
   Version     3.6.0-SNAPSHOT
   Date        2024-10-17T21:46:01Z

Parameters:
   Targets                      [file:/Users/jpadams/proj/pds/pdsen/workspace/validate/src/test/resources/github597/document/spiceds_v001.xml]
   Severity Level               WARNING
   Recurse Directories          true
   File Filters Used            [*.xml, *.XML]
   Data Content Validation      on
   Product Level Validation     on
   Max Errors                   100000
   Registered Contexts File     /Users/jpadams/proj/pds/pdsen/workspace/validate/validate-3.6.0-SNAPSHOT/resources/registered_context_products.json



Product Level Validation Results

  PASS: file:/Users/jpadams/proj/pds/pdsen/workspace/validate/src/test/resources/github597/document/spiceds_v001.xml
      WARNING  [warning.label.context_ref_mismatch]   line 37: Context reference name mismatch. Value: 'Mars 2020 Perseverance Rover Mission' Expected one of: '[Mars 2020]'
      WARNING  [warning.label.context_ref_mismatch]   line 46: Context reference name mismatch. Value: 'Mars 2020 Perseverance Rover' Expected one of: '[Perseverance]'
        1 product validation(s) completed

Summary:

  1 product(s)
  0 error(s)
  2 warning(s)

  Product Validation Summary:
    1          product(s) passed
    0          product(s) failed
    0          product(s) skipped
    1          product(s) total

  Referential Integrity Check Summary:
    0          check(s) passed
    0          check(s) failed
    0          check(s) skipped
    0          check(s) total

  Message Types:
    2            warning.label.context_ref_mismatch

End of Report
Completed execution in 4617 ms

## ♻️ Related Issues
<!--
    Reference related issues here and use `Fixes` or `Resolves` in order to automatically close the issue upon merge. For more information on autolinking to tickets see https://docs.github.com/en/github/writing-on-github/autolinked-references-and-urls.

    * for issues in this repo:
        - fixes #1
        - fixes #2
        - refs #3
    * for issues in other repos: NASA-PDS/my_repo#1, NASA-PDS/her_repo#2
-->

Resolves #675 

Al Niessner and others added 5 commits August 30, 2023 13:48
Change the validate.properties to use the registry-api. Adjusted code to get all data then walk through the results as lists and maps of the JSON response object.
* updated to use `search-after`
* added `useDelimiter` to Scanner in order to keep all whitespace
* updated search query to ignore extraneous context products

resolves #675
@jordanpadams jordanpadams requested a review from a team as a code owner October 17, 2024 21:54
Copy link
Member

@nutjob4life nutjob4life left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reproduced just fine:

$ git checkout issue_675
branch 'issue_675' set up to track 'origin/issue_675'.
Switched to a new branch 'issue_675'
$ mvn -Dmaven.test.skip=true clean package
[INFO] Scanning for projects...
[INFO] Inspecting build with total of 1 modules...

[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  33.590 s
[INFO] Finished at: 2024-10-17T16:56:51-05:00
[INFO] ------------------------------------------------------------------------
66.594u 8.499s 0:36.63 204.9%   0+0k 0+0io 3201pf+0w
$ tar xzf target/validate-3.6.0-SNAPSHOT-bin.tar.gz
$ validate-3.6.0-SNAPSHOT/bin/validate -u

PDS Validate Tool Report

Configuration:
   Version     3.6.0-SNAPSHOT
   Date        2024-10-17T21:57:23Z

Parameters:
   Targets                      []
   Severity Level               WARNING
   Recurse Directories          true
   File Filters Used            [*.xml, *.XML]
   Data Content Validation      on
   Product Level Validation     on
   Max Errors                   100000
   Registered Contexts File     /Users/kelly/Documents/Clients/JPL/PDS/Development/nasa-pds/validate/validate-3.6.0-SNAPSHOT/resources/registered_context_products.json

Summary:

  1 product(s)
  0 error(s)
  0 warning(s)

  Product Validation Summary:
    1          product(s) passed
    0          product(s) failed
    0          product(s) skipped
    1          product(s) total

  Referential Integrity Check Summary:
    0          check(s) passed
    0          check(s) failed
    0          check(s) skipped
    0          check(s) total


End of Report
Completed execution in 55810 ms

Copy link
Contributor

@al-niessner al-niessner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this removes all solr (it should) then remove solr from pom please.

ditch solr
@al-niessner al-niessner dismissed their stale review October 17, 2024 23:17

I removed solr from pom

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants