tre: add MATESA support #497
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
pressler-vsc
force-pushed
the
MATESA_integration
branch
from
a4fed9a
to
1088c90
Compare
pressler-vsc
approved these changes
Mar 4, 2024
There was a problem hiding this comment.
This appears consistent with both the STDI-0002 Volume 1 Appendix AK and the TRE in the sample NITF on the nsgreg. Nice PR and thanks for linking the sample data.
tests pass
SARPY_TEST_PATH=/data/sarpy_test nox
nox > Running session test(version='3.6')
nox > Creating conda env in .nox/test-version-3-6 with python
nox > conda install --yes --prefix /home/vscuser/git/glweb/software/third-party/sarpy/.nox/test-version-3-6 python=3.6
nox > python -m pip install '.[all]'
nox > pytest tests
======================================================================= test session starts ========================================================================
platform linux -- Python 3.6.13, pytest-7.0.1, pluggy-1.0.0
rootdir: /home/vscuser/git/glweb/software/third-party/sarpy
collected 543 items
tests/test_class_string.py . [ 0%]
tests/consistency/test_consistency.py ........ [ 1%]
tests/consistency/test_cphd_consistency.py .............................................................. [ 13%]
tests/consistency/test_sicd_consistency.py ... [ 13%]
tests/consistency/test_sidd_consistency.py .... [ 14%]
tests/geometry/test_geocoords.py .......... [ 16%]
tests/geometry/test_geometry_elements.py .......... [ 18%]
tests/geometry/test_latlon.py ... [ 18%]
tests/geometry/test_point_projection.py ............. [ 20%]
tests/io/test_kml.py . [ 21%]
tests/io/DEM/test_geoid.py . [ 21%]
tests/io/DEM/test_geotiff1deg_list.py .... [ 22%]
tests/io/DEM/test_geotiff1deg_reader.py ........ [ 23%]
tests/io/complex/test_other_nitf.py ... [ 24%]
tests/io/complex/test_reader.py .ssssssssss [ 26%]
tests/io/complex/test_remote.py s [ 26%]
tests/io/complex/test_sicd.py . [ 26%]
tests/io/complex/test_utils.py . [ 26%]
tests/io/complex/sicd_elements/test_sicd_elements_antenna.py ... [ 27%]
tests/io/complex/sicd_elements/test_sicd_elements_base.py ... [ 27%]
tests/io/complex/sicd_elements/test_sicd_elements_blocks.py ..................... [ 31%]
tests/io/complex/sicd_elements/test_sicd_elements_collectioninfo.py .. [ 32%]
tests/io/complex/sicd_elements/test_sicd_elements_errorstatistics.py . [ 32%]
tests/io/complex/sicd_elements/test_sicd_elements_geodata.py ... [ 32%]
tests/io/complex/sicd_elements/test_sicd_elements_grid.py .. [ 33%]
tests/io/complex/sicd_elements/test_sicd_elements_imagecreation.py . [ 33%]
tests/io/complex/sicd_elements/test_sicd_elements_imagedata.py . [ 33%]
tests/io/complex/sicd_elements/test_sicd_elements_imageformation.py ...... [ 34%]
tests/io/complex/sicd_elements/test_sicd_elements_matchinfo.py . [ 34%]
tests/io/complex/sicd_elements/test_sicd_elements_pfa.py . [ 34%]
tests/io/complex/sicd_elements/test_sicd_elements_position.py ... [ 35%]
tests/io/complex/sicd_elements/test_sicd_elements_radarcollection.py .............. [ 38%]
tests/io/complex/sicd_elements/test_sicd_elements_radiometric.py . [ 38%]
tests/io/complex/sicd_elements/test_sicd_elements_rgazcomp.py . [ 38%]
tests/io/complex/sicd_elements/test_sicd_elements_rma.py ... [ 39%]
tests/io/complex/sicd_elements/test_sicd_elements_scpcoa.py ................... [ 42%]
tests/io/complex/sicd_elements/test_sicd_elements_sicd.py .................................. [ 48%]
tests/io/complex/sicd_elements/test_sicd_elements_timeline.py .................. [ 52%]
tests/io/complex/sicd_elements/test_sicd_elements_utils.py ... [ 52%]
tests/io/general/test_base.py ... [ 53%]
tests/io/general/test_data_segment.py .......... [ 55%]
tests/io/general/test_format_function.py ....... [ 56%]
tests/io/general/test_nitf_headers.py . [ 56%]
tests/io/general/test_tre.py .. [ 56%]
tests/io/phase_history/test_cphd.py ..... [ 57%]
tests/io/phase_history/test_cphd_versions.py ..... [ 58%]
tests/io/phase_history/cphd1_elements/test_cphd.py .... [ 59%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_antenna.py ..... [ 60%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_blocks.py .... [ 61%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_channel.py . [ 61%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_data.py . [ 61%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_dwell.py . [ 61%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_geoinfo.py ... [ 62%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_global.py .. [ 62%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_pvp.py . [ 62%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_referencegeometry.py . [ 62%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_supportarray.py .. [ 63%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_txrcv.py . [ 63%]
tests/io/phase_history/cphd1_elements/test_utils.py . [ 63%]
tests/io/product/test_reader.py ..s [ 64%]
tests/io/product/test_sidd_schema.py ......... [ 65%]
tests/io/product/test_sidd_writing.py . [ 66%]
tests/io/product/sidd3_elements/test_exploitationfeatures.py ............................................................................................... [ 83%]
................................... [ 90%]
tests/io/product/sidd3_elements/test_sidd3_elements.py ..... [ 90%]
tests/io/received/test_crsd.py . [ 91%]
tests/processing/sicd/test_spectral_taper.py ........ss........... [ 95%]
tests/utils/test_sicd_sidelobe_control.py .. [ 95%]
tests/visualization/test_cphd_kmz_product_creation.py .. [ 95%]
tests/visualization/test_crsd_kmz_product_creation.py . [ 95%]
tests/visualization/test_kmz_product_creation.py .x [ 96%]
tests/visualization/test_remap.py .................... [100%]
========================================================================= warnings summary =========================================================================
tests/geometry/test_point_projection.py::test_image_to_ground_errors
tests/geometry/test_point_projection.py::test_image_to_ground_dem
/home/vscuser/git/glweb/software/third-party/sarpy/sarpy/geometry/point_projection.py:1903: RuntimeWarning: divide by zero encountered in double_scalars
lon_grid_size = min(10, lat_grid_size/abs(numpy.sin(numpy.deg2rad(ref_llh[0]))))
tests/visualization/test_kmz_product_creation.py::test_create_kmz[True]
/home/vscuser/git/glweb/software/third-party/sarpy/sarpy/visualization/kmz_utils.py:169: RuntimeWarning: invalid value encountered in double_scalars
+ dir_z**2 * semiaxis_a**2 * semiaxis_b**2
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================== 528 passed, 14 skipped, 1 xfailed, 3 warnings in 39.28s ======================================================
nox > Session test(version='3.6') was successful.
nox > Running session test(version='3.11')
nox > Creating conda env in .nox/test-version-3-11 with python
nox > conda install --yes --prefix /home/vscuser/git/glweb/software/third-party/sarpy/.nox/test-version-3-11 python=3.11
nox > python -m pip install '.[all]'
nox > pytest tests
======================================================================= test session starts ========================================================================
platform linux -- Python 3.11.8, pytest-8.0.2, pluggy-1.4.0
rootdir: /home/vscuser/git/glweb/software/third-party/sarpy
collected 543 items
tests/consistency/test_consistency.py ........ [ 1%]
tests/consistency/test_cphd_consistency.py .............................................................. [ 12%]
tests/consistency/test_sicd_consistency.py ... [ 13%]
tests/consistency/test_sidd_consistency.py .... [ 14%]
tests/geometry/test_geocoords.py .......... [ 16%]
tests/geometry/test_geometry_elements.py .......... [ 17%]
tests/geometry/test_latlon.py ... [ 18%]
tests/geometry/test_point_projection.py ............. [ 20%]
tests/io/DEM/test_geoid.py . [ 20%]
tests/io/DEM/test_geotiff1deg_list.py .... [ 21%]
tests/io/DEM/test_geotiff1deg_reader.py ........ [ 23%]
tests/io/complex/sicd_elements/test_sicd_elements_antenna.py ... [ 23%]
tests/io/complex/sicd_elements/test_sicd_elements_base.py ... [ 24%]
tests/io/complex/sicd_elements/test_sicd_elements_blocks.py ..................... [ 28%]
tests/io/complex/sicd_elements/test_sicd_elements_collectioninfo.py .. [ 28%]
tests/io/complex/sicd_elements/test_sicd_elements_errorstatistics.py . [ 28%]
tests/io/complex/sicd_elements/test_sicd_elements_geodata.py ... [ 29%]
tests/io/complex/sicd_elements/test_sicd_elements_grid.py .. [ 29%]
tests/io/complex/sicd_elements/test_sicd_elements_imagecreation.py . [ 29%]
tests/io/complex/sicd_elements/test_sicd_elements_imagedata.py . [ 30%]
tests/io/complex/sicd_elements/test_sicd_elements_imageformation.py ...... [ 31%]
tests/io/complex/sicd_elements/test_sicd_elements_matchinfo.py . [ 31%]
tests/io/complex/sicd_elements/test_sicd_elements_pfa.py . [ 31%]
tests/io/complex/sicd_elements/test_sicd_elements_position.py ... [ 32%]
tests/io/complex/sicd_elements/test_sicd_elements_radarcollection.py .............. [ 34%]
tests/io/complex/sicd_elements/test_sicd_elements_radiometric.py . [ 34%]
tests/io/complex/sicd_elements/test_sicd_elements_rgazcomp.py . [ 34%]
tests/io/complex/sicd_elements/test_sicd_elements_rma.py ... [ 35%]
tests/io/complex/sicd_elements/test_sicd_elements_scpcoa.py ................... [ 39%]
tests/io/complex/sicd_elements/test_sicd_elements_sicd.py .................................. [ 45%]
tests/io/complex/sicd_elements/test_sicd_elements_timeline.py .................. [ 48%]
tests/io/complex/sicd_elements/test_sicd_elements_utils.py ... [ 49%]
tests/io/complex/test_other_nitf.py ... [ 49%]
tests/io/complex/test_reader.py .ssssssssss [ 51%]
tests/io/complex/test_remote.py s [ 51%]
tests/io/complex/test_sicd.py . [ 52%]
tests/io/complex/test_utils.py . [ 52%]
tests/io/general/test_base.py ... [ 52%]
tests/io/general/test_data_segment.py .......... [ 54%]
tests/io/general/test_format_function.py ....... [ 55%]
tests/io/general/test_nitf_headers.py . [ 56%]
tests/io/general/test_tre.py .. [ 56%]
tests/io/phase_history/cphd1_elements/test_cphd.py .... [ 57%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_antenna.py ..... [ 58%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_blocks.py .... [ 58%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_channel.py . [ 59%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_data.py . [ 59%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_dwell.py . [ 59%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_geoinfo.py ... [ 60%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_global.py .. [ 60%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_pvp.py . [ 60%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_referencegeometry.py . [ 60%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_supportarray.py .. [ 61%]
tests/io/phase_history/cphd1_elements/test_cphd1_elements_txrcv.py . [ 61%]
tests/io/phase_history/cphd1_elements/test_utils.py . [ 61%]
tests/io/phase_history/test_cphd.py ..... [ 62%]
tests/io/phase_history/test_cphd_versions.py ..... [ 63%]
tests/io/product/sidd3_elements/test_exploitationfeatures.py ............................................................................................... [ 80%]
................................... [ 87%]
tests/io/product/sidd3_elements/test_sidd3_elements.py ..... [ 88%]
tests/io/product/test_reader.py ..s [ 88%]
tests/io/product/test_sidd_schema.py ......... [ 90%]
tests/io/product/test_sidd_writing.py . [ 90%]
tests/io/received/test_crsd.py . [ 90%]
tests/io/test_kml.py . [ 90%]
tests/processing/sicd/test_spectral_taper.py ..................... [ 94%]
tests/test_class_string.py . [ 95%]
tests/utils/test_sicd_sidelobe_control.py .. [ 95%]
tests/visualization/test_cphd_kmz_product_creation.py .. [ 95%]
tests/visualization/test_crsd_kmz_product_creation.py . [ 95%]
tests/visualization/test_kmz_product_creation.py .x [ 96%]
tests/visualization/test_remap.py .................... [100%]
========================================================================= warnings summary =========================================================================
tests/consistency/test_sidd_consistency.py: 4 warnings
tests/visualization/test_remap.py: 13 warnings
/home/vscuser/git/glweb/software/third-party/sarpy/sarpy/visualization/remap.py:1751: MatplotlibDeprecationWarning: The get_cmap function was deprecated in Matplo
tlib 3.7 and will be removed two minor releases later. Use ``matplotlib.colormaps[name]`` or ``matplotlib.colormaps.get_cmap(obj)`` instead. cmap = cm.get_cmap(value, max_out_size+1)
tests/geometry/test_point_projection.py::test_image_to_ground_errors
tests/geometry/test_point_projection.py::test_image_to_ground_dem
/home/vscuser/git/glweb/software/third-party/sarpy/sarpy/geometry/point_projection.py:1903: RuntimeWarning: divide by zero encountered in scalar divide
lon_grid_size = min(10, lat_grid_size/abs(numpy.sin(numpy.deg2rad(ref_llh[0]))))
tests/test_class_string.py::TestClassString::test_class_str
/home/vscuser/git/glweb/software/third-party/sarpy/sarpy/annotation/afrl_rde_schema/__init__.py:7: DeprecationWarning: pkg_resources is deprecated as an API. See
https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources
tests/visualization/test_cphd_kmz_product_creation.py::test_create_kmz
tests/visualization/test_cphd_kmz_product_creation.py::test_create_kmz
tests/visualization/test_cphd_kmz_product_creation.py::test_create_kmz
tests/visualization/test_cphd_kmz_product_creation.py::test_create_kmz
tests/visualization/test_cphd_kmz_product_creation.py::test_create_kmz
tests/visualization/test_cphd_kmz_product_creation.py::test_create_kmz
/home/vscuser/git/glweb/software/third-party/sarpy/sarpy/visualization/kmz_utils.py:302: MatplotlibDeprecationWarning: The collections attribute was deprecated in
Matplotlib 3.8 and will be removed two minor releases later. for path in contour_sets.collections[0].get_paths()
tests/visualization/test_kmz_product_creation.py::test_create_kmz[True]
/home/vscuser/git/glweb/software/third-party/sarpy/sarpy/visualization/kmz_utils.py:145: RuntimeWarning: invalid value encountered in scalar divide
distance = (
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================== 530 passed, 12 skipped, 1 xfailed, 27 warnings in 31.14s =====================================================
nox > Session test(version='3.11') was successful.
nox > Ran multiple sessions:
nox > * test(version='3.6'): success
nox > * test(version='3.11'): success
|
I do plan to do some more TREs (probably not much that is SAR specific / relevant). Is the "smoke test" change (1088c90) something that you'd like followed as a pattern? |
Yes please, if an example dataset is publicly/easily available and the test data stays on the order of the sizes of other repo files then I think they'll prove to be useful demo/review aids. There have been a handful of bugs, typos, etc. (#486, #487 semi-recently) that likely would've benefitted from such tests and the presence of test data should make any future bugfixes/changes easier in the future. |
|
I've modified the data extraction code to be generic, in support of using the same data source to produce examples of other TREs. |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement MATESA TRE, per STDI-0002 Appendix AK.
See https://nsgreg.nga.mil/doc/view?i=5516 for a sample of this.