[WIP] Add optional near-field corrections to the phasing function #1482

baron-de-montblanc · 2024-10-08T23:35:55Z

Description

Included code to perform near-field corrections through keyword arguments passed to the phase() function.

I added two new keys to the phase_dict dictionary (cat_dist_units and cat_near_field), allowing the user to specify the units associated with the cat_dist key, and whether or not to apply near-field corrections (boolean value). As a result, a lot of tests are breaking because these extra keys didn't exist when they were written. I'm having trouble tracking down every instance of phase_dict to patch this.

It also still isn't clear whether or not an unproject_near_field() function is necessary, so I have left in some commented-out lines of code while I work on figuring this out (to be removed or uncommented when this is resolved).

Motivation and Context

This feature allows users to specify the RA, DEC, and distance to a near-field source, and "focus" on that source (it will then appear in focus in an image, for example). This is done in practice by calculating the geometric near-field w values for each visibility, updating those in the uvw_array, computing the resulting phase differences, and updating the phase directly for each visibility. This feature is turned off by default (i.e. the default behavior for the phase() function is just to perform the traditional far-field phasing).

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation change (documentation changes only)
Version change
Build or continuous integration change

Checklist:

I have read the contribution guide.
My code follows the code style of this project.

New feature checklist:

I have added or updated the docstrings associated with my feature using the numpy docstring format.
I have updated the tutorial to highlight my new feature (if appropriate).
I have added tests to cover my new feature.
All new and existing tests pass.
I have updated the CHANGELOG.

bhazelton · 2024-10-29T15:29:55Z

Rather than adding new phase_center_dict keys, let's use a new cat type for near-field phased objects, maybe `"near_field"? Then encourage and support passing a Quantity for distance, but if a scalar is passed interpret it as in meters for near field and parsecs for ephem cat types. Update the docs to be very clear about this.

github-actions

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite	Current: `ba8783f`	Previous: `bca6cf0`	Ratio
`tests/utils/test_bls.py::test_ants_to_bls[min=4259840-len=10]`	`215140.27267544658` iter/sec (`stddev: 0.000005969824238735253`)	`434661.1807555354` iter/sec (`stddev: 0.0000013012435924847232`)	`2.02`

This comment was automatically generated by workflow using github-action-benchmark.

codecov · 2025-01-10T23:50:04Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.93%. Comparing base (c62e1e2) to head (87b64e4).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1482   +/-   ##
=======================================
  Coverage   99.93%   99.93%           
=======================================
  Files          63       63           
  Lines       21852    21943   +91     
=======================================
+ Hits        21837    21928   +91     
  Misses         15       15

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…o be passed as an astropy.units.Quantity object.

…. This will first apply sidereal far-field phasing, then apply near-field corrections.

…ally unprojects the near field corrections.

…ed data to file formats other than uvh5. Also added more code coverage.

…iriad and ms file formats on windows devices

bhazelton

This is looking really good! Just a few small comments.

bhazelton · 2025-01-24T18:11:55Z

src/pyuvdata/utils/phasing.py

+    Return the x,y,z coordinates of the focal point.
+
+    The focal point corresponds to the location of
+    the NEAR-FIELD object of interest in the MWA-centred


This isn't specific to MWA, maybe "in the telescope centered ENU frame"?

bhazelton · 2025-01-24T18:14:34Z

src/pyuvdata/utils/phasing.py

+    # The centre of the ENU frame should be located at the MEDIAN position of the array
+    loc = uvd.telescope.location.itrs.cartesian.xyz.value
+    antpos = uvd.telescope.antenna_positions + loc
+    x, y, z = np.median(antpos, axis=0)


This is interesting and not what I expected from the docstring. From the docstring I'd assumed you were using the nominal telescope location (uvd.telescope.location) for the center of the ENU frame. I think this makes sense, but it'd be good to update the docstring to say that it's centered at the median position of the antennas.

bhazelton · 2025-01-24T18:14:48Z

src/pyuvdata/utils/phasing.py

+    antpos = uvd.telescope.antenna_positions + loc
+    x, y, z = np.median(antpos, axis=0)
+
+    # Initialize EarthLocation object centred on MWA


again, not MWA specific

bhazelton · 2025-01-24T18:15:02Z

src/pyuvdata/utils/phasing.py

+    # Initialize EarthLocation object centred on MWA
+    mwa = EarthLocation(x, y, z, unit=units.m)
+
+    # Convert sky object to an AltAz frame centred on the MWA


bhazelton · 2025-01-24T18:17:56Z

src/pyuvdata/utils/phasing.py

+    # Antenna positions in ENU frame
+    antpos = uvd.telescope.get_enu_antpos() - np.median(
+        uvd.telescope.get_enu_antpos(), axis=0
+    )


This ENU frame is different than the one you used in _get_focus_xyz because it's centered on the telescope location not on the median antenna position. Is that what you want?

bhazelton · 2025-01-25T00:31:31Z

src/pyuvdata/uvdata/uvdata.py

-            elif not ((phase_dict[key] is None) or isinstance(phase_dict[key], str)):
-                phase_dict[key] = float(phase_dict[key])
-


Why did you remove these lines?

bhazelton · 2025-01-25T00:35:28Z

tests/uvdata/test_uvfits.py

+# @pytest.mark.filterwarnings("ignore:The uvw_array does not match the expected values")
+# @pytest.mark.parametrize(
+#     "cat_type",
+#     ["sidereal", "near_field"]
+# )
+# def test_readwriteread_phase(tmp_path, casa_uvfits, cat_type):
+#     """
+#     CASA tutorial uvfits loopback test including phasing corrections.
+#     """
+#     uv_in = casa_uvfits
+
+#     uv_out = UVData()
+#     write_file = str(tmp_path / "outtest_casa_phased.uvfits")
+
+#     uv_in.phase(cat_name='foo', cat_type=cat_type, ra=0.8, dec=-0.4, dist=10000)
+
+#     uv_in.write_uvfits(write_file)
+#     file_read = write_file
+#     uv_out.read(file_read)
+
+#     assert uv_in.telescope._location.frame == uv_out.telescope._location.frame
+#     assert uv_in.telescope._location.ellipsoid == uv_out.telescope._location.ellipsoid
+
+#     uv_in.filename = uv_out.filename
+#     uv_out._consolidate_phase_center_catalogs(
+#         reference_catalog=uv_in.phase_center_catalog,
+#     )
+
+#     assert uv_in == uv_out
+#     return
+
+


Let's remove this. If there's a good reason we might need it in the future it can go into an issue.

baron-de-montblanc added the help wanted label Oct 8, 2024

bhazelton added the UVData label Oct 10, 2024

baron-de-montblanc force-pushed the near_field branch from f89c9cc to 87f6fb3 Compare October 15, 2024 16:02

baron-de-montblanc force-pushed the near_field branch from 87f6fb3 to ba8783f Compare January 7, 2025 17:17

github-actions bot reviewed Jan 7, 2025

View reviewed changes

baron-de-montblanc force-pushed the near_field branch from ba8783f to 8819458 Compare January 10, 2025 18:10

baron-de-montblanc force-pushed the near_field branch from 8819458 to 83966fe Compare January 16, 2025 20:00

[email protected] and others added 8 commits January 23, 2025 18:23

Included code to perform near-field corrections.

52dc20c

Fixed an indentation error.

8434561

Added a function to unproject near-field corrections and allow dist t…

7c63409

…o be passed as an astropy.units.Quantity object.

Now correctly references the Telescope object

a13de68

near_field can now be specified as the cat_type in the phasing method…

0fa27c3

…. This will first apply sidereal far-field phasing, then apply near-field corrections.

Removed unproject_near_field function since unproject_phase automatic…

85a5798

…ally unprojects the near field corrections.

Added unit tests for near-field phasing corrections.

944420d

Added NotImplementedError when trying to write out near-field correct…

9df7658

…ed data to file formats other than uvh5. Also added more code coverage.

baron-de-montblanc force-pushed the near_field branch from 83966fe to 9df7658 Compare January 23, 2025 23:23

Restructured the test_write_near_field_err test to skip writing out m…

87b64e4

…iriad and ms file formats on windows devices

bhazelton reviewed Jan 25, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Add optional near-field corrections to the phasing function #1482

[WIP] Add optional near-field corrections to the phasing function #1482

baron-de-montblanc commented Oct 8, 2024

bhazelton commented Oct 29, 2024

github-actions bot left a comment •

edited

Loading

codecov bot commented Jan 10, 2025 •

edited

Loading

bhazelton left a comment

bhazelton Jan 24, 2025

bhazelton Jan 24, 2025

bhazelton Jan 24, 2025

bhazelton Jan 24, 2025

bhazelton Jan 24, 2025

bhazelton Jan 25, 2025

bhazelton Jan 25, 2025

		elif not ((phase_dict[key] is None) or isinstance(phase_dict[key], str)):
		phase_dict[key] = float(phase_dict[key])

[WIP] Add optional near-field corrections to the phasing function #1482

Are you sure you want to change the base?

[WIP] Add optional near-field corrections to the phasing function #1482

Conversation

baron-de-montblanc commented Oct 8, 2024

Description

Motivation and Context

Types of changes

Checklist:

bhazelton commented Oct 29, 2024

github-actions bot left a comment • edited Loading

Choose a reason for hiding this comment

⚠️ Performance Alert ⚠️

codecov bot commented Jan 10, 2025 • edited Loading

Codecov Report

bhazelton left a comment

Choose a reason for hiding this comment

bhazelton Jan 24, 2025

Choose a reason for hiding this comment

bhazelton Jan 24, 2025

Choose a reason for hiding this comment

bhazelton Jan 24, 2025

Choose a reason for hiding this comment

bhazelton Jan 24, 2025

Choose a reason for hiding this comment

bhazelton Jan 24, 2025

Choose a reason for hiding this comment

bhazelton Jan 25, 2025

Choose a reason for hiding this comment

bhazelton Jan 25, 2025

Choose a reason for hiding this comment

github-actions bot left a comment •

edited

Loading

codecov bot commented Jan 10, 2025 •

edited

Loading