Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Healpix requirement for easy cluster management #143

Open
nicholebarry opened this issue Sep 4, 2020 · 3 comments
Open

Remove Healpix requirement for easy cluster management #143

nicholebarry opened this issue Sep 4, 2020 · 3 comments

Comments

@nicholebarry
Copy link
Contributor

When refreshing the info file, the Healpix cubes are required, even if the uvf_cubes exist. It would be best to extract the information from the Healpix cubes into the uvf_cubes so that the Healpix cubes don't need to be downloaded for aws users.

Example: Mike downloaded the uvf cubes from aws to a local computer. He then tried to refresh_info.

IDL> ps_wrapper, "/Volumes/Faramir/2014_GS/2014_PS/GS_freq_broadcast_dgpb", "Combined_obs_2014_GS_plus_one", /refresh_info, /refresh_beam, /png
% PS_FILENAMES: No cube or info files found in folder /Volumes/Faramir/2014_GS/2014_PS/GS_freq_broadcast_dgpb
% Execution halted at: PS_FILENAMES 1201 /Users/mikewilensky/Repos/eppsilon/ps_wrappers/ps_filenames.pro
% PS_WRAPPER 122

@bhazelton
Copy link
Member

So all the required information from the HEALPix files is extracted and saved in the info file. So the solution is to not have to refresh info. Why was that needed?

@nicholebarry
Copy link
Contributor Author

I copied the uvf cubes and the info file from the cluster onto my local computer, and tried to run eppsilon to see if the refresh_info keyword was needed or not.

IDL> ps_wrapper, '/Users/nabarry/MWA/data/fhd_nb_test/','1095954144',/full_image,/png
datafile = /Users/nabarry/MWA/data/fhd_nb_test/ps/1095954144_059-078_gridded_uvf__even_odd_joint_uvf_info.idlsave
file setup time: 0.010020971
n_vis difference between even & odd cubes: 0
n_vis % difference between even & odd cubes: 0
n_obs:            1           1
getvar_savefile: file /fred/oz048/MWA/CODE/FHD/fhd_nb_gaussbeam_1fits_ultralow2/1095954144_059-078_even_gridded_uvf.sav not found
% PS_IMAGE_TO_UVF: Error getting data out of file: /fred/oz048/MWA/CODE/FHD/fhd_nb_gaussbeam_1fits_ultralow2/1095954144_059-078_even_gridded_uvf.sav If the file name has changed,
                    set "/refresh_info" in the ps_wrapper call.
% Execution halted at: PS_IMAGE_TO_UVF   254 /Users/nabarry/MWA/eppsilon/ps_core/ps_image_to_uvf.pro
%                      PS_KCUBE          163 /Users/nabarry/MWA/eppsilon/ps_core/ps_kcube.pro
%                      PS_POWER           20 /Users/nabarry/MWA/eppsilon/ps_core/ps_power.pro
%                      PS_MAIN_PLOTS     272 /Users/nabarry/MWA/eppsilon/ps_core/ps_main_plots.pro
%                      PS_WRAPPER        304 /Users/nabarry/MWA/eppsilon/ps_wrappers/ps_wrapper.pro
%                      $MAIN$          
IDL> ps_wrapper, '/Users/nabarry/MWA/data/fhd_nb_test/','1095954144',/full_image,/png,/refresh_info
bash: hostname: command not found
% PS_FILENAMES: No cube or info files found in folder /Users/nabarry/MWA/data/fhd_nb_test
% Execution halted at: PS_FILENAMES     1201 /Users/nabarry/MWA/eppsilon/ps_wrappers/ps_filenames.pro
%                      PS_WRAPPER        123 /Users/nabarry/MWA/eppsilon/ps_wrappers/ps_wrapper.pro
%                      PS_IMAGE_TO_UVF   254 /Users/nabarry/MWA/eppsilon/ps_core/ps_image_to_uvf.pro
%                      PS_KCUBE          163 /Users/nabarry/MWA/eppsilon/ps_core/ps_kcube.pro
%                      PS_POWER           20 /Users/nabarry/MWA/eppsilon/ps_core/ps_power.pro
%                      PS_MAIN_PLOTS     272 /Users/nabarry/MWA/eppsilon/ps_core/ps_main_plots.pro
%                      PS_WRAPPER        304 /Users/nabarry/MWA/eppsilon/ps_wrappers/ps_wrapper.pro
%                      $MAIN$          
IDL> 

@bhazelton
Copy link
Member

bhazelton commented Nov 20, 2020

So I just ran into this and I think I understand the confusion better now. The issue (at least in my case) is that in addition to the uvf_cubes, eppsilon also needs the beam_cubes, which are stored in the ps folder under data/beam_cubes. If those are present, the info file should not need to be refreshed and the Healpix cubes should not be needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants