Commissioning extraction

CHANGES.md

@@ -1,3 +1,22 @@
+0.9.0 (2024-04-02)
+------------------
+- Fixes based on Joey's comments
+- Deployment fixes
+- We now don't keep the filepath of the standards in the db. We assume they are in the archive
+  or are in the install director
+
+0.8.0 (2024-03-18)
+------------------
+- Increased the memory limit on the containers to accomodate stacking
+- Simplified the triggering flat stacking to make it more testable
+
+0.7.0
+-----
+- Deployment fixes
+0.6.0
+-----
+- Deployment fixes
+
 0.5.0 (2023-11-03)
 ------------------
 - Helm value fixes to get the pipeline scheduled on nodes.

README.rst

@@ -5,6 +5,11 @@ BANZAI Data Reduction for FLOYDS spectra
     :target: http://www.astropy.org
     :alt: Powered by Astropy Badge
 
+Installation
+------------
+```
+pip install .
+```
 
 License
 -------

banzai_floyds/data/extinction.dat

@@ -0,0 +1,221 @@
+# wavelength mag
+3005.00  2.50872
+3030.00  2.20848
+3055.00  1.94126
+3080.00  1.70745
+3105.00  1.51826
+3130.00  1.35833
+3155.00  1.15734
+3180.00  1.01678
+3205.00  0.91633
+3230.00  0.84665
+3255.00  0.79306
+3280.00  0.75106
+3305.00  0.71229
+3330.00  0.67891
+3355.00  0.65186
+3380.00  0.63322
+3405.00  0.61623
+3430.00  0.60518
+3455.00  0.58919
+3480.00  0.56207
+3505.00  0.53454
+3530.00  0.51535
+3555.00  0.50268
+3580.00  0.49307
+3605.00  0.48353
+3630.00  0.47173
+3655.00  0.45890
+3680.00  0.44605
+3705.00  0.43449
+3730.00  0.42465
+3755.00  0.41634
+3780.00  0.40872
+3805.00  0.40099
+3830.00  0.39270
+3855.00  0.38369
+3880.00  0.37369
+3905.00  0.36302
+3930.00  0.35140
+3955.00  0.33946
+3980.00  0.32795
+4005.00  0.31757
+4030.00  0.30859
+4055.00  0.30068
+4080.00  0.29355
+4105.00  0.28660
+4130.00  0.27997
+4155.00  0.27339
+4180.00  0.26698
+4205.00  0.26117
+4230.00  0.25566
+4255.00  0.25045
+4280.00  0.24554
+4305.00  0.24051
+4330.00  0.23565
+4355.00  0.23053
+4380.00  0.22558
+4405.00  0.22064
+4430.00  0.21560
+4455.00  0.21084
+4480.00  0.20610
+4505.00  0.20165
+4530.00  0.19748
+4555.00  0.19345
+4580.00  0.18969
+4605.00  0.18608
+4630.00  0.18247
+4655.00  0.17901
+4680.00  0.17581
+4705.00  0.17288
+4730.00  0.17021
+4755.00  0.16780
+4780.00  0.16540
+4805.00  0.16300
+4830.00  0.16060
+4855.00  0.15834
+4880.00  0.15608
+4905.00  0.15395
+4930.00  0.15208
+4955.00  0.15045
+4980.00  0.14883
+5005.00  0.14746
+5030.00  0.14622
+5055.00  0.14498
+5080.00  0.14386
+5105.00  0.14275
+5130.00  0.14163
+5155.00  0.14065
+5180.00  0.13953
+5205.00  0.13842
+5230.00  0.13719
+5255.00  0.13608
+5280.00  0.13485
+5305.00  0.13362
+5330.00  0.13240
+5355.00  0.13129
+5380.00  0.13031
+5405.00  0.12933
+5430.00  0.12836
+5455.00  0.12762
+5480.00  0.12689
+5505.00  0.12628
+5530.00  0.12567
+5555.00  0.12518
+5580.00  0.12482
+5605.00  0.12445
+5630.00  0.12409
+5655.00  0.12372
+5680.00  0.12336
+5705.00  0.12299
+5730.00  0.12251
+5755.00  0.12190
+5780.00  0.12105
+5805.00  0.12020
+5830.00  0.11899
+5855.00  0.11766
+5880.00  0.11608
+5905.00  0.11427
+5930.00  0.11222
+5955.00  0.10994
+5980.00  0.10754
+6005.00  0.10490
+6030.00  0.10204
+6055.00  0.09906
+6080.00  0.09609
+6105.00  0.09325
+6130.00  0.09065
+6155.00  0.08817
+6180.00  0.08606
+6220.00  0.08781
+6290.00  0.08196
+6365.00  0.07725
+6436.00  0.07623
+6473.00  0.07570
+6530.00  0.07488
+6600.00  0.07388
+6670.00  0.07287
+6740.00  0.07187
+6790.00  0.07115
+6850.00  0.06914
+6975.00  0.06484
+7055.00  0.06208
+7100.00  0.06054
+7150.00  0.05882
+7220.00  0.05641
+7270.00  0.05539
+7365.00  0.05455
+7460.00  0.05370
+7550.00  0.05291
+7580.00  0.05264
+7725.00  0.05191
+7780.00  0.05235
+7820.00  0.05268
+7910.00  0.05341
+8000.00  0.05414
+8090.00  0.05487
+8210.00  0.05404
+8260.00  0.05311
+8370.00  0.05106
+8405.00  0.04806
+8430.00  0.04840
+8455.00  0.04829
+8480.00  0.04783
+8505.00  0.04704
+8530.00  0.04613
+8555.00  0.04523
+8580.00  0.04455
+8605.00  0.04410
+8630.00  0.04398
+8655.00  0.04443
+8680.00  0.04511
+10005.00  0.04466
+10030.00  0.04568
+10055.00  0.04670
+10080.00  0.04749
+10105.00  0.04829
+10130.00  0.04897
+10155.00  0.04965
+10180.00  0.05011
+10205.00  0.05056
+10230.00  0.05102
+10255.00  0.05124
+10280.00  0.05147
+10305.00  0.05170
+10330.00  0.05181
+10355.00  0.05181
+10380.00  0.05181
+10405.00  0.05170
+10430.00  0.05158
+10455.00  0.05147
+10480.00  0.05124
+10505.00  0.05102
+10530.00  0.05067
+10555.00  0.05033
+10580.00  0.04988
+10605.00  0.04954
+10630.00  0.04908
+10655.00  0.04851
+10680.00  0.04806
+10705.00  0.04749
+10730.00  0.04693
+10755.00  0.04636
+10780.00  0.04579
+10805.00  0.04511
+10830.00  0.04455
+10855.00  0.04387
+10880.00  0.04330
+10905.00  0.04263
+10930.00  0.04195
+10955.00  0.04139
+10980.00  0.04071
+11005.00  0.04003
+11030.00  0.03947
+11055.00  0.03891
+11080.00  0.03823
+11105.00  0.03767
+11130.00  0.03711
+11155.00  0.03655
+11180.00  0.03610
+11205.00  0.03565
+11230.00  0.03509

banzai_floyds/data/standards/README.md

@@ -0,0 +1,41 @@
+Relevant links:
+https://www.eso.org/sci/observing/tools/standards/spectra/stanlis.html
+https://ftp.eso.org/pub/usg/standards/ctiostan/
+https://ftp.eso.org/pub/stecf/standards/okestan/
+
+We prioritize the HST standards, followed by the X-Shooter standards, followed by the traditional CTIO observations.
+
+To generate the standard files in the pipeline format, download them from the ESO ftp servers and run the following:
+
+```
+from astropy.io import fits, ascii
+from astropy.table import Table
+
+
+standards = [{'name': 'gd108', 'input_file': 'fgd108.dat', 'ra': 150.196859, 'dec': -7.558548},
+             {'name': 'eg274', 'input_file': 'fEG274.dat', 'ra': 245.890989, 'dec': -39.229487},
+             {'name': 'feige110', 'input_file': 'ffeige110.dat', 'ra': 349.99332558, 'dec': -5.16560011},
+             {'name': 'feige34', 'input_file': 'ffeige34.dat', 'ra': 159.903066, 'dec': 43.102559},
+             {'name': 'bdp284211', 'input_file': 'fbd28d4211.dat', 'ra': 327.795923, 'dec': 28.863988}]
+
+for standard in standards:
+    eso_data = ascii.read(standard['input_file'])
+    data = Table({'wavelength': eso_data['col1'], 'flux': eso_data['col2']})
+    hdu_list = fits.HDUList([fits.PrimaryHDU(header=fits.Header({'RA': standard['ra'], 'DEC': standard['dec'], 
+                                                                 'OBSTYPE': 'fluxstandard'})), 
+                             fits.BinTableHDU(data)])
+    hdu_list.writeto(f'{standard["name"]}.fits', overwrite=True)
+```
+Note that we use the same units as ESO: $\frac{ergs}{s \cdot cm^2 \cdot \unicode{x212B}}$.
+
+I have currently omitted L745-46A as a standard because it was not available through ESO and appears to not have been calibrated
+since 1984 (Baldwin & Stone). We should bootstrap this fluxed file based on standards observed on the same night with FLOYDS. LTT 3218 is a decent candidate to cross calibrate with. EG21 was reproduced in Hamuy 1994 but at very low resolution. The fluxes from Hamuy also do not look telluric corrected to me. We should probably use GD50 to bootstrap the EG21. Feige 67 will need to be bootstrapped from the HST CALSPEC spectrum of GD153. HZ44 will also need to be derived from its observations in CALSPEC + a model for minor extrapolation. Similarly for G191-B2B.
+
+Calibration Source:
+| Object | Source                                                    |
+| ------ | --------------------------------------------------------- |
+| GD108  | ftp://ftp.eso.org/pub/stecf/standards/hststan/fgd108.dat  |
+| EG274  | ftp://ftp.eso.org/pub/stecf/standards/Xshooter/fEG274.dat |
+| FEIGE110 | ftp://ftp.eso.org/pub/stecf/standards/hststan/ffeige110.dat |
+| FEIGE34 | ftp://ftp.eso.org/pub/stecf/standards/hststan/ffeige110.dat |
+| BD+28-4211 | ftp://ftp.eso.org/pub/stecf/standards/hststan/fbd28d4211.dat |