Skip to content

Reduction Reference

Jump to bottom
Epzack edited this page Sep 20, 2021 · 14 revisions

Below is a list of all functions that are part of the reduction module, grouped by filename.

Preprocess

  • preproc.py - preprocessing functions.
    • choose_calib - calibration images visualization and selection.
    • choose_science - display science images for verification by user.
    • mkmaster - make master calibration frames (bias, dark, flat).

Main Reduction

Reduction Steps

  • photopipe/reduction/steps - reduction processing steps.

Prepare

  • autopipedefaults - set commonly used variables to use throughout each step.
  • autopipeprepare - update image headers and performs bias/dark subtraction.
    • pipeprepare - Normalize header keywords in FITS files.
    • find_sats - Creates a binary image detailing saturated pixels.

Flatten

  • autopipeimflatten - flatten data using flat with matching filter name.
    • flatpipeproc - Check if flat is same size as data, then divide for correct filter.

Make Sky

Sky Subtraction

  • autopipeskysub - subtract both master sky and median.
    • skypipeproc - Subtract sky flat from data, then subtract median of that from remaining data.
  • autopipeskysubmed - subtract median, does NOT use master sky.

Cosmic Ray Cleaning

  • autopipecrcleanim - remove cosmic rays.
    • cosmiczap - Remove cosmic rays using Laplacian cosmic ray identification written in cosmics.py.

Astrometry

  • autopipeastrometry - calculate astrometry of image files to fix WCS coordinates.
    • astrometry - Run sextractor and scamp to refine astrometric solution.

Zero Point

Stacking

Automatic Processing Dependencies

  • autoproc_depend.py - reduction processing dependencies.
    • findcals - Locates calibration files.
    • write_fits - Creates a FITS file.
    • findsexobj - Find sextractor objects with optional inputs. Estimates seeing from stars found.
    • calc_zpt - Find zeropoint using robust scatter.
    • robust_scat - Calculate robust scatter and set the weight of those above this limit to 0.
    • identify_matches - Use a kd-tree (3d) to match two lists of stars, using full spherical coordinate distances.

Step Functions

pipeprepare

<> pipeprepare(filename, [outname=None, biasfile=None, darkfile=None, verbose=1])

Adds additional header keywords needed later on in the pipeline and removes unnecessary header keywords by looking through a list of mandatory keywords. Also runs bias and dark subtraction for filters with an existing master bias/dark (CCDs). The prepared images are written to disk with outname.

Params

  • filename {str or list of str}: Absolute path to FITS file, or list of paths, or file w/list of paths.
  • outname {str, optional}: Specify output file to write to disk.
  • biasfile {str, optional}: Absolute path to bias file.
  • darkfile {str, optional}: Absolute path to dark file.
  • verbose {bool, optional}: Print out comments.

find_sats

find_sats(filename, data, header)

Uses 'SATURATE' keyword to determine which pixels in the original image are saturated. Creates/saves a binary image of saturated pixels (0: saturated, 1: unsaturated).

Params

  • filename {str}: Absolute path to FITS file.
  • data {list}: Data array from FITS file.
  • header {list}: Header information form FITS file

flatpipeproc

<> flatpipeproc(filename, flatname, [flatminval=0, flatmaxval=0])

Checks if flat is same size as data, then divides for correct filter.

Params

  • filename {str or list of str}: Absolute path to FITS file, or array of paths, or file w/list of paths.
  • flatname {str}: Absolute path to FITS master flat file.
  • flatminval {float, optional}: If not set to 0 below this value will set to NaNs.
  • flatmaxval {float, optional}: If not set to 0 below this value will set to NaNs.

call_skypipecombine

call_skypipecombine(files, outflatname, filt, pipevar)

Ensures at least two frames exist in the filter of the master sky about to be constructed. Calls skypipecombine to remove multiple instances of the call.

Params

  • files {str or list of str}: Absolute path to FITS file, or list of paths to be combined into a master sky
  • outflatname {str}: Name for output master flat fits file.
  • filt {str}: Absolute path to filter file.
  • pipevar {dict}: (Input pipeline parameters)[https://github.com/astroumd/photometrypipeline/wiki/autoproc_steps.py#pipeline-input-parameters].

skypipecombine

<> skypipecombine(filelist, outfile, filt, pipevar, [removeobjects=None, objthresh=6, algorithm='median', trimlo=None, trimhi=None, mincounts=1, maxcounts=55000, satlevel=30000, type=None])

Create sigma clipped median sky flat. Scales each file based on the overall sigma clipped median, then removes objects selected with sextractor (uses flux fraction radius) in each file. Removes saturated pixels. Calculates sigma clipped median of each pixel and saves anything with non-finite values (saturated or source) to the median of the entire frame. Save with outfile name.

Params

  • filename {str or list of str}: Absolute path to FITS file, or list of paths, or file w/list of paths.
  • outfile {str}: Name for output fits file.
  • filt {str}: Absolute path to filter file.
  • pipevar {dict}: (Input pipeline parameters)[https://github.com/astroumd/photometrypipeline/wiki/autoproc_steps.py#pipeline-input-parameters].
  • removeobjects {bool, optional}: Specifies if objects should be removed.
  • objthresh {float, optional}: Sets sigma in removeobjects (default is 6).
  • algorithm {str, optional}: Algorithm to solve (mean or median, default is median).
  • trimlo {bool, optional}: Trim off bottom of data in mean algorithm mode (default is 25%).
  • trimhi {bool, optional}: Trim off top of data in mean algorithm mode (default is 25%).
  • mincounts {int, optional}: Sets minimum counts allowed (default is 1).
  • maxcounts {int, optional}: Sets minimum counts allowed (default is 55000).
  • satlevel {int, optional}: Sets saturation level (default is 30000).
  • type {str, optional}: Sets SKYTYPE keyword in header of outfile to this string.

Dependencies

medclip, sextractor.

Example & Usage

See autoproc_steps.py at line 277.

skypipecombine( files[skyflats], 
                outflatname, 
                file,
                pipevar, 
                removeobjects=True, 
                type='sky' )

masked_interpolation

masked_interpolation(image, method='nearest')

Interpolates over masked locations

Params

  • image {np.array}: Array to interpolate over.
  • method {str,optional}: Interpolation method {‘linear’, ‘nearest’, ‘cubic’}

skypipeproc

<> skypipeproc(filename, flatname, outfile, [flatminval=None, flatmaxval=None])

Subtracts sky flat from data and then subtracts median of that from remaining data.

Params

  • filename {str or list of str}: Absolute path to FITS file, or list of paths to be sky subtracted.
  • flatname {str}: Absolute path to sky flat FITS file.
  • outfile {str}: Name for output fits file.
  • flatminval {float, optional}: Minimum required value in flat (default for SKYCTS calculation is 0.1).
  • flatmaxval {float, optional}: Maximum required value in flat.

cosmiczap

<> cosmiczap(filename, outname, [sigclip=6.0, maxiter=3, verbose=True])

Removes cosmic rays using Laplacian cosmic ray identification written in cosmics.py.

Params

  • filename {str or list of str}: Absolute path to FITS file, or list of paths to be cosmic ray zapped.
  • outfile {str}: Name for output fits file.
  • sigclip {float, optional}: Sigma to clip.
  • maxiter {int, optional}: Maximum number of times to iterate loop.
  • verbose {bool, optional}: Print out comments.

Example & Usage

See autoproc_steps.py at line 484.

cosmiczap( file, 
           outfile, 
           sigclip=6.0, 
           maxiter=1, 
           verbose=pipevar['verbose'] )

astrometry

<> astrometry(atfimages, [scamprun=1, pipevar=None])

Run sextractor and scamp to refine astrometric solution.

Params

  • atfimages {list of str}: List of absolute paths of images to run through scamp.
  • scamprun {int, optional}: The first run does a LOOSE run with distortion degree 1, any other run will look for high distortion parameters, if it finds it will use distortion degree 7, otherwise 3 (will also cut out FLXSCALE on runs after 1).
  • pipevar {dict}: (Input pipeline parameters)[https://github.com/astroumd/photometrypipeline/wiki/autoproc_steps.py#pipeline-input-parameters].

Example & Usage

See autoproc_steps.py at line 619 and line 622.

medclip

<> medclip(indata, [clipsig=3.0, maxiter=5, verbose=0])

Median iterative sigma-clipping.

Params

  • indata {list}: Array to be clipped.
  • clipsig {float, optional}: Sigma to clip around.
  • maxiter {int, optional}: Maximum number of times to clip.
  • verbose {bool, optional}: Print out comments.

Example & Usage

See autoproc_depend.py (skypipecombine) at line 305, line 352.

medclip2d

<> medclip(indata, [clipsig=3.0, maxiter=5, verbose=0, overaxis=0])

Median iterative sigma-clipping.

Params

  • indata {list}: Array to be clipped.
  • clipsig {float, optional}: Sigma to clip around.
  • maxiter {int, optional}: Maximum number of times to clip.
  • verbose {bool, optional}: Print out comments.
  • overaxis {int, optional}: Axis that we want to take median over.

Example & Usage

See autoproc_depend.py (skypipecombine) at line 377.

autoproc_depend.py Functions

findcals

findcals(pipevar, file_format_str)

Locates calibration files of a specified type. Files must be in pipevar['caldir'], pipevar['datadir'], or pipevar['imworkingdir'].

Params

  • pipevar {dict}: Input pipeline parameters.
  • file_format_str {str}: String designating the type of calibration (bias*.fits, dark*.fits, flat*.fits).

write_fits

write_fits(filename, data, header):

Creates a FITS file with the specified parameters.

Params

  • filename {str}: Name of file being created.
  • data {list}: Data of FITS file.
  • header {list}: Header information of FITS file.

findsexobj

<> findsexobj(file, sigma, pipevar, [masksfx=None, zeropt=25.0, maptype='MAP_WEIGHT', wtimage=None, fwhm=1.5, pix=0.3787, aperture=5.0, elong_cut=1.5, quite=0])

Finds sextractor objects with optional inputs. Estimates seeing from stars found.

Params

  • file {str}: Absolute path to FITS file to run sextractor on.
  • sigma {float}: Detection threshold and analysis threshold for sextractor.
  • pipevar {dict}: Input pipeline parameters.
  • masksfx {str, optional}: Identifier for sextractor CHECKIMAGE_NAME.
  • zeropt {float, optional}: Input value for sextractor MAG_ZEROPOINT.
  • wtimage {str, optional}: Absolute file for input for sextractor WEIGHT_IMAGE.
  • fwhm {float, optional}: Input value for sextractor SEEING_FWHM.
  • pix {float, optional}: Input value for sextractor PIXEL_SCALE.
  • aperture {float, optional}: Input value for sextractor PHOT_APERTURES.
  • elong_cut {float, optional}: Cutoff limit for FWHM calculation of elongation to eliminate non-stars.
  • quiet {bool, optional}: Disable output from sextractor.

Example & Usage

See autoproc_steps.py (autopipestack) at line 745, line 935 and line 944.

calc_zpt

<> calc_zpt(catmag, obsmag, wts, [sigma=3.0, plotter=None])

Find zeropoint using robust scatter.

Params

  • catmag {list}: 2d array with catalog magnitudes catmag[nobs,nstar].
  • obsmag {list}: 2d array with catalog magnitudes obsmag[nobs,nstar].
  • wts {list}: 2d array with weights wts[nobs,nstar].
  • sigma {float, optional}: Sigma value for how far values can be from robust scatter value.
  • plotter {str, optional}: Absolute path to save zeropoint plot.

Output

  • z2 - Zeropoint correction.
  • scats - Robust scatter of each observation.
  • rmss - Standard deviation (without bad weight points) of each observation.

Example & Usage

See autoproc_steps.py (autopipestack) at line 844, and line 1039.

robust_scat

<> robust_scat(diff, wts, nobs, nstars, sigma)

Calculate robust scatter and set the weight of those above this limit to 0.

Params

  • diff {list}: Values to calculate robust scatter over.
  • wts {list}: Weights (0 is bad).
  • nobs {int}: Number of observations to iterate over.
  • nstars {int, optional}: Number of stars to iterate over.
  • sigma {float, optional}: Sigma*robust scatter that is acceptable.

Output

  • scats - robust scatter of each observation.
  • rmss - standard deviation (without bad weight points) of each observation.

Example & Usage

See autoproc_depend.py (calc_zpt) at line 829.

identify_matches

<> identify_matches(queried_stars, found_stars, [match_radius=3.0])

Use a kd-tree (3d) to match two lists of stars, using full spherical coordinate distances.

Params

  • queried_stars {list}: Numpy arrays of [ [ra,dec],[ra,dec], ... ] (all in decimal degrees).
  • found_stars {list}: Numpy arrays of [ [ra,dec],[ra,dec], ... ] (all in decimal degrees).
  • match_radius {float, optional}: Max distance (in arcseconds) allowed to identify a match between two stars.

Example & Usage

See autoproc_steps.py (autopipestack) at line 790, and line 988.

Installation

Installation Overview

Docker

From Source

Usage

Reference Documentation

Overview

Development

Overview

Clone this wiki locally