documented stage example

.gitignore

@@ -0,0 +1,9 @@
+*/__pycache__
+dist/
+build/
+BBPipe.egg-info/
+*.pyc
+test/outputs
+test/logs
+log.txt
+runinfo

CONTRIBUTING.md

@@ -10,5 +10,3 @@ If you think something is missing in `BBPipe` or any of the associated pipelines
  - Make a pull request (PR) for your new branch as soon as you're happy to do it. You don't have to finish the work to open the PR. Often it's useful to open PRs early on so other contributors know that you're working on this specific issue. For the same reason, it's probably a good idea to advertise the new work in the slack channel if you think others will be interested.
  - As soon as you think your work is done, ask for volunteers to review your PR. Note that you can nominate specific PR reviewers by clicking on 'Reviewers', on the top right corner of your PR.
  - When your PR gets accepted, merge it onto master!
-
-

README.md

@@ -23,6 +23,8 @@ Creating a new pipeline stage involves creating a python module. Note that this
 - Each stage is defined by a class which must inherit from `bbpipe.PipelineStage`. Each class must have its own `name`, `inputs` and `outputs` attributes (essentially the names of the expected input and output data), and a `run` method that executes the stage.
 - The `run` method should use the parent methods from `PipelineStage` to get its inputs and outputs etc.
 
+Have a look at the extended comments in [bbpower_test/mask_preproc.py](bbpower_test/mask_preproc.py) for more details on the structure of any pipeline stage.
+
 To create the yaml file that puts your pipeline together, have a look at the [test file](test/test.yml). This file should contain:
 - A list of modules where the different pipeline stages are to be found.
 - The launcher type (to be used by PARSL to launch each stage). Currently the only defined launcher type is the `local` one (i.e. launch jobs serially in your machine), but more will be defined. They will be located in [`bbpipe/sites`](bbpipe/sites).

bbpower/mask_preproc.py

@@ -4,7 +4,7 @@
 
 class BBMaskPreproc(PipelineStage):
     """
-    Template for a map pre-processing stage
+    Template for a mask pre-processing stage
     """
     name='BBMaskPreproc'
     inputs= [('binary_mask',FitsFile),('source_data',TextFile)]

bbpower_test/mask_preproc.py

@@ -4,11 +4,28 @@
 
 class BBMaskPreproc(PipelineStage):
     """
-    Template for a map pre-processing stage
+    Template for a mask pre-processing stage
     """
+    #The following 3 attributes are required for any PipelineStage
+    #Name. Make it coincide with the class name for convenience.
     name='BBMaskPreproc'
+    #Inputs. List of tuples.
+    # Each tuple contains the name of the dataset and the class describing its format.
+    #   See types.py for a few examples of format classes
+    # The name of the dataset is either the key used in a yaml config file or the key
+    #   assigned to it by a preceding pipeline stage (e.g. the name of an output)
+    # Note that you can access the file names associated with each input by using
+    #   the method PipelineStage.get_input(input_name).
     inputs= [('binary_mask',FitsFile),('source_data',TextFile)]
+    #Outputs. List of tuples.
+    # Each tuple contains the same information as the `inputs` tuples.
+    # The name will be used by subsequent pipline stages in their inputs.
+    # Note that BBPipe assigns file names to each output based on their name, on the
+    #   contents of the configuration files and on their data type. To retrieve the
+    #   file names use the method PipelineStage.get_output(output_name)
     outputs=[('window_function',FitsFile)]
+    #You can also define default configuration options. These get overwritten by any
+    #options set in this stage's configuration.
     config_options={'aposize_edges':1.0,
                     'apotype_edges':'C1',
                     'aposize_srcs':0.1,
@@ -20,7 +37,7 @@ def run(self) :
         mask_raw=hp.read_map(self.get_input('binary_mask'),verbose=False)
 
         #Read point source data
-        #Right now this is a simple text file, but this is probably not ideal.
+        # Right now this is a simple text file, but this is probably not ideal.
         ps_ra,ps_dec,ps_size=np.loadtxt(self.get_input('source_data'),unpack=True,ndmin=2)
 
         #Now we should do stuff (apodization, inverse-variance weighting etc.)