README

This folder contains Python scripts: 
- mat2h5.py, make_nwb.py and mat2nwb.py, that allow conversion of data files 
  from MAT to NWB format (MAT => NWB), together with 
- several utility scripts for exploring the data in both 
  HDF5 and NWB files, and 
- a library file h5lib.py which defines the functions shared by all other scripts. 

For a single input file, the MAT => NWB conversion can be performed in 2 steps:
- MAT => HDF5 conversion, performed by the script mat2h5.py the resulting
              HDF5 file preserves the structure of the original MAT file 
              as closely as possible; this step is not data-specific, i.e. 
              applicable to any MAT file, possibly comprising nested data 
              structures with arbitrary number of nesting levels; and
- HDF5 => NWB conversion, performed by script make_nwb.py, which employs
              NWB API (see the upper level folder "api-python");
              this step is very data-specific; the current version of script
              make_nwb.py supports conversion of data generated by 
              Dr. Karel Svoboda lab at Janelia Farm research facility of HHMI.

The script mat2nwb.py combines the two conversion steps in a pipeline that allows 
parallel conversion of large amounts of data using SGI compute cluster.

NOTE: script mat2h5.py makes use of the loadmat function of scipy package.
      We observed an issue (truncation of strings) when using this function 
      with Python 2.x on Scientific Linux platform ( see
      https://github.com/scipy/scipy/issues/5984 ).
      Because of this issue, the 1st of the two steps of conversion is
      recommended to be performed using Python 3.x. 
      The second step works fine with Python 2.x.

In order to see usage of any script, type its name.

To see available options for scripts mat2h5.py, make_nwb.py and mat2nwb.py, 
type the name of the script followed by flag "-h".


Example of usage of the script mat2h5.py:

    python3 mat2h5.py <data>.mat               #   file <data>.h5     will be produced
    python3 mat2h5.py <metadata>.mat           #   file <metadata>.h5 will be produced

Examples of usage of the script make_nwb.py:

- if metadata is not included in the data file:
    make_nwb.py  <data>.h5 <metadata>.h5      #   file <data>.nwb will be produced
- otherwise:
    make_nwb.py  <data>.h5                    #   file <data>.nwb will be produced    

Examples of usage of the script mat2h5.py:
- on motor cortex data:
    mat2nwb.py data_dir -M 
- on somatosensory cortex data:
    mat2nwb.py data_dir -S

where data_dir is a folder containing data and metadata files 
in the case of motor cortex data conversion and data files 
in the case of somatosensory cortex data conversion.

To make use of these scripts, the user must define environment variable
NWB_DATA, which points to the full path to the root data directory.
This directory, in particular, will contain files data_collection.txt,
experiment_description.txt, and surgery.txt, which are used as additional metadata.
All the other data passed to the scripts may be specified as paths ralative to this 
root directory.