Module Guidelines
The goal of this document is to demonstrate how to add a new module to ReVAS. If you don't have time to read it all, jump to Summary of Guidelines.
Each module in ReVAS must have two input arguments. The first one (also referred to as primary input) must be a full path to a file (video, or .mat file) or an input array (3D array for videos, 2D array for position, etc.). The second argument (secondary input) must be a params structure, which defines the parameters to be used in this particular module. The params structure can have any number or type of fields, however it must have at least an overwrite field, which will determine whether or not the output of this module will overwrite an existing file with the same name.
Depending on the type of operation that our new module will do, there might be additional restrictions or some required fields. For instance, if our new module is going to operate on videos, a badFrames field (which is a logical array indicating which frames are "bad" due to blinks or subject movement, etc.) is also required to propagate that info downstream in the processing pipeline, even if it is not used in our new module.
If we want to use the ReVAS GUI to show some progress or visualize the output of our module, we must add a field called enableVerbosity to params structure, along with the axesHandles field, indicating which axes object(s) will be used to do the visualization.
Each ReVAS module must have at least two output arguments. The first output argument (also referred to as primary output) is used as an input to the next module in the pipeline, therefore, a decision has to be made here. If the immediate output of our module is going to be fed into the next module in the pipeline, then this output variable (i.e., processed video, extracted motion, filtered motion, etc.) can be passed to the first output argument. If this module only acts like an intermediate processing step (e.g., creating a reference frame but passing the input video to the next module untouched), then first output argument must be the same as the first input argument and the primary product of the current module must be passed onto subsequent modules via the params structure.
The second output argument must be the params structure. This is (1) to enable chaining multiple modules back to back, (2) passing output of a bypass module to later stages in the pipeline.
Finally, a module might have three or more output arguments. MATLAB's varargout must be used for modules with more than two output arguments.
Conceptually, there are three types of modules.
Core modules are the ones which take in a video array or eye motion traces or a file path to one of these, perform some processing steps on this input, and return a new output (in the form of a modified video, or processed motion traces). For instance, BandpassFilter.m takes in a video and applies a bandpass filter to each frame, and returns another video with these filtered frames.
Bypass modules also take one of the aforementioned inputs, however, they pass the input to the output as is. Bypass modules are used to create intermediate or auxiliary parameters or dependencies for subsequent modules in the pipeline. For instance, FindBlinkFrames.m module takes in a video, detects bad frames during which video quality drops or there is a blink, passes the input video as is to its first output argument and an array of labels badFrames as a field of its second output argument params. Therefore, FindBlinkFrames.m can be considered as a Bypass module.
Hybrid modules can be considered as a combination of the previous two types. StripAnalysis.m module, for instance, takes in a video and returns extracted eye motion traces in its primary output. In addition, it returns same and additional information (e.g., position, timeSec, peakValueArray, etc.) as new fields under params structure.
The following table classifies existing ReVAS modules into these different types.
| Core | Bypass | Hybrid |
|---|---|---|
|
|
|
We want to create a new Hybrid module. Let us assume that this module takes in a video, inverts pixel values of each frame (just for the sake of an example), and randomly selects one of its frames as a reference frame. We will call this new module DummyModule.m. Conceptually, it is going to be a Hybrid module. Although it may not make much sense, for the sake of demonstration, we will return the selected frame as third output argument as well.
Note that there is already a module called
MakeReference.min ReVAS to create a reference frame, however, it has limitations and definitely a lot of room for improvement. Therefore, one might want to implement their existing and arguable better reference frame synthesis module, and incorporate it into ReVAS.
function [inputVideo, params, varargout] = DummyModule(inputVideo, params)
%[inputVideo, params, varargout] = DummyModule(inputVideo, params)
%
% This is a dummy module, used to demonstrate how to add a new module to
% ReVAS.All modules in ReVAS determine whether the output will be written to a new file, or it will be returned as an output argument by looking at the variable type of the first input argument. In this example, inputVideo is our primary input. If first input is not specified, then the module will error out.
if ischar(inputVideo)
% A path was passed in.
% Read the video and once finished with this module, write the result.
writeResult = true;
else
% A video matrix was passed in.
% Do not write the result; return it instead.
writeResult = false;
endNext, we need to get params fields for this module, their default values and validation functions. We can ValidateField.m function to check if values input to the function violate the constraints in any of the fields. If there is a missing field, this function adds that with the corresponding default value.
if nargin < 2
params = struct;
end
% validate params
[~,callerStr] = fileparts(mfilename);
[default, validate] = GetDefaults(callerStr);
params = ValidateField(params,default,validate,callerStr);Of course, this assumes that you defined fields, default values, and validation functions for your new module inside the GetDefaults.m function. GetDefaults.m is one the two utility functions in ReVAS (the other one is Filename.m but we will come to that later) that you need to modify to add/modify a given module. For the DummyModule.m example, we added the following case to GetDefaults.m.
function [default, validate, before, after, keyword, axesHandles] = GetDefaults(module)
.
.
.
switch module
case 'dummymodule'
% default values
default.overwrite = false;
default.enableVerbosity = 'frame';
default.badFrames = false;
default.axesHandles = [];
% validation functions
validate.overwrite = @islogical;
validate.enableVerbosity = @(x) CategoryOrLogicalOrNumeric(x,{'none','video','frame'});
validate.badFrames = @(x) all(islogical(x));
validate.axesHandles = @(x) isempty(x) | all(ishandle(x));
% list which modules can preceed or succeed this one
before = {'degree2pixel','stripanalysis'};
after = {'none','stripanalysis'};
% keyword to be used in filenames
keyword = 'dummy';
% axes handle tags.. useful only in GUI mode
axesHandles = {'imAx'};-
Let's break it down. The
casename is simply the name of our module, all lowercase letters and without file extension,dummymodule. -
We have four required fields. We create two structures
defaultandvalidateto store default values and validation functions.
-
overwrite: a logical flag indicating that the output file will be overwritten if it already exists. Defaults tofalse, i.e., do not overwrite if a file already exists.
default.overwrite = false;
validate.overwrite = @islogical;-
enableVerbosity: a character array indicating the level of visualization requested. Defaults toframemeaning that we will get a visualization at every frame. Available modes arenone(no visualization, andvideo(one visualization per video). Some other modules use simplerenableVerbosityfields which is simply a true/false flag.
default.enableVerbosity = 'frame';
validate.enableVerbosity = @(x) CategoryOrLogicalOrNumeric(x,{'none','video','frame'});-
badFrames: A logical array indicating which frames are to be skipped during processing. Setting it tofalseis equivalent tofalse(numberOfFrames,1).
default.badFrames = false;
validate.badFrames = @(x) all(islogical(x));-
axesHandles: An array to hold axes object(s) to be used for visualization. Defaults to[], empty. This is to make sure ReVAS does not crash when used independent from the GUI.
default.axesHandles = [];
validate.axesHandles = @(x) isempty(x) | all(ishandle(x));ReVAS GUI uses additional output arguments of GetDefaults.m to properly populate axes handles for each module. To ensure that we create a variable axesHandles and set it to the name of one of the axes objects in ReVAS GUI.
% axes handle tags.. useful only in GUI mode
axesHandles = {'imAx'};Available axes objects are given in the table below, numbers refer to the annotated image of the blank ReVAS GUI.
| Number | Axes Object |
|---|---|
| 4 | imAx |
| 5 | posAx |
| 6 | peakAx |
| 7 | motAx |
| 8 | stimAx |
| 9 | blinkAx |
- We define the connectivity of this module with other modules.
% list which modules can preceed or succeed this one
before = {'degree2pixel','stripanalysis'};
after = {'none','stripanalysis'};-
beforeis a cell array of module names. These modules can come before ourDummyModule.min the pipeline. -
afteris also a cell array of modules that can succeed our new module.
- Finally, we choose a keyword for our new module. This keyword will be appended to input file name when saving the output of our module. So choose something short but meaningful.
When creating a module for ReVAS, keep in mind that it will be used both with and without GUI. In GUI mode, we use specific user interface elements; axes objects for visualization, text box objects for displaying messages/logs, push/toggle button objects to interrupt processing. A perfect UI design completely separates the GUI from actual processing routines. However, ReVAS is imperfect in the sense that depending on in which mode it operates (as a toolbox or an app with a GUI), it may need additional input. As of 5/11/2020, the only extra input ReVAS modules have to look for are logBox and abort fields under params structure. If a module is called from ReVAS GUI, the params structure will have these two fields to enable the module to write to log box (number 10 in the annotated figure above) and to allow user to prematurely end (abort) the processing by pressing a push button. This is handled with the code block below.
%% Handle GUI mode
% params can have a field called 'logBox' to show messages/warnings
if isfield(params,'logBox')
logBox = params.logBox;
isGUI = true;
else
logBox = [];
isGUI = false;
end
% params will have access to a uicontrol object in GUI mode. so if it does
% not already have that, create the field and set it to false so that this
% module can be used without the GUI
if ~isfield(params,'abort')
params.abort.Value = false;
end%% Handle verbosity
if ischar(params.enableVerbosity)
params.enableVerbosity = find(contains({'none','video','frame'},params.enableVerbosity))-1;
end
% check if axes handles are provided, if not, create axes.
if params.enableVerbosity && isempty(params.axesHandles)
fh = figure(2020);
set(fh,'name','Dummy',...
'units','normalized',...
'outerposition',[0.16 0.053 0.4 0.51],...
'menubar','none',...
'toolbar','none',...
'numbertitle','off');
params.axesHandles(1) = subplot(1,1,1);
end
% clear axes
if params.enableVerbosity
cla(params.axesHandles(1))
tb = get(params.axesHandles(1),'toolbar');
tb.Visible = 'on';
end