Skip to content

Commit

Permalink
Merge pull request #120 from ModECI/development
Browse files Browse the repository at this point in the history
MDF v0.2
  • Loading branch information
pgleeson authored Sep 29, 2021
2 parents 5e0ae11 + 7216631 commit 92dcfe1
Show file tree
Hide file tree
Showing 134 changed files with 11,348 additions and 9,598 deletions.
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -220,3 +220,10 @@ Thumbs.db #thumbnail cache on Windows
/examples/MDF/ABCD.nmllite.json
/examples/MDF/Simple.nmllite.json
/examples/MDF/abcd_3
/src/modeci_mdf/interfaces/graphviz/*gv.png
/src/modeci_mdf/*.png
/examples/PyTorch/inception
/examples/PyTorch/ddm
/examples/MDF/abcd.png
/examples/MDF/abcd_3.png
/test.json
10 changes: 4 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,13 @@
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

# ModECI Model Description Format (MDF)

**Note: MDF is still in development! See the [open issues related to the specification](https://github.com/ModECI/MDF/issues?q=is%3Aissue+is%3Aopen+label%3Aspecification) or go [here](http://modeci.org/#aboutPage) to get in contact regarding MDF.**

*The MDF format was first proposed following a meeting organised at Princeton in July 2019 by Russ Poldrack of the Center for Reproducible Neuroscience (CRN) at Stanford and the [Brain Imaging Data Standard (BIDS)](https://bids.neuroimaging.io/) initiative. For more on the previous work in this area, see [here](https://github.com/OpenSourceBrain/PsyNeuLinkShowcase/tree/master/BIDS-MDF).*


## Overview

MDF is intended to be an open source, community-supported standard and associated library of tools for expressing computational models in a form that allows them to be exchanged between diverse programming languages and execution environments. It consists of a specification for expressing models in a serialized format (currently a JSON representation, though others such as YAML and HDF5 are planned) and a set of Python tools for implementing a model described using MDF. The serialized format can be used when importing a model into a supported target environment to execute it; and, conversely, when exporting a model built in a supported environment so that it can be re-used in other environments.
MDF is intended to be an open source, community-supported standard and associated library of tools for expressing computational models in a form that allows them to be exchanged between diverse programming languages and execution environments. It consists of a specification for expressing models in a serialized format (currently a JSON representation, though others such as YAML and HDF5 are planned) and a set of Python tools for implementing a model described using MDF. The serialized format can be used when importing a model into a supported target environment to execute it; and, conversely, when exporting a model built in a supported environment so that it can be re-used in other environments.

The MDF Python API can be used to create or load an MDF model for inspection and validation. It also includes a basic scheduler for simulating models in the format. However, this is not intended as a general purpose simulation environment, nor is MDF intended as a programming language. Rather, the primary purpose of the Python API is to facilitate and validate the exchange of models between existing environments that serve different communities. Accordingly, these Python tools include bi-directional support for importing to and exporting from widely-used programming environments in a range of disciplines, and for easily extending these to other environments.

Expand All @@ -26,12 +24,12 @@ The implementation and dissemination of the MDF language and associated tools is
<a href="examples/README.md"><img alt="mdf interacts" width="400" src="https://raw.githubusercontent.com/ModECI/MDF/main/examples/ModECI_MDF.svg"/></a>
<br/><sup><i><b>Fig 1:</b> Some of the current and planned formats which MDF will interact with. Click on the image for more information.</i></sup></p>

Successful interfacing of MDF to existing disciplinary standards (such as [ONNX](http://onnx.ai) in machine learning, and [NeuroML](https://neuroml.org) in neuroscience) as well as general purpose simulation environments (such as [WebGME](https://webgme.org)) will permit bridging between these environments, and translation to the broader set of environments supported by those standards (such as [Tensorflow](https://www.tensorflow.org) & [Keras](https://keras.io) in the case of ONNX, and [The Virtual Brain](https://www.thevirtualbrain.org) and [SONATA](https://github.com/AllenInstitute/sonata) in the case of NeuroML). Efforts are also underway, in collaboration with projects in the NSF Accelerator Track C (Quantum Technology), to use MDF for facilitating the implementation of computational models on [quantum hardware](https://github.com/ModECI/MDF/blob/readme_update/examples/Quantum).
Successful interfacing of MDF to existing disciplinary standards (such as [ONNX](http://onnx.ai) in machine learning, and [NeuroML](https://neuroml.org) in neuroscience) as well as general purpose simulation environments (such as [WebGME](https://webgme.org)) will permit bridging between these environments, and translation to the broader set of environments supported by those standards (such as [Tensorflow](https://www.tensorflow.org) & [Keras](https://keras.io) in the case of ONNX, and [The Virtual Brain](https://www.thevirtualbrain.org) and [SONATA](https://github.com/AllenInstitute/sonata) in the case of NeuroML). Efforts are also underway, in collaboration with projects in the NSF Accelerator Track C (Quantum Technology), to use MDF for facilitating the implementation of computational models on [quantum hardware](https://github.com/ModECI/MDF/blob/readme_update/examples/Quantum).


### The core elements of the MDF standard

**[Models](https://github.com/ModECI/MDF/blob/main/docs/README.md#model)** The highest level construct in MDF is a model that consists of one or more graphs and model attributes. The former describe the operational features of the model (its structure and execution), while the latter provide additional information useful for executing and evaluating it (e.g., test data and benchmark results).
**[Models](https://github.com/ModECI/MDF/blob/main/docs/README.md#model)** The highest level construct in MDF is a model that consists of one or more graphs and model attributes. The former describe the operational features of the model (its structure and execution), while the latter provide additional information useful for executing and evaluating it (e.g., test data and benchmark results).

**[Graphs](https://github.com/ModECI/MDF/blob/main/docs/README.md#graph)** A graph specifies the structure and process flow of a model. The most fundamental element of a graph is a node, which specifies some unit of computation as one or more functions. Functions reference executable implementations in a standardized ontology (with bindings to well-established existing ontologies, such as ONNX, where available). Nodes are connected to other nodes via directed edges, which, in the absence of additional conditions, define the computational flow of the model.

Expand All @@ -41,7 +39,7 @@ Successful interfacing of MDF to existing disciplinary standards (such as [ONNX]

**[Conditions](https://github.com/ModECI/MDF/blob/main/docs/README.md#condition)** These are a core and distinctive element of the MDF specification, that complement other computational graph-based formats by providing a high-level set of descriptors for specifying conditional execution of nodes. This allows models with relatively complex execution requirements (e.g., containing cycles, branches, and/or temporal dependencies) to be expressed as graphs in a sufficiently abstract form that facilities exchange among high-level modeling environments without requiring that they be “lowered” to and then recovered from more elaborated procedural descriptions.

**[Parameters and Arguments](https://github.com/ModECI/MDF/blob/main/docs/README.md#node)** Attributes that determine the configuration and operation of nodes and edges, such as function parameters and weight matrices, can be defined in the MDF using parameters. In the case of parameters specifying large data structures (e.g., weight-matrices), arrays in widely used formats (e.g. numpy arrays) can be used, and serialisation in portable binary formats (e.g. HDF5) will be supported.. Functions can have dynamically-set attributes in the form of arguments, often sourced from an input port on the node containing that function. Conditions may use arguments which are both static and dynamic. For example, a threshold condition may compare a dynamically-changing value, passed as an argument, against a static threshold parameter.
**[Parameters and Arguments](https://github.com/ModECI/MDF/blob/main/docs/README.md#node)** Attributes that determine the configuration and operation of nodes and edges, such as function parameters and weight matrices, can be defined in the MDF using parameters. In the case of parameters specifying large data structures (e.g., weight-matrices), arrays in widely used formats (e.g. numpy arrays) can be used, and serialisation in portable binary formats (e.g. HDF5) will be supported.. Functions can have dynamically-set attributes in the form of arguments, often sourced from an input port on the node containing that function. Conditions may use arguments which are both static and dynamic. For example, a threshold condition may compare a dynamically-changing value, passed as an argument, against a static threshold parameter.

**[States](https://github.com/ModECI/MDF/blob/main/docs/README.md#state)** For information that must persist between executions of a node (such as integrator functions), MDF nodes support states, which can be modified and accessed on a given execution, and will persist and be available in subsequent executions.

Expand Down
Loading

0 comments on commit 92dcfe1

Please sign in to comment.