Automatic Code Generation

MEI Customization process

The core MEI schema is written using an XML "meta-schema" language developed by the Text Encoding Initiative. "ODD," or "One-Document-Does-it-all" allows developers to create formal rules for the encoding language (MEI or TEI) but also provides the ability to supply the documentation inline with the rules.

Once these rules and documentation are in place, the Roma processor can be used to generate a formal XML Schema language (RelaxNG, W3C Schema, i.e., XSD, or DTD) RelaxNG is the preferred schema language for validating MEI-encoded files. Roma may also be used to generate HTML or PDF documentation for the schema.

One further advantage to using ODD is the idea of customizations. Roma requires two ODD-encoded files: A "source" containing all possible element and attribute definitions separated into logical modules (e.g. MEI.mensural for mensural notation, MEI.neumes for neume notation, etc.), and a "customization" which defines which modules should be in the resulting XML schema and documentation. MEI comes with a default customization which turns on all modules.

Also included in this customization process is the ability to define new elements or attributes, or even re-define existing elements and attributes in the source. Adding a new type of music notation to MEI would simply require a customization file that defines the logic for that specific type of notation. See, for example, an existing customization file for defining a new approach to neume notation. How this works is beyond the scope of this document, but it is important to know that this exists.

Generating a custom libmei library

The parseschema2.py script in the tools/ directory allows developers to supply an MEI customization and the script will generate custom C++ code that defines explicit classes for each element in MEI, and getters and setters for every attribute defined on them. Libmei ships with the most recent versions of the MEI modules in the src/modules directory.

The easiest way to generate a new library is by using a pre-compiled ODD file. You can either install the roma2 script from the TEI project (Instructions) or you can use the MEI Customization web service. Either of these will generate a "Compiled ODD" file.

To generate new C++ modules, run the parseschema2.py script with the compiled ODD, e.g.,:

$> python parseschema2.py -m compiled_odd.xml -o src -l cpp

Where -m is the path to the compiled ODD, -o is the output directory for the modules, and -l is the language you wish to output. This will generate the .cpp and .h files for the elements in each of the MEI modules.

The Includes process

While getters and setters on the attributes of all the elements expose quite a bit of functionality, there are other functions that are specific to some elements that it would be nice to support, like getMembers() on a <tie> element. However, since all the code is auto-generated from the MEI spec, we need a way to "include" extra methods that doesn't get clobbered when we re-generate the library.

When we auto-generate the MEI module code, we place special comments on each element that look like this:

C++

/* include <elementname> */

Python

# <elementname>

These serve as markers for the parseschema2.py script for inserting extra methods on each member class. In the tools/includes directory you will find a directory for each language that we support (currently C++ and Python). In there are files that are named after the modules where the elements are found. We'll start with looking at the C++ module support.

C++ Includes

Each include needs files for both the .h and .cpp files. For example, if we wanted to add additional methods to the <note> element, this is contained in the shared module. So, we will name our file shared.h.inc and shared.cpp.inc.

At the top of this file are any extra C++ modules that are required to run the code. This will be placed in the header section of the generated code. This is specified in a block delineated with the comment /* #include_block */, e.g.,

/* #include_block */
#include "meielement.h"
#include "meidocument.h"
#include <vector>
#include <algorithm>
#include <iostream>

using std::cout;
using std::endl;

using std::vector;
using std::find;
using mei::MeiElement;
using mei::MeiDocument;
/* #include_block */

Then, for each element you wish to add extra methods to, you simply mark out the blocks with the tag name in a comment, like this:

shared.cpp

/* <note> */
string mei::Note::getLayerIdent() {
    return this->getAncestor("layer")->getAttribute("n")->getValue();
}

MeiElement* mei::Note::getLayer() {
    return this->getAncestor("layer");
}

string mei::Note::getStaffIdent() {
    return this->getAncestor("staff")->getAttribute("n")->getValue();
}

MeiElement* mei::Note::getStaff() {
    return this->getAncestor("staff");
}

MeiElement* mei::Note::getSystem() {
    return this->lookBack("sb");
}
/* </note> */

shared.h

/* <note> */
        std::string getLayerIdent();
        MeiElement* getLayer();

        std::string getStaffIdent();
        MeiElement* getStaff();

        MeiElement* getSystem();

/* </note> */

To include the generated code, simply give the path to the includes directory to the parseschema2.py script, e.g.,

$> python parseschema2.py -m compiled_odd.xml -o src -i includes/cpp -l cpp

Now when you re-generate the MEI schema the extra methods you have written to support working with certain MEI elements will not be clobbered.

Python Includes

Python includes function in much the same way as the C++ includes. You will need an include file named after the module, e.g., shared.py.inc, and then each method in the file marked in the same way:

# <note>
    def foo(self):
        print "bar"
# </note>

This will add the foo() method to the note_() class definition in shared.py. Extra include modules are supported in exactly the same way as C++:

# include_block
import os
import sys
# include_block

Using your modules

After generating your new modules, you'll have to (re)install libmei before being able to use them.

C++

Move the generated .cpp and .h files into the src/modules folder and build according to the instructions.

Python

This should be done after installing libmei with the C++ version of your new modules. Move the generated .py files into python/pymei/Modules. Rebuild and reinstall.