cobigen core_development

Table of Contents

• CobiGen Core Development
  • 1. Extension Mechanism
  • 3. Plugin Registry
    • 3-1. Mergers, Matchers, Trigger Interpreters and Input Readers (provisional title)
    • 3-2. LoadPlugin
  • 4. CobiGen Initialization
    • 4-1. FreeMarker Initialization
    • 4-2. Context Configuration
  • 5. Perform Generation
    • 5-1. Single Non Container Input
    • 5-2. Single Container Input or multiple files selection
    • 5-3. Summary

CobiGen Core Development

CobiGen uses Apache FreeMarker as engine for generation through FreeMarker templates.

Note

To know more about development using Apache FreeMarker, please refer to official documentation. To know more about FreeMarker template development, please refer to the official template development documentation and the guide of this wiki for each plugin: Java Plugin Xml Plugin Property Plugin Text Plugin Helpful Links

1. Extension Mechanism

The extension package contains the interfaces to be implemented if necessary by the sub plugins:

• GeneratorPluginActivator.java

• InputReader.java

• MatcherInterpreter.java

• Merger.java

• TriggerInterpreter.java

• ModelBuilder.java

The ModelBuilder is an interface for accessing the internal model builder instance. Is implemented by ModelBuilder.java from the model package that provides the methods to call the createModel() from the correspondent input reader from the correspondent trigger interpreter to create the object models for a given object.

The extension package has the to sub package that have the transfer objects of template, matcher, increment and variable assignment classes that will be used as "communication channel" between the core and sub plugins methods

3. Plugin Registry

The core must load all the sub plugins to get their Merger, Matcher, TriggerInterpreter and input reader (only for Java and Xml at the moment). That elements must implement their respective interfaces from the core.

3-1. Mergers, Matchers, Trigger Interpreters and Input Readers (provisional title)

At the case of Java plugin, an extension of the InputReader is used because defines a method to get items recursively for the case of a multiple selection or a package as input.

The xml input reader implements directly the input reader interface.

For the Text Appender and Property Merger plugins is not needed a trigger interpreter, a matcher neither an input reader.

3-2. LoadPlugin

The process of loading plugins to the core is done at the eclipse-plugin initialization.

Each sub plugin has an activator class that extends the GeneratorPluginActivator interface from the extension package. That class implements the methods bindMerger() and bindTriggerInterpreter().

This is the class passed as argument to the loadPlugin() method of PluginRegister.java of the pluginmanager package.

This method registers the mergers and the trigger interpreter of the sub plugins to the core. The trigger interpreter has the correspondent input reader of the plugin.

Note	How to add a new input reader

4. CobiGen Initialization

The CobiGen initialization must initialize the context configuration and the FreeMarker configuration

4-1. FreeMarker Initialization

When a CobiGen object is instantiated, the constructor initializes the Freemarker configuration creating a configuration instance from the class freemarker.template.Configuration and adjust its settings.

import freemarker.template.Configuration;
import freemarker.template.DefaultObjectWrapper;
import freemarker.template.DefaultObjectWrapperBuilder;
/**
 * The {@link CobiGen} provides the API for generating Code/Files from FreeMarker templates.
 *
 */
public class CobiGen {
    /**
    * The ContextConfiguration for this instance
    */
    private ContextConfiguration contextConfiguration;
    /**
    * The FreeMarker configuration
    */
    private Configuration freeMarkerConfig;
    /**
    * Configuration folder of CobiGen.
    */
    private Path configFolder;
    public CobiGen(URI configFileOrFolder) throws InvalidConfigurationException, IOException {
        if (configFileOrFolder == null) {
            throw new IllegalArgumentException("The configuration file could not be null");
        }
        configFolder = FileSystemUtil.createFileSystemDependentPath(configFileOrFolder);
        contextConfiguration = new ContextConfiguration(configFolder);
        freeMarkerConfig = new Configuration(Configuration.VERSION_2_3_23);
        freeMarkerConfig.setObjectWrapper(new DefaultObjectWrapperBuilder(Configuration.VERSION_2_3_23).build());
        freeMarkerConfig.clearEncodingMap();
        freeMarkerConfig.setDefaultEncoding("UTF-8");
        freeMarkerConfig.setLocalizedLookup(false);
        freeMarkerConfig.setTemplateLoader(new NioFileSystemTemplateLoader(configFolder));
    }
    ...
    ..
    .
}

Using the FileSystemUtil from the util package the URI of the root folder containing the context.xml and all templates, configurations etc…​ is converted to a Path object passing it as argument to the ContextConfiguration constructor. The ContextConfiguration creates a new ContextConfiguration from the config package with the contents initially loaded from the context.xml

Note	How the ContextConfiguration works explained deeply here.

The Configuration initialization requires the version of Freemarker to be used and at the ObjectWrapper initialization aswell. The DefaultObjectWrapperBuilder creates an DefaultObjectWrapper object that maps Java objects to the type-system of FreeMarker Template Language (FTL) with the given incompatibleImprovements specified by the version used as argument.

The configuration of Freemarker requires to specify to a TemplateLoader. A TemplateLoader is an interface provided by Freemarker library that the developer should implement to fit the needs. The TemplateLoader implementation at CobiGen is the class NioFileSystemTemplateLoader.java from the config.nio package.

4-2. Context Configuration

The context configuration reads the context.xml file from the template project (default: CobiGen_Templates) passing the path as argument to the constructor. At the constructor, it is created an instance of ContextConfigurationReader.java from the config.reader package.

Note	Please, check the CobiGen configuration for extended information about the context.xml and templates.xml configuration.

That reader uses the JAXB, JAXB (Java Architecture for XML Binding) provides a fast and convenient way to bind XML schemas and Java representations, making it easy for Java developers to incorporate XML data and processing functions in Java applications. As part of this process, JAXB provides methods for unmarshalling (reading) XML instance documents into Java content trees.

4-2-1. JAXB

JAXB auto generates the Java object within the JAXBContext especified at the xmlns attribute of the contextConfiguration field from the context.xml file

Unmarshaller unmarschaller = JAXBContext.newInstance(ContextConfiguration.class).createUnmarshaller();

That autogeneration follows the contextConfiguration.xsd schema. Each Java object follows the template specified with the field <xs:CompleType> from the schema file.

<xs:complexType name="trigger">
    <xs:sequence>
         <xs:element name="containerMatcher" type="tns:containerMatcher" minOccurs="0" maxOccurs="unbounded"/>
         <xs:element name="matcher" type="tns:matcher" minOccurs="0" maxOccurs="unbounded"/>
    </xs:sequence>
    <xs:attribute name="id" use="required" type="xs:NCName"/>
    <xs:attribute name="type" use="required" type="xs:string"/>
    <xs:attribute name="templateFolder" use="required" type="xs:string"/>
    <xs:attribute name="inputCharset" use="optional" type="xs:string" default="UTF-8"/>
 </xs:complexType>
 <xs:complexType name="matcher">
    <xs:sequence>
        <xs:element name="variableAssignment" type="tns:variableAssignment" minOccurs="0" maxOccurs="unbounded"/>
    </xs:sequence>
    <xs:attribute name="type" type="xs:string" use="required"/>
    <xs:attribute name="value" type="xs:string" use="required"/>
    <xs:attribute name="accumulationType" type="tns:accumulationType" use="optional" default="OR"/>
  </xs:complexType>

The generated Java objects has the elements and attributes specified at the schema:

@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "trigger", namespace = "http://capgemini.com/devonfw/cobigen/ContextConfiguration", propOrder = {
    "containerMatcher",
    "matcher"
})
public class Trigger {
    @XmlElement(namespace = "http://capgemini.com/devonfw/cobigen/ContextConfiguration")
    protected List<ContainerMatcher> containerMatcher;
    @XmlElement(namespace = "http://capgemini.com/devonfw/cobigen/ContextConfiguration")
    protected List<Matcher> matcher;
    @XmlAttribute(name = "id", required = true)
    @XmlJavaTypeAdapter(CollapsedStringAdapter.class)
    @XmlSchemaType(name = "NCName")
    protected String id;
    @XmlAttribute(name = "type", required = true)
    protected String type;
    @XmlAttribute(name = "templateFolder", required = true)
    protected String templateFolder;
    @XmlAttribute(name = "inputCharset")
    protected String inputCharset;
    ...
    ..
    .
}

This process it is done when calling the unmarshal() method.

Object rootNode = unmarschaller.unmarshal(Files.newInputStream(contextFile));

Note	For extended information about JAXB check the offical documentation.

4-2-2. Version Validation

If the version retrieved after the unmarshal process is null, an InvalidConfigurationException defined at exceptions package will be thrown.

If it is not null, will be compared using the validate() method from VersionValidator.java from config.versioning package with the project version retrieved by the MavenMetadata.java. The MavenMetadata.java file is provided by the POM while building the JAR file

<build>
    <plugins>
      <!-- Inject Maven Properties in java-templates source folder -->
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>templating-maven-plugin</artifactId>
        <executions>
          <execution>
            <id>generate-version-class</id>
            <goals>
              <goal>filter-sources</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      ...
      ..
      .
    </plugins>
</build>

MavenMetadata gets the current CobiGen version by reading the <version> label inside the <project> label from the POM file

public class MavenMetadata {
    /** Maven version */
    public static final String VERSION = "${project.version}";
}

<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
  xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <artifactId>cobigen-core</artifactId>
  <name>CobiGen</name>
  <version>2.2.0-SNAPSHOT</version>
  <packaging>jar</packaging>
  ...
  ..
  .
}

The comparison has three possibilities:

1. Versions are equal → Valid

2. context.xml version is greater than current CobiGen version → InvalidConfigurationException

3. Current CobiGen version is greater that context.xml version → Compatible if there not exists a version step (breaking change) in between, otherwise, throw an error.

Reaching this point, the configuration version and root node has been validated. Unmarshal with schema checks for checking the correctness and give the user more hints to correct his failures.

SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
ContextConfigurationVersion latestConfigurationVersion = ContextConfigurationVersion.getLatest();
try (
    InputStream schemaStream = getClass().getResourceAsStream("/schema/" + latestConfigurationVersion
                                                              + "/contextConfiguration.xsd");
    InputStream configInputStream = Files.newInputStream(contextFile)) {
    Schema schema = schemaFactory.newSchema(new StreamSource(schemaStream));
    unmarschaller.setSchema(schema);
    rootNode = unmarschaller.unmarshal(configInputStream);
    contextNode = (ContextConfiguration) rootNode;
}

4-2-3. Load Triggers, Matchers, containerMatcher, AccumulationTypes and VariableAssigments

To finish the context configuration initialization, the, trigger, matchers, container matchers, accumulation types and variables assigments are retrieved from the correspondent Java objects generated by JAXB.

/**
* Loads all Triggers of the static context into the local representation
* @return a List containing all the Triggers
*/
public Map<String, Trigger> loadTriggers() {
    Map<String, Trigger> triggers = Maps.newHashMap();
    for (com.capgemini.cobigen.config.entity.io.Trigger t : contextNode.getTrigger()) {
        triggers.put(t.getId(), new Trigger(t.getId(), t.getType(), t.getTemplateFolder(),
            Charset.forName(t.getInputCharset()), loadMatchers(t), loadContainerMatchers(t)));
    }
    return triggers;
}
/**
* Loads all Matchers of a given com.capgemini.cobigen.config.entity.io.Trigger
* @param trigger
*            com.capgemini.cobigen.config.entity.io.Trigger to retrieve the Matchers from
* @return the {@link List} of {@link Matcher}s
*/
private List<Matcher> loadMatchers(com.capgemini.cobigen.config.entity.io.Trigger trigger) {
    List<Matcher> matcher = new LinkedList<>();
    for (com.capgemini.cobigen.config.entity.io.Matcher m : trigger.getMatcher()) {
        matcher.add(
        new Matcher(m.getType(), m.getValue(), loadVariableAssignments(m), m.getAccumulationType()));
    }
    return matcher;
}
/**
* Loads all ContainerMatchers of a given com.capgemini.cobigen.config.entity.io.Trigger
* @param trigger
*            com.capgemini.cobigen.config.entity.io.Trigger to retrieve the Matchers from
* @return the List of Matchers
*/
private List<ContainerMatcher> loadContainerMatchers(com.capgemini.cobigen.config.entity.io.Trigger trigger) {
    List<ContainerMatcher> containerMatchers = Lists.newLinkedList();
    for (com.capgemini.cobigen.config.entity.io.ContainerMatcher cm : trigger.getContainerMatcher()) {
       containerMatchers.add(new ContainerMatcher(cm.getType(), cm.getValue(), cm.isRetrieveObjectsRecursively()));
    }
    return containerMatchers;
}
/**
* Loads all VariableAssignments from a given com.capgemini.cobigen.config.entity.io.Matcher}
* @param matcher
*            com.capgemini.cobigen.config.entity.io.Matcher to retrieve the
*            VariableAssignment from
* @return the List of Matchers
*/
private List<VariableAssignment> loadVariableAssignments(com.capgemini.cobigen.config.entity.io.Matcher matcher) {
    List<VariableAssignment> variableAssignments = new LinkedList<>();
    for (com.capgemini.cobigen.config.entity.io.VariableAssignment va : matcher.getVariableAssignment()) {
        variableAssignments.add(new VariableAssignment(va.getType(), va.getKey(), va.getValue()));
    }
    return variableAssignments;
}

5. Perform Generation

Depending on the input, the generation process can begin from two different generate() methods called at the CobiGenWrapper from the eclipse-plugin:

public void generate(TemplateTo template, boolean forceOverride) throws IOException, TemplateException, MergeException {
    if (singleNonContainerInput) {
        // if we only consider one input, we want to allow some customizations of the generation
        Map<String, Object> model = cobiGen.getModelBuilder(inputs.get(0), template.getTriggerId()).createModel();
        adaptModel(model);
        cobiGen.generate(inputs.get(0), template, model, forceOverride);
    } else {
        for (Object input : inputs) {
            cobiGen.generate(input, template, forceOverride);
        }
    }
}

5-1. Single Non Container Input

If the input is a single non container input, first step is to create the model, then allow customization by the user (adaptModel()) and finally call the generate() method from CobiGen using the input, template, model and the boolean forceOverride. The generation process in this case will follow this main steps:

1. Check if the input is not null

2. Get the trigger interpreter for the type of the trigger of the template

   public void generate(Object generatorInput, TemplateTo template, Map<String, Object> model, boolean forceOverride)
                        throws InvalidConfigurationException, IOException, TemplateException, MergeException {
       //1. test if input is not null
       InputValidator.validateInputsUnequalNull(generatorInput, template, model);
       //2. get the trigger of the template from the configuration context
       Trigger trigger = contextConfiguration.getTrigger(template.getTriggerId());
       //2. use the trigger to get the trigger interpreter
       ITriggerInterpreter triggerInterpreter = PluginRegistry.getTriggerInterpreter(trigger.getType());
       generate(generatorInput, template, triggerInterpreter, forceOverride, model);
   }

3. Set the root folder for the templates to use for the generation

4. Get the input reader for the trigger interpreter retrieved

5. Test if the input is a package.
   This only can be possible in the case of java inputs. As the input is a single non container input, this check will fail and the execution will continue.

6. Check if the model parameter is null and if it is, create a new model
   As the model has been created at the CobiGenWrapper, there is no need to create it again.

7. Get the destination file.

8. Check if the destination file already exists
   If it exists, but the forceOverride is set to true or the merge strategy of the template is null, the file will be overwritten, not merged. Otherwise, first generate output into a writter object, get the merger and merge the original file with the writter and write the file with the merge result.

9. If the file does not exists, simple write the file.

5-2. Single Container Input or multiple files selection

The other case is, or the input is multiple files selection, the generation process will be performed for each individual file of the selection, but the model will be created at the step 6 of the steps of the Single Non Container Input and not allowing the user customization.

5-3. Summary