Plans wrt to enRoute

I would like to propose to use my enRoute work as the theme for the hackaton (like baselining was last year). There are a number of issues I would like to have solved.

• Plugins & Templates should become fully repository based
• Templates should be executable by bndlib
• We should have a built-in Plugin repository which is provided by bndlib. It is currently a big mess, grown way beyond what it was supposed to do.
• Repositories should not hold any JARs. Github slows down commits with binaries and in general it is inefficient. All repos should hold a URL for their source and eagerly fetch their content.
• The content/scope of the repository must be under source control
• A central cache for the binaries that is forbidden to hold state. I.e. the presence or absence of an artifact must not indicate membership or convey other information noticeable in the build (except its potential delay downloading it).
• Editing the repository's scope can be done via editing a standardized text file or dragging/dropping URLs. These can be (non-file) urls to the resource or they can be specialized URLs that represents collections or specific artifacts.
• A service interface needs to be made to search a remote repository. This search should be requirement based, free text, per sha(s), and provide update information. This service interface must also support handling the specialized URLs that are dropped on the repository.
• We need to integrate bindex inside bnd. The Require/Provide Capability model has become such an intrinsic part of our model that the separation is hurting us. I looked at it (and already copied) the base classes in bndlib for indexing. I think this is very little work to port because bindex was based on similar classes as in bnd (it originally had the same source) except for one part (which stopped me). The current bindex has a built in heuristics database that provides Requirements and Provisions for resources that do not have any encoded (e.g. older extenders that do not provide an extender capability. Bindex can also accept an external file as well that I think we need to discuss. These heuristics are necessary but we need to see if this the right way. Or we make this db easier to share (via repos) or we provide fragments for these old/incompatible bundles.
• We need to clean up our offline build system act. See later.

Fortunately, I think I have all the parts working (like Baselining last year), it just needs integration with bnd(tools) and I need other people to start using it to get the right feedback.

Offline build

I wanted to setup gradle for the enRoute template so I had to dive into it again. And though it works well now thanks to Ferry's stubborn efforts despite us ignoring him. However, unfortunately I disagree with the philosophy, intentions, and some of its execution so we need to discuss this.

bnd is about the engine that built. This means that any function we offer should be represented in bnd and not in the offline build tool. I.e. the purpose of the build tool is to provide the user the capability to customize bnd.

bnd has a number of primitives. Primitives are not coupled (i.e. a build does not do a compile first). The following primitives are built into bnd:

• compile(test) — Compile the source code to the bin directory
• build(test) — Create any JAR files in the generated directory
• test — Start a framework, run OSGi test cases
• junit — Run JUNit testcases in the test directory
• release — Release the JAR files into the release repository
• debug — Provided extensive project setup information

I think the task of the offline build tool is to:

• Schedule the bnd primitives
• Execute a primitive safely, i.e. ensure that all its dependencies are available.
• Allow a custom action before and after the bnd primitive execution
• Allow custom actions to be added on workspace level
• Allow custom actions to be added on project level

The key quality measure for me is how small the setup files are, not how much functions they provide. Let me explain. Build tools are all about customizing. They are nothing more than a fancy script language that has some knowledge of dependencies. The whole purpose of the tool is to do YOUR thing in there. What was happening with the ant scripts, and what I feel happened with the gradle scripts, is that the script language is becoming the tool to create a tool, and most of the time the build language is a very lousy tool for that.

The whole idea of a script language is that you hardcode it to do what you want it to do. The idea of a updateable template is an anathema for me. If you select a build tool, you get a blank template that offers full support for the bnd primitives, nothing more, with clear markers where people could fill in their own tasks. This file is never updated by us after that. For us hard code programmers it always feels horrible to hard code stuff but that is the whole idea behind a script language. In compiled languages this used to be problematic because it was hard to change but with a script language it is easy to change, so please hard code stuff and thereby keep it simple.

The purpose of the workspace ant file was never to be an updateable template. It was meant to be the file that you customized.

The reason I am rather strong on this is that it just does not scale. The needs of people using this stuff is infinite. I.e. we support findbugs but not everybody wants that. It is easy to say that people do not have to use it but anything that is in the base, will be in the face of all those people trying to understand the build scripts. This is what creates complexity. The whole idea of build tools is is that you find snippets on the net that you then paste into your build file.

Another aspect is that however nice Gradle is, it is not the only tool out there. I think we must support the following tools:

• Gradle — New kid on the block, integrates well with bnd, Intellij effect
• Maven — By far the most popular, already installed
• Ant — Already installed, widely used, a bit tired

I would love to add good old make since it is still by far the easiest build tool I know :-)

So if we have multiple build tools we need a pane to select the correct template. The rules for a build tool should be:

• Each tool has one directory in cnf (i.e. cnf/gradle) for a (minimal) set of support files.
• 1 file in the root
• 1 file in cnf for workspace actions
• 1 file per project for specific project actions
• We never update any of those files unless they are identical to the previous version
• A checked out build should not automatically download the tool but should verify the version of the tool.
• There is a bnd plugin for each build tool that helps bndtools to create the appropriate files at workspace and project creation. I.e. it would be great if we could create workspaces and projects from the bnd command line.

So I like to include Gradle in the enRoute work but it will need to become closer to the previous description.