Support for OSGi bundles from a Maven repository

[ghost]

Hi, I've recently started building an RCP application using Tycho. Right now, I'm still using a p2 repository for deployment but in the future, I would like to switch to a Maven repository instead.

The problem is that the application is split into multiple components, which are built independently and which depend on one another. Right now this is solved by adding the required repositories to the root pom such that the Manifest dependencies can be properly resolved.

<repositories>
	<repository>
		<id>...</id>
		<url>...</url>
		<layout>p2</layout>
	</repository>
</repositories>

However, this approach only works if the repository has a p2 layout.

One way to fix this is deploy the bundles to a Maven repository and then add them either to the target platform definition using the m2e integration or to the Maven dependencies in the root pom. But this requires that I add each plugin individually which, given the scope of the project, is just not feasible.

I think it would be beneficial if there is an easier way to include those artifacts and to further close the gap between Maven and Eclipse.

A solution could be to attach the dependencies to the pom.xml file and include it in the OSGi bundle during the packaging phase. This way, transitive dependencies are automatically included as well.

In this case, I could them collect the plugins in a feature, which is then added as a dependency. This would make maintaining the project a lot easier and also provides users with an easy way to add OSGi bundles to a pure Maven project.

[laeubi]

I'm not quite sure what is suggested here, you can have "pom" features (that is a pom grouping different dependencies together) and include this in your target or reference it via maven.

Maybe an example project would help to understand the use-case here.

[vogella]

@laeubi I think this one is about providing Tycho build artifacts via a Maven repo using pomless builds.

1.) Tycho builds a bunch of plug-ins and features
2.) These should be pushed to a Maven repo, e.g. Maven central.

How can the published Maven artifacts (created via Tycho) describe their requirements, e.g. how can a published feature tell Maven which components it includes?

[laeubi]

As tycho is a regular maven extension you can deploy any artifact to a maven repository you like (this is independent from pomless extension).

A feature is also a regular artifact and could be deployed as such, I just don't know of any project actually publishing features so there is not much support in consuming such features in tycho/m2e.

Maybe a p2-maven site would fulfill the requirements described here but as mentioned this is rather unclear to me what is actually expected here as one can't consume a "maven repository" as a whole nexus supports OSGi Bundle Repositories what would be a possible source for dependencies see for example #34

[joeshannon]

They can be deployed to a Maven repository but they do not have the sufficient Maven metadata to be used in standalone Maven projects for example if the Tycho build has bundles A and B, and B depends on A, there is no dependency info in the generated pom.xml (e.g. included in META-INF/maven/<groupId>/<artifactId>/pom.xml) for B to express this.

I have noticed that the Eclipse platform itself does achieve this, e.g. looking at the dependencies of org.eclipse.core.resources on Maven Central, what mechanism is used there to add this dependency info?

[mickaelistria]

I think if you need to consume artifacts as Maven artifacts, you should develop them as Maven artifacts: pom-first, using bnd or Felix. The thing is that there are now plenty good ways to consume Maven artifacts in p2/Tycho world, but there is about nothing to do it well the other way round (consume p2 artifacts in Maven). So Maven-first would still works much better here.
I imagine there could be some possible ways to implement a p2->m2 translation, but it would be a lot of work; while m2->p2 translation is already here and in good shape...

[laeubi]

The problem is that a bundle does not have "dependencies" as in the maven world, it has Capabilities but these are not necessarily fulfilled by one specific item nor by an artifact at all (e.g. a bundle might require a service) and even if by multiple ones (e.g. multiple bundles can provide the same service).

All of this could not reliable be mapped to maven, even if, not all OSGi bundles are maven artifacts.

Tycho generates appropriate dependencies for reactor projects but this is just a snapshot of the build and does not strictly resembles what will be resolved at runtime.

[ghost]

Thank you for all the responses.

I apologize for the poor explanation, so I have created a simple example to help me illustrate my use case.
I have a (pomless) project consisting of 2 plugins and 1 feature which I build using Tycho and then upload to a m2 repository. If I now want to use classes from this project in a separate project, then I have to add the required plugins to the target platform. E.g. by adding following entry, with the groupId derived from the parent pom and the artifact id derived from the manifest:

<dependency>
    <groupId>example</groupId>
    <artifactId>example.project2</artifactId>
    <version>1.0.0-SNAPSHOT</version>
</dependency>

This is something Tycho already supports and which works fine, if it is just a couple of plugins. But in my situation, I would have to add 400-500 plugins like this, which is just not maintainable. I also have to manually add all of their required plugins, as well.

My idea is to generate a pom file together with the artifacts, like @joeshannon mentioned, so that both can be deployed to a m2 repository. For example, example.plugin2 depends on example.plugin1 and therefore its pom may look something like this:

<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<modelVersion>4.0.0</modelVersion>
	<groupId>example</groupId>
	<artifactId>example.plugin2</artifactId>
	<version>1.0.0-SNAPSHOT</version>
	
	<dependencies>
		<dependency>
			<groupId>example</groupId>
			<artifactId>example.plugin1</artifactId>
			<version>1.0.0-SNAPSHOT</version>
		</dependency>
	</dependencies>
</project>

If I now add example.plugin2 to the target platform, both this plugin as well as its dependency will be available to the application. Furthermore, the same logic could be applied for features, which could either contain other features or the plugins directly. All of which are included automatically.
Because I'm using a feature-based application, this would mean that I only have to maintain its features, which would drastically reduce the amount of work that has to be put into it, to keep everything up-to-date.

@laeubi

The problem is that a bundle does not have "dependencies" as in the maven world, it has Capabilities but these are not necessarily fulfilled by one specific item nor by an artifact at all (e.g. a bundle might require a service) and even if by multiple ones (e.g. multiple bundles can provide the same service).

Couldn't this issue be circumvented by adding those plugins as "Required Bundles"? This functionality is only relevant if a developer wants to make an OSGi plugin available to the Maven world. So I would argue that it's their responsibility to make sure that the plugin is in a format in which the required dependencies can be determined.

All of this could not reliable be mapped to maven, even if, not all OSGi bundles are maven artifacts.

That is true. And I'm not exactly sure how one would go about finding a matching Maven artifact for an OSGi bundle.

Tycho generates appropriate dependencies for reactor projects but this is just a snapshot of the build and does not strictly resembles what will be resolved at runtime.

At least for my use case, the compile dependencies of the individual plugins are more than sufficient. And I think the generated dependencies are a good starting point.

[laeubi]

If I run mvn dependency:list on the example.plugin2 i get:

[INFO] --- maven-dependency-plugin:2.8:list (default-cli) @ example.plugin2 ---
[INFO] 
[INFO] The following files have been resolved:
[INFO]    p2.eclipse-plugin:example.plugin1:jar:1.0.0.202112151602:system
[INFO] 

what is good, but this info seem not to end up in the pom.xml in the target folder. @mickaelistria is this expected (maybe because it is system scoped)?

Interestingly for example.feature i get an empty dependency list, again I think @mickaelistria can best explain if this is to be expected?

I think we can lower the pain here to have at least those items mentioned that are resolved in the current reactor and already listed? We should also be able to add the desired dependency info for:

1. Items originating from pom locations
2. Items that have p2 data indicating the artifact.id/group.id
3. Items from the current reactor

@mickaelistria what do you think?

[mickaelistria]

this info seem not to end up in the pom.xml in the target folder. @mickaelistria is this expected (maybe because it is system scoped)?

There is no strong reason against adding it to the target pom. I imagine it was not implemented so far because it was not valuable enough for active contirbutors.
But the question becomes what would a dependency against p2.eclipse-plugin:example.plugin1:jar:1.0.0.202112151602:system add as benefit? It wouldn't be possible to consume it with plain Maven anyway because those coordinates are not Maven coordinates, so this location cannot be resolved with plain Maven. So if coordinates are not usable, it doesn't seem worth the effort of adding them.
What would be interesting, although maybe very tricky, would be to use the Maven artifact coordinates in place of p2-installable-unit when this jar can be identified as a Maven artifacts. That can typically be deduced by the maven-groupId, maven-artifactId, maven-version & maven-classifier properties in artifact metadata.
IMO, resolving dependencies using regular Maven coordinates would be the key for stronger integration with Maven.

Interestingly for example.feature i get an empty dependency list

That would be worth a separate bug/discussion. I have no idea why or why not things are implemented that way; but it's worth discussing the point and re-evaluate the pros/cons of exposing feature deps to Maven model (feature deps are computed in Tycho, so exposing them to Maven model wouldn't be too hard).

[laeubi]

It wouldn't be possible to consume it with plain Maven anyway because those coordinates are not Maven coordinates

I more would expect that this is transformed to regular maven coordinates here in this case, maybe this would be the first step to use the regular maven GAV if possible at this place.

I have no idea why or why not things are implemented that way

I have just wondered because I thought that these "Dependecies" where used to calculate the build order but it seems there are some other factor that influence the outcome of the order.

resolving dependencies using regular Maven coordinates would be the key for stronger integration with Maven

I agree and now better understand the issue described here even though I thought tycho would already support this to some degree.

I'll take a look at this but don't expect it to be on the top of my list as I'm currently have no use for this kind of feature an none of my customers demanding it. Anyways, if anyone has a strong business-case and likes to sponsor this feature let me know so I can give higher priority on this.

[mickaelistria]

I didn't verify (yet?), but I don't think there is a strong requirement on Maven coordinates being of a form or another once dependencies are computed, so a 1st iteration could be to change to Maven coordinates for deps where possible but keep the system scope. I think it would bring some better integration, without disrupting too much how Tycho works.
If this work, we can more easily consider adding the dependencies to target pom file; and also more easily consider getting rid of the system scope and specific path when we deps pre-exist as regular Maven artifacts.

However, I still believe that a Nexus plugin that looks up all artifacts that have p2 or OSGi metadata and generates a p2 repo fron that p2-able content in the Maven repo would be a solution here; and a correct one (after all, a repository manager that's used to manage repositories doesn't seem like a too disruptive idea).

[mickaelistria]

There is a specific issue on the topic of streamlining dependency coordinates: #443

[vogella]

I like this feature request.

Ultimately this feature would alllow a "pure" Maven dependency flow (and remove the need for p2 repositories) for Eclipse based development.

Tycho builds plug-ins as Maven libs -> they are published to a Maven repo and used by Tycho in the next build via the Maven based Target platform.