Managing OSGi Dependency Hell

Question

UPDATE 2: Since my blog is a bit dead the links got degraded so you can view the articles here:

https://www.dropbox.com/s/xvobgzqnl43kcda/Managing_OSGi_Transitive_Dependencies__Part_1____CitizenRandom.pdf?dl=0

https://www.dropbox.com/s/0bdooux4yhrf8lf/Managing%20OSGi%20Transitive%20Dependencies%20%28...pdf?dl=0

https://www.dropbox.com/s/km3mxqah6oy23iq/Why%20using%20Require-Bundle%20is%20a%20bad%20pract...pdf?dl=0

https://www.dropbox.com/s/mtenchtjopcrmr8/How%20many%20ways%20can%20we%20import%20bundles%20in%20OSGi_%20_%20CitizenRandom.pdf?dl=0

https://www.dropbox.com/s/sldxynx3fl8vn61/Managing%20OSGi%20Transitive%20Dependencies%20%282...pdf?dl=0

I have a maven project, using the very famous felix maven bundle plugin configured in my POM.XML in the following way:

<packaging>bundle</packaging>

(...)

<plugin>
            <groupId>org.apache.felix</groupId>
            <artifactId>maven-bundle-plugin</artifactId>
            <extensions>true</extensions>
            <configuration>
                <instructions>
                    <Bundle-SymbolicName>${project.artifactId};singleton:=true</Bundle-SymbolicName>
                    <Bundle-Version>${project.version}</Bundle-Version>
                    <Export-Package>jlifx.*</Export-Package>
                    <!-- <Embed-Dependency>*</Embed-Dependency> -->
                </instructions>
            </configuration>
        </plugin>

And then I have some 1st degree/level dependencies included in my POM as well:

<dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.11</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-lang3</artifactId>
        <version>3.1</version>
    </dependency>
    <dependency>
        <groupId>commons-io</groupId>
        <artifactId>commons-io</artifactId>
        <version>2.4</version>
    </dependency>
    <dependency>
        <groupId>commons-logging</groupId>
        <artifactId>commons-logging</artifactId>
        <version>1.1</version>
    </dependency>
    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-all</artifactId>
        <version>5.0.0.Alpha1</version>
    </dependency>
</dependencies>

And now my problem begins... If I do mvn install I'll get my bundle well built with its very great MANIFEST.MF and all, but I won't get the other dependencies bundles, meaning that if I grab my bundle file and drop it on my OSGi framework instance I'll get something like "Unable to resolve 1.0: missing requirement [1.0] osgi.wiring.package; (&(osgi.wiring.package= etc..."

So one way I've found to create my 1st level dependencies' bundles was by creating a profile in my POM like this:

<profiles>
    <!-- http://www.lucamasini.net/Home/osgi-with-felix/creating-osgi-bundles-of-your-maven-dependencies -->
    <!-- -Pcreate-osgi-bundles-from-dependencies bundle:wrap -->
    <profile>
        <id>create-osgi-bundles-from-dependencies</id>
        <build>
            <directory>${basedir}/bundles</directory>
            <plugins>
                <plugin>
                    <groupId>org.apache.felix</groupId>
                    <artifactId>maven-bundle-plugin</artifactId>
                    <version>2.0.1</version>
                    <extensions>true</extensions>
                    <executions>
                        <execution>
                            <id>wrap-my-dependency</id>
                            <goals>
                                <goal>wrap</goal>
                            </goals>
                            <configuration>
                                <wrapImportPackage>;</wrapImportPackage>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

And this way when I execute mvn -Pcreate-osgi-bundles-from-dependencies bundle:wrap I'll get the bundles, well constructed and working. But, here comes the real deal. Those bundles also have dependencies on their own, so they'll need to have their dependencies wrapped around as bundles. According to numerous webpages, long time ago we had the mvn org.apache.felix:maven-bundle-plugin:bundleall goal to do that for us, but I've tried, it is buggy and return exceptions and it is marked as deprecated, according to Stuart it will be removed in maven 2.4.1 and later (ref: https://issues.apache.org/jira/browse/FELIX-4145).

So my only solution now is to, manually, check each manifest of my 1st level dependencies and go google for the jars containing the required packages, add them to my POM.XML as maven dependencies, and then run mvn -Pcreate-osgi-bundles-from-dependencies bundle:wrap to wrap them as bundles.

This is what is known as the dependency hell...

Is there any way to automate this task of solving nth level dependencies for a maven-bundle osgi project? I.e. make maven study each of my 1st level dependencies' manifest file, read the import packages, look in the central repo for jar's that provide such packages, download them and wrap them as bundles?

Note: Please provide detailed instructions on how to achieve this, don't just link to this tool or that tool that may solve this issue. The major problem with these tools is their lack of examples and documentation. For example bundleall is deprecated, but there seems to be no tool to replace it, at least in their official documentation of maven bundle plugin, which is deprecated so far... I'm certain that I may have crossed with tools capable of doing this, but the lack of documentation forbids the newbie user from knowing that...

Thanks!

---

---

---

---

EDIT-1:

Thank you for your answers so far :) I think I didn't explain my situation in the most proper way and I'm feeling some difficulties doing it just by plain text. Or maybe I din't understand your answer. I'm quite "fresh-new" in OSGi and the only knowledge I have comes from books (OSGi in Action and alike) and from Google.

Imagine that my bundle imports packages A and B. However package A imports package C, and package B imports C also. But now C imports packages D, E, F and G. On the other hand package D imports a ton of other packages and so does E, F and G.

The only bundles I have in my computer are my own bundle and the bundles that provide the packages A and B because they are my 1st level dependencies. However I don't have any of the other required bundles, even if they are installed as jars in my JDK installation folder, I don't have them as bundles, nor even do I know where can I get the jars to wrap them (actually I know but lets imagine I don't).

What I would expect the build tool to do was to run an algorithm similar to the following:

1) Go to my bundle MANIFEST.MF and read the Import-Package field. Enumerate all the required packages and their resp. versions.

2) Search somewhere on the Internet for the jars or bundles of my required libraries.

3) Download each one and check if they are just plain jars or have a valid osgi manifest file (i.e. they are a bundle)

3.1) if they are a bundle, copy them to my bundles/ folder.

3.2) else wrap the jar into a bundle using whatever tool to do that and copy the bundle to my bundle/ folder.

4) Now, for each new bundle downloaded/created repeat the steps 1),2),3) and 4).

What I want as the final result: A bundle for each library to which I have a direct or indirect dependency, so that I can install them on-the-fly in my OSGi Framework instance, such as felix or equinox.

What I don't want:

1) Have to do this manually, because if I try to solve each dependency of dependency I may spent hours or days collecting and wrapping jars.

2) Embed all dependencies into a ubber/mega bundle. That's a bad practice according to several books I've read, it is harder to maintain each dependency individually and also, it corrupts modularity.

Note: my problem is not about the particular task of wrapping a jar into a bundle, but about doing it recursively to each bundle's imports, even if they need to be downloaded from an online repository such as maven's central.

Is there a way to do this automatically or I'm missing something very big about OSGi? So big that I would never need to do this that I'm asking for?

EDIT-2:

Some, if not all, Import-Package dependencies could be solved at run time. Imagine the OSGi Framework trying to start a bundle, but instead of showing the error message of "Unable to resolve 8.0: missing requirement [8.0] osgi.wiring.package;" it would search for that package online, download it and install it on-the-fly. Life would be so much easier.

Answer 1

If you want to wrap jars that not are bundles but just need these jars to be bundles as a librarys in OSGi framework you can wrap it with BND tool. See the link How to create/generate OSGi bundles from existing third-party jars?

But I can see you use very common libaries than already has been converted as osgi bundles. You can find a lot of libraries converted to bundles in the SpringSource Enterprise Bundle Repository.

But if you can't find any library that is not converted to a good OSGi bundle, you can use WRAP protocol from PAX-URL to install these dependencies in OSGi framework. To use these protocol depends on the OSGi framework that you are using but in apache karaf is installed by default. For example, to install a library from maven repository:

root@karaf> osgi:install -s 'wrap:mvn:commons-lang/commons-lang/2.4$Bundle-SymbolicName=commons-lang&Bundle-Version=2.4'

This instruction install the commons-lang library from maven repository into OSGi framework and wrap it as OSGi bundle with the headers appear in the line.

For automatic install dependencies like you say in the second edit, there are several solutions but is a little work. There are two main solutions for provisioning automatically bundles to OSGi framewor, the "Felix OBR repository" and "Felix Provisioning" bundles and Equinox p2 repository. Both of them have console commands to automatically install bundles and features. The problem is that actually I can't find a good public repository of bundles. You need to build your own repository with all bundles than you need.

If you use the maven-bundle-plugin, when you install artifacts into your local maven repository, the plugin update the file "repository.xml" located in the root of your repository to reflect the requires and capabilities of your bundles. This file is a OBR repository file.