Easye/osgi/20211110a

[easye]

@alanruttenberg This is my re-working of your changes for OSGI et. al. that avoids using CL-USER, and doesn't have ABCL-ASDF extend the JSS namespace.

When I try to run the test asdf definition included in the comment to the second patch I get the following bundle resolution problem. Any ideas on what this would be about?

Java exception 'org.osgi.framework.BundleException: Unable to resolve uk.ac.manchester.cs.owl.factplusplus [1](R 1.0): missing requirement [uk.ac.manchester.cs.owl.factplusplus [1](R 1.0)] osgi.native; (|(&(|(osgi.native.osname~=win)(osgi.native.osname~=windowsnt)(osgi.native.osname~=windows nt)(osgi.native.osname~=winnt)(osgi.native.osname~=windowsxp)(osgi.native.osname~=windows xp)(osgi.native.osname~=winxp)(osgi.native.osname~=windowsvista)(osgi.native.osname~=windows vista)(osgi.native.osname~=windows2003)(osgi.native.osname~=windows2008)(osgi.native.osname~=windows7)(osgi.native.osname~=windows 7)(osgi.native.osname~=windows8)(osgi.native.osname~=windows 8)(osgi.native.osname~=windows8.1)(osgi.native.osname~=windows 8.1))(|(osgi.native.processor~=x86_64)(osgi.native.processor~=amd64)))(&(|(osgi.native.osname~=win)(osgi.native.osname~=windows95)(osgi.native.osname~=windows 95)(osgi.native.osname~=win95)(osgi.native.osname~=windows98)(osgi.native.osname~=windows 98)(osgi.native.osname~=win98)(osgi.native.osname~=windowsnt)(osgi.native.osname~=windows nt)(osgi.native.osname~=winnt)(osgi.native.osname~=windowsce)(osgi.native.osname~=winndows ce)(osgi.native.osname~=wince)(osgi.native.osname~=windowsxp)(osgi.native.osname~=windows xp)(osgi.native.osname~=winxp)(osgi.native.osname~=windowsvista)(osgi.native.osname~=windows vista)(osgi.native.osname~=windows2003)(osgi.native.osname~=windows2008)(osgi.native.osname~=windows7)(osgi.native.osname~=windows 7)(osgi.native.osname~=windows8)(osgi.native.osname~=windows 8)(osgi.native.osname~=windows8.1)(osgi.native.osname~=windows 8.1))(|(osgi.native.processor~=i386)(osgi.native.processor~=x86)))(&(osgi.native.osname~=linux)(|(osgi.native.processor~=x86_64)(osgi.native.processor~=amd64)))(&(osgi.native.osname~=linux)(osgi.native.processor~=i386))(&(|(osgi.native.osname~=macosx)(osgi.native.osname~=mac os x))(|(osgi.native.processor~=ppc64)(osgi.native.processor~=x86_64)))(&(|(osgi.native.osname~=macosx)(osgi.native.osname~=mac os x))(|(osgi.native.processor~=ppc)(osgi.native.processor~=i386)))) Unresolved requirements: [[uk.ac.manchester.cs.owl.factplusplus [1](R 1.0)] osgi.native; (|(&(|(osgi.native.osname~=win)(osgi.native.osname~=windowsnt)(osgi.native.osname~=windows nt)(osgi.native.osname~=winnt)(osgi.native.osname~=windowsxp)(osgi.native.osname~=windows xp)(osgi.native.osname~=winxp)(osgi.native.osname~=windowsvista)(osgi.native.osname~=windows vista)(osgi.native.osname~=windows2003)(osgi.native.osname~=windows2008)(osgi.native.osname~=windows7)(osgi.native.osname~=windows 7)(osgi.native.osname~=windows8)(osgi.native.osname~=windows 8)(osgi.native.osname~=windows8.1)(osgi.native.osname~=windows 8.1))(|(osgi.native.processor~=x86_64)(osgi.native.processor~=amd64)))(&(|(osgi.native.osname~=win)(osgi.native.osname~=windows95)(osgi.native.osname~=windows 95)(osgi.native.osname~=win95)(osgi.native.osname~=windows98)(osgi.native.osname~=windows 98)(osgi.native.osname~=win98)(osgi.native.osname~=windowsnt)(osgi.native.osname~=windows nt)(osgi.native.osname~=winnt)(osgi.native.osname~=windowsce)(osgi.native.osname~=winndows ce)(osgi.native.osname~=wince)(osgi.native.osname~=windowsxp)(osgi.native.osname~=windows xp)(osgi.native.osname~=winxp)(osgi.native.osname~=windowsvista)(osgi.native.osname~=windows vista)(osgi.native.osname~=windows2003)(osgi.native.osname~=windows2008)(osgi.native.osname~=windows7)(osgi.native.osname~=windows 7)(osgi.native.osname~=windows8)(osgi.native.osname~=windows 8)(osgi.native.osname~=windows8.1)(osgi.native.osname~=windows 8.1))(|(osgi.native.processor~=i386)(osgi.native.processor~=x86)))(&(osgi.native.osname~=linux)(|(osgi.native.processor~=x86_64)(osgi.native.processor~=amd64)))(&(osgi.native.osname~=linux)(osgi.native.processor~=i386))(&(|(osgi.native.osname~=macosx)(osgi.native.osname~=mac os x))(|(osgi.native.processor~=ppc64)(osgi.native.processor~=x86_64)))(&(|(osgi.native.osname~=macosx)(osgi.native.osname~=mac os x))(|(osgi.native.processor~=ppc)(osgi.native.processor~=i386))))]'.

[alanruttenberg]

[edited as I had the comparison backwards]
Why add all the explicit package prefixes instead of importing the symbols into :abcl-asdf/osgi?

Regarding the exception, did you get the bundle and put it in the same directory as the sysdef? It is at https://github.com/alanruttenberg/lsw2/blob/owlapiv4/owl2/lib/uk.ac.manchester.cs.owl.factplusplus-1.6.5.jar

[easye]

Why add all the explicit package prefixes instead of importing the symbols into :abcl-asdf/osgi?

That's just my convention for coding, as it makes me more aware of the package abstraction boundaries.

[easye]

Regarding the exception, did you get the bundle and put it in the same directory as the sysdef?

Yes, I did. There seems to be something missing in the dependency graph, or maybe it is me using java17. Or maybe it is because I am running on an arm64 Mac for which there isn't a native library. Does the factplusplus thunk down to native code?

[alanruttenberg]

Yes, I did. There seems to be something missing in the dependency graph, or maybe it is me using java17. Or maybe it is because I am running on an arm64 Mac for which there isn't a native library. Does the factplusplus thunk down to native code?

If I had to guess, I'd blame java17. I've only tested on 8 and 11. I can't use java17 until I figure out how to get around the restrictions on reflection. I don't know what the status of OSGI in 17 is.

One of the benefits of using the bundle is that it deals with packaged native libraries, unpacking the right ones for the architecture into the felix cache and arranging for them to be visible where necessary. I don't have experience with the arm machines but aren't they supposed to emulate x64?

Regarding the removal of the hook, all of the bootdelegation and system-packages for all bundles have to be added to *osgi-configuration* before OSGI is initialized and the first bundle added, but after the OSGI code is loaded. Even with the hook, though, I realize that's a problem as one sysdef might load a bundle, initializing OSGI before some other sysdef gets loaded and needs to add something to the config as well. My hook doesn't work for that unless all systems that are going to use bundles are defined before any of them is loaded. Assuming abcl-asdf is loaded by the time my system definitions are means I can execute the config change immediately - the hook doesn't help. If there is more than one system with different needs then I guess the configuration needs to go into .abclrc, or all the system definitions that care need to be loaded before any of the systems are.

There's one more thing I noticed that I thought I should bring to your attention, which is the OSGI cache location, which defaults to ~/abcl-felix-cache

• I'm not sure whether that directory needs to exist before bundles are loaded and I don't create it in my code. I guess it should be created if absent in ensure-osgi-initialized.
• I'm not sure whether there is somewhere else that would be preferable to have the default location be.

Also just noticed that the doc mentions ensure-osgi-framework instead of ensure-osgi-initialized.

Re:package abstraction boundaries. Seems a little schizo to, on the one hand go granular on package definitions, but on the other hand not take advantage of all of what the package system brings, such as imports...

[alanruttenberg]

In case it matters:

CLASSPATH=/Users/alanr/repos/lsw2/lib/abcl-aio.jar:/Users/alanr/repos/lsw2/lib/postgresql-9.3-1101.jdbc41.jar:/Users/alanr/repos/lsw2/lib/Access_JDBC30.jar:/Users/alanr/repos/lsw2/lib/virtjdbc3.jar:/Users/alanr/repos/lsw2/lib/script-api.jar:/Users/alanr/repos/lsw2/lib/sqlite-jdbc-3.7.2.jar:/Users/alanr/repos/lsw2/lib/mysql-connector-java-5.1.31-bin.jar:/Users/alanr/repos/lsw2/lib/httpserver.jar:/Users/alanr/repos/lsw2/lib/sqljdbc4.jar:/Users/alanr/repos/lsw2/lib/ojdbc7.jar:/Users/alanr/repos/lsw2/lib/asm-all-3.1.jar

[easye]

I have your example working better under x86_64-linux, as apparently arm64-macos and x86_64-freebsd are too exotic of architectures to run the factpluplus JNI libraries included in the jar.

In any advent, this branch doesn't quite work as jss:maybe-resolve-class-against-imports and jss:japropos need to reference the list of loaded OSGI bundles which is now available as abcl-asdf/osgi:*loaded-osgi-bundles*.

We could just redefine these functions when asdf-osgi-bundle is loaded, but I'd rather cleanly abstract OSGI as an additional location for the JSS code to search somehow. Something like a hook I guess, but I feel like we should somehow express japropos and maybe-resolve-class-against-imports would be better rewritten based on a more primitive function that we could somehow have hooks for classname search strategies. Any advice on how this might look would be appreciated.

[alanruttenberg]

They do. Not sure what's happening on your side...

(jss::japropos 'factplusplus)
->
uk.ac.manchester.cs.factplusplus.AxiomPointer: Java Class, Bundle: uk.ac.manchester.cs.owl.factplusplus
uk.ac.manchester.cs.factplusplus.ClassPointer: Java Class, Bundle: uk.ac.manchester.cs.owl.factplusplus
uk.ac.manchester.cs.factplusplus.DataPropertyPointer: Java Class, Bundle: uk.ac.manchester.cs.owl.factplusplus
...

(jss::maybe-resolve-class-against-imports 'FaCTPlusPlusRenderingViewComponent)
->
(#<org.apache.felix.framework.BundleImpl uk.ac.manchester.cs.owl.factplusplus [1] {624A21E3}> "uk.ac.manchester.cs.factplusplus.protege.FaCTPlusPlusRenderingViewComponent")

[easye]

They do. Not sure what's happening on your side...

I'm not sure which question that you are responding to here, but there is no way this current patch works due to the absence of the jss:*loaded-osgi-bundles* symbol referred to in invoke.lisp at https://github.com/armedbear/abcl/pull/412/files#diff-fe4a97fadd57be689031c9a7f8e5dcd173c4585611ab297be1655313c2b10318R152 and https://github.com/armedbear/abcl/pull/412/files#diff-fe4a97fadd57be689031c9a7f8e5dcd173c4585611ab297be1655313c2b10318R588

[alanruttenberg]

OK, we're not looking at the same thing. Even in your branch, screenshot attached

.

[alanruttenberg]

Have a look at https://github.com/easye/abcl/blob/easye/osgi/20211110a/contrib/jss/invoke.lisp and search for `*loaded-osgi-bundles*
It's the same version I have except for a couple of bits unrelated to OSGI

[alanruttenberg]

I went to both the links you have in the previous messages and successfully searched for *loaded-osgi-bundles*

[easye]

I went to both the links you have in the previous messages and successfully searched for loaded-osgi-bundles

Yes, but they refer to the symbol is in the jss package, whereas in my branch I am trying to have osgi.lisp define this symbol in the abcl-asdf/osgi package. Hence my question about the best abstraction to have the abcl-asdf-osgi system add functionality to JSS.

[alanruttenberg]

If OSGI is loaded as part of ABCL-ASDF, then the symbols are available and should just be imported into the jss package. Yes we could write something more general, but I don't see the benefit of doing this. Maybe if we land up with a 3rd or 4th mechanism for getting handles on classes it's then worthwhile to find a general solution.

[alanruttenberg]

If OSGI isn't necessarily loaded, at least define its package so the symbols can be shared, and then, if JSS is going to be used without OSGI, protect its use with a boundp check.

[alanruttenberg]

Or, put the shared symbols in a neutral package like ... cl-user.