Felix GoGo Shell throws MalformedURLException: unknown protocol: bundleresource in Equinox

I am writing an extension to a Java application called ZAP. The extension is a fat/uber jar that has the Equinox bundle (org.eclipse.osgi) inlined. Additional bundles are also included in this jar, within a sub-directory bundles. The OSGi framework is being launched as outlined in How To Embed OSGi.

I'm trying to get Felix GoGo shell to work, following the steps outlined via Eclipse's "Console Shell" documentation. It specifies the following required bundles:

org.apache.felix.gogo.command
org.apache.felix.gogo.runtime
org.apache.felix.gogo.shell
org.eclipse.equinox.console

In addition to Equinox / org.eclipse.osgi. Since Equinox is not on the class path, I set org.osgi.framework.system.packages.extra configuration to the Export-Package value specified by Equinox's bundle, visible here.

Once the framework is in the ACTIVE state, I see the following stack trace in Eclipse (or via the command line outside of Eclipse):

Starting OSGi framework...
OSGi framework state: 32
gogo: MalformedURLException: unknown protocol: bundleresource
java.net.MalformedURLException: unknown protocol: bundleresource
  at java.net.URL.<init>(URL.java:593)
  at java.net.URL.<init>(URL.java:483)
  at java.net.URL.<init>(URL.java:432)
  at java.net.URI.toURL(URI.java:1089)
  at org.apache.felix.gogo.shell.Shell.readScript(Shell.java:209)
  at org.apache.felix.gogo.shell.Shell.source(Shell.java:192)
  at org.apache.felix.gogo.shell.Shell.gosh(Shell.java:109)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:497)
  at org.apache.felix.gogo.runtime.Reflective.invoke(Reflective.java:137)
  at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:82)
  at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:477)
  at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:403)
  at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)
  at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:183)
  at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:120)
  at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:89)
  at org.apache.felix.gogo.shell.Activator.run(Activator.java:75)
  at java.lang.Thread.run(Thread.java:745)

It seems the Equinox URL handlers are not being used by GoGo. I noticed the following packages related to them were not in Equinox's Export-Package:

org.eclipse.osgi.storage.url
org.eclipse.osgi.storage.url.bundleresource
org.eclipse.osgi.storage.url.bundleentry

So I also included those in the org.osgi.framework.system.packages.extra package list. That didn't help.

The bundles are installed and started without any obvious errors otherwise.

How do I make GoGo utilize Equinox's custom URL handlers?

Update 1: Maybe Red Herring

In random troubleshoot I wanted to execute the final shaded jar as outlined here. I copied both the original org.eclipse.osgi bundle and the shaded jar in the same directory. There's a configuration/config.ini specifying the bundles to load.

Executing the stock bundle brings up the console:

$ java -cp ~/.p2/pool/plugins/org.eclipse.osgi_3.11.0.v20160121-2005.jar org.eclipse.core.runtime.adaptor.EclipseStarter -console
osgi> ss
"Framework is launched."


id  State       Bundle
0 ACTIVE      org.eclipse.osgi_3.11.0.v20160121-2005
1 ACTIVE      org.apache.felix.gogo.runtime_0.10.0.v201209301036
2 ACTIVE      org.apache.felix.gogo.command_0.10.0.v201209301215
3 ACTIVE      org.apache.felix.gogo.shell_0.10.0.v201212101605
4 ACTIVE      org.eclipse.equinox.console_1.1.200.v20150929-1405
osgi> exit
Really want to stop Equinox? (y/n; default=y)

Executing the shaded jar, not so much:

java -cp semiotics-alpha-1.zap org.eclipse.core.runtime.adaptor.EclipseStarter -console
java.lang.NullPointerException: A null service reference is not allowed.
  at org.eclipse.osgi.internal.framework.BundleContextImpl.getService(BundleContextImpl.java:617)
  at org.eclipse.core.runtime.adaptor.EclipseStarter.startup(EclipseStarter.java:299)
  at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:231)
  at org.eclipse.core.runtime.adaptor.EclipseStarter.main(EclipseStarter.java:208)

This could be a total red herring... I'm very ignorant on OSGi, this being my first hands-on experience with it. However, I would expect the shaded jar to execute no differently than the stock Equinox jar. Since I don't understand why there's a difference I'm gonna dig there for lack of any better spots to dig...

Update 2: Equinox on Classpath Works Around Issue

I added the Equinox bundle to ZAP's classpath via its lib directory and removed it from being inlined into the extension jar. Doing this and adjusting the ServiceLoader.load call to not include a class loader (defaulting to system class loader) works:

Starting bundle: org.apache.felix.gogo.shell_0.10.0.v201212101605 [1]
Starting bundle: org.eclipse.equinox.console_1.1.100.v20141023-1406 [2]
Starting bundle: org.apache.felix.gogo.command_0.10.0.v201209301215 [3]
Starting bundle: org.eclipse.emf.ecore_2.11.2.v20160208-0816 [4]
Starting bundle: org.eclipse.emf.common_2.11.1.v20160208-0816 [5]
Starting bundle: org.apache.felix.gogo.runtime_0.10.0.v201209301036 [6]
osgi> ss
"Framework is launched."


id  State       Bundle
0 ACTIVE      org.eclipse.osgi_3.11.0.v20160121-2005
1 ACTIVE      org.apache.felix.gogo.shell_0.10.0.v201212101605
2 ACTIVE      org.eclipse.equinox.console_1.1.100.v20141023-1406
3 ACTIVE      org.apache.felix.gogo.command_0.10.0.v201209301215
4 ACTIVE      org.eclipse.emf.ecore_2.11.2.v20160208-0816
5 ACTIVE      org.eclipse.emf.common_2.11.1.v20160208-0816
6 ACTIVE      org.apache.felix.gogo.runtime_0.10.0.v201209301036
osgi>

The underlying issue might be how the jar is being inlined, possibly related to the MANIFEST.MF file. Possibly related, I now don't need to supply a value for org.osgi.framework.system.packages.extra.

I'm going to leave this question unanswered since the original point was to include the OSGi library as a shaded artifact. Maybe bndtools can help?

Solution

OK, it seems the lack of a descriptive META-INF/MANIFEST.MF was the culprit:

Removed (jar uM...) then updated (jar um...) the shaded jar's MANIFEST.MF to be a copy of Equinox's manifest, modifying the Bundle-Version number to end in 1766 versus 1700.
Removed Equinox from ZAP's classpath

Then I loaded the extension as is. GoGo shell starts:

osgi> 13177 [AWT-EventQueue-0] INFO org.parosproxy.paros.control.Control  - New Session
ss
"Framework is launched."


id  State       Bundle
0 ACTIVE      org.eclipse.osgi_3.10.102.v20160118-1766
1 ACTIVE      org.apache.felix.gogo.runtime_0.10.0.v201209301036
2 ACTIVE      org.eclipse.emf.common_2.11.1.v20160208-0816
3 ACTIVE      org.apache.felix.gogo.shell_0.10.0.v201212101605
4 ACTIVE      org.eclipse.equinox.console_1.1.100.v20141023-1406
5 ACTIVE      org.apache.felix.gogo.command_0.10.0.v201209301215
6 ACTIVE      org.eclipse.emf.ecore_2.11.2.v20160208-0816
osgi>

I see the system bundle ends in the same modified version number, so it's honoring the added MANIFEST.MF. So the underlying issue was an incomplete system bundle manifest. Copying an existing one (Equinox's) works well enough. It also works for a straight commandline test:

$ java -cp semiotics-alpha-1.zap org.eclipse.core.runtime.adaptor.EclipseStarter -console
osgi> exit
Really want to stop Equinox? (y/n; default=y)  y

Dunno if I'll use this approach or not. At least I know what I need to do if I want to use this approach.