I have a relatively high volume/requests system in which we are using OSGi. We get close to 800M requests per day.

We are seeing some issues currently where threads are getting blocked. For every request that comes in, we forward the event/data to an osgi bundle using registerService to pass the payload/data to the OSGi bundle which is listening to this service.

Like this: bundleContext.registerService(Map.class.getName(), dataHolderMap, null);

Where dataHolderMap is nothing but a regular java hashmap

Here is the threaddump using JStack:

===========================================================================

"RequestThread" prio=10 tid=0x00000000421ab800 nid=0x1042 runnable [0x00007fbdd3867000] java.lang.Thread.State: RUNNABLE at org.apache.felix.framework.ServiceRegistry.getService(ServiceRegistry.java:295) - locked <0x0000000700e2c590> (a org.apache.felix.framework.ServiceRegistry) at org.apache.felix.framework.Felix.getService(Felix.java:3568) at org.apache.felix.framework.BundleContextImpl.getService(BundleContextImpl.java:468) at org.osgi.util.tracker.ServiceTracker.addingService(ServiceTracker.java:411) at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:932) at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:864) at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256) at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:229) at org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:894) at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:932) at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:793) at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:543) at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4419) at org.apache.felix.framework.Felix.registerService(Felix.java:3423) at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) at com.mypackage.person.bs.processor.Processor.sendEvent(Processor.java:56) at com.mypackage.jetMyStream.event.support.AbstractEventSource.fireSendEvent(AbstractEventSource.java:97) at com.mypackage.jetMyStream.event.channel.messaging.InboundMessagingChannel.fireEvent(InboundMessagingChannel.java:113) at com.mypackage.jetMyStream.event.channel.messaging.InboundMessagingChannel.onMessage(InboundMessagingChannel.java:204) at com.mypackage.jetMyStream.messaging.MessageService.dispatchMessageForContext(MessageService.java:349) at com.mypackage.jetMyStream.messaging.MessageService.dispatch(MessageService.java:259) at com.mypackage.jetMyStream.messaging.MessageServiceRequest.execute(MessageServiceRequest.java:40) at com.mypackage.jetMyStream.util.RequestThread.run(RequestThread.java:69)

"RequestThread" prio=10 tid=0x00000000425f8800 nid=0x1041 waiting for monitor entry [0x00007fbdd3968000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.felix.framework.ServiceRegistry.registerService(ServiceRegistry.java:109) - waiting to lock <0x0000000700e2c590> (a org.apache.felix.framework.ServiceRegistry) at org.apache.felix.framework.Felix.registerService(Felix.java:3393) at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:346) at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:320) at com.mypackage.person.bs.processor.Processor.sendEvent(BullseyeModelProcessor.java:56) at com.mypackage.jetMyStream.event.support.AbstractEventSource.fireSendEvent(AbstractEventSource.java:97) at com.mypackage.jetMyStream.event.channel.messaging.InboundMessagingChannel.fireEvent(InboundMessagingChannel.java:113) at com.mypackage.jetMyStream.event.channel.messaging.InboundMessagingChannel.onMessage(InboundMessagingChannel.java:204) at com.mypackage.jetMyStream.messaging.MessageService.dispatchMessageForContext(MessageService.java:349) at com.mypackage.jetMyStream.messaging.MessageService.dispatch(MessageService.java:259) at com.mypackage.jetMyStream.messaging.MessageServiceRequest.execute(MessageServiceRequest.java:40)

at com.mypackage.jetMyStream.util.RequestThread.run(RequestThread.java:69)

A couple of question on this:

1. Am I doing something wrong in sending a map through registerService. If so what are my alternatives?

2. Any ideas on how I can get this to work? We have 26 nodes and need to only process about 400 requests per second using this mechanism

   Anyone had similar issues? Any pointers are highly appreciated

Thanks Masti

"Am I doing something wrong in sending a map through registerService. If so what are my alternatives?"

YES! The service registry is NOT intended to be used this way; it is not a messaging bus.

As for alternatives... why not use a messaging bus? You could look at OSGi Event Admin but there are many other implementations of this idea.

"Any ideas on how I can get this to work? We have 26 nodes and need to only process about 400 requests per second using this mechanism"

You have to be a bit more specific about what you're trying to achieve. The question is too abstract. Please describe where the messages are coming from, where they need to be delivered, what processing (if any) you need to do in the middle, etc.