httptimeoutejbmessage-driven-bean

Call to slow service over HTTP from within message-driven bean (MDB)


I have a message driven bean which serves messages in a following way:

  1. It takes data from incoming message.
  2. Calls external service via HTTP (literally, sends GET requests using HttpURLConnection), using the data from step 1. No matter how long the call takes - the message MUST NOT be dropped.
  3. Uses the outcome from step 2 to persist data (using entity beans).

Rate of incoming messages is: I. Low most of the time: an order of units / tens in a day. II. Sometimes high: order of hundreds in a few minutes.

QUESTION: Having that service in step (2) is relatively slow (20 seconds per request and degrades upon increasing workload), what is the best way to deal with situation II?

WHAT I TRIED:

  1. Letting MDB to wait until service is executed, no matter how long it takes. This tends to rollback MDB transactions by timeout and to re-deliver message, increasing workload and making things even worse.
  2. Setting timeout for HttpURLConnection gives some guarantees in terms of completion time of MDB onMessage() method, but leaves an open question: how to proceed with 'timed out' messages.

Solution

  • This is what I ended up with (mostly, this is application server configuration):

    1. Relatively short (comparing to transaction timeout) timeout for HTTP call. The rationale: long-running transactions from my experience tend to have adverse side effects such as threads which are "hung" from app. server point of view, or extra attention to database configuration, etc.
      I chose 80 seconds as timeout value.
    2. Increased up to several minutes re-delivery interval for failed messages.
    3. Careful adjustment of the number of threads which handle messages simultaneously. I balanced this value with throughput of HTTP service.