Right now in my application, at certain points, we are logging some heavy stuff into the log files.
Basically only for logging, we are creating JSON of the data available and then logging into Log files. This is a business requirement to log data in JSON format.
Now creating JSON from the data available and then logging to FILE takes a lot of time and impacts the original request return time. Now idea is to improve the situation.
One of the things that we have discussed is creating a thread pool using
Executors.newSingleThreadExecutor()
in our code and then submitting the task to it which does the conversion of data into JSON and subsequent logging.
Is it a good approach to do this? As we are managing the thread pool itself, is it going to create some issues?
I would appreciate it if someone can share better solutions. Someway to use Log4j for this. I tried to use AsyncAppender but didn't achieve any desired result.
We are using EJB 3, JBoss 5.0, Log4j, and Java6.
I believe you are on right track in terms of using a separate thread pool for logging. In lot of products you will see the asynchronous logging feature. Logs are accumulated and pushed to log files using a separate thread than the request thread. Especially in prodcution environments, where are millions of incoming request and your response time need to be less than few seconds. You cannot afford anything such as logging to slow down the system. So the approach used is to add logs in a memory buffer and push them asynchronously in reasonably sized chunks.
A word of caution while using thread pool for logging As multiple threads will be working on the log file(s) and on a memory log buffer, you need to be careful about the logging. You need to add logs in a FIFO kind of a buffer to be sure that logs are printed in the log files sorted by time stamp. Also make sure the file access is synchronized and you don't run into situation where log file is all upside down or messed up.