I have the following code in Scala that uses the Apache pool2
library. The object to pool is OlapConnection
(an olap4j
class, similar to an SQL connection).
Problem is that I can't make the pool close the connections automatically when the number of pooled objects exceeds the maximum.
If I return an object to the pool (with pool.returnObject
) that triggers passivateObject
. If I close the connection in passivateObject
, I would close it every time I return an object, and that's not what I want - I need the open connections to be cached. If I don't close the connection in passivateObject
, then it will never be closed.
How to make this work?
Scala code:
class OlapConnectionUtil (val pool: ObjectPool[OlapConnection]) {
def connect = {
pool.borrowObject
}
def close(olapConnection: OlapConnection) = {
pool.returnObject(olapConnection)
}
}
class OlapConnectionFactory extends BasePooledObjectFactory[OlapConnection] {
override def create = {
val connectionString = "jdbc:mondrian:Jdbc=jdbc:mysql://localhost:3306/foodmart?" +
"user=x&password=x;Catalog=FoodMart.xml;JdbcDrivers=com.mysql.jdbc.Driver"
val connection = DriverManager.getConnection(connectionString)
connection.unwrap(classOf[OlapConnection])
}
override def wrap(olapConnection: OlapConnection) =
new DefaultPooledObject(olapConnection)
override def passivateObject(pooledObject: PooledObject[OlapConnection] ) {
println("passivateObject WAS CALLED")
pooledObject.getObject.close
}
}
class Test {
val olapConnectionFactory = new OlapConnectionFactory
def test = {
val config = new GenericObjectPoolConfig
config.setMaxIdle(5)
config.setMaxTotal(10)
val util = new OlapConnectionUtil(
new GenericObjectPool[OlapConnection](olapConnectionFactory,config))
val olapConnection = util.connect
// do stuff with olapConnection
util.close(olapConnection)
If you open the JavaDocs for PooledObjectFactory
you may see that
passivateObject(org.apache.commons.pool2.PooledObject<T>)
is invoked on every instance when it is returned to the pool.destroyObject(org.apache.commons.pool2.PooledObject<T>)
is invoked on every instance when it is being "dropped" from the pool (whether due to the response fromvalidateObject(org.apache.commons.pool2.PooledObject<T>)
, or for reasons specific to the pool implementation.) There is no guarantee that the instance being destroyed will be considered active, passive or in a generally consistent state.
In other words you should put your resource de-allocation logic into the destroyObject
. Here is a modified test of yours with a fake OlapConnection
implementation
import org.apache.commons.pool2._
import org.apache.commons.pool2.impl._
// fake!
case class OlapConnection(val id: Int) {
def close(): Unit = {
println(s"Close was called for $this")
}
}
class OlapConnectionUtil(val pool: ObjectPool[OlapConnection]) {
def connect = {
pool.borrowObject
}
def close(olapConnection: OlapConnection) = {
pool.returnObject(olapConnection)
}
}
class OlapConnectionFactory extends BasePooledObjectFactory[OlapConnection] {
var cnt = 0
override def create = {
cnt += 1
new OlapConnection(cnt)
}
override def wrap(olapConnection: OlapConnection) =
new DefaultPooledObject(olapConnection)
override def passivateObject(pooledObject: PooledObject[OlapConnection]) {
println("-passivateObject was called")
}
override def destroyObject(pooledObject: PooledObject[OlapConnection]): Unit = {
super.destroyObject(pooledObject)
println("--destroyObject was called")
pooledObject.getObject.close
}
}
object Test {
val olapConnectionFactory = new OlapConnectionFactory
def test = {
val config = new GenericObjectPoolConfig
config.setMaxIdle(5)
config.setMaxTotal(10)
val util = new OlapConnectionUtil(new GenericObjectPool[OlapConnection](olapConnectionFactory, config))
val initConnections = (1 to 10).map(i => util.connect).toList
val closeCons = initConnections
Thread.sleep(100)
println("Start closing")
closeCons.zipWithIndex.foreach(ci => {
Thread.sleep(100)
println(s"Before close ${ci._2 + 1}")
util.close(ci._1)
println(s"After close ${ci._2 + 1}")
})
}
}
The output produced by this code is:
Before close 1
-passivateObject was called
After close 1
Before close 2
-passivateObject was called
After close 2
Before close 3
-passivateObject was called
After close 3
Before close 4
-passivateObject was called
After close 4
Before close 5
-passivateObject was called
After close 5
Before close 6
-passivateObject was called
--destroyObject was called
Close was called for OlapConnection(6)
After close 6
Before close 7
-passivateObject was called
--destroyObject was called
Close was called for OlapConnection(7)
After close 7
Before close 8
-passivateObject was called
--destroyObject was called
Close was called for OlapConnection(8)
After close 8
Before close 9
-passivateObject was called
--destroyObject was called
Close was called for OlapConnection(9)
After close 9
Before close 10
-passivateObject was called
--destroyObject was called
Close was called for OlapConnection(10)
After close 10
As you can see in this output close
is being called since OlapConnection(6)
as one would expect given the pool configuration.