javadatabasems-accessjackcess

Jackcess Table Limit?


I'm currently trying to resolve an issue related to a program written by someone else that uses Jackcess version 1.1.8 to write information out to an Access database. In a run which should add 4 million rows to a given Access table, the following exception occurs:

08/29/2016 06:01:47 | ERROR | java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Unknown Source)
at java.nio.HeapByteBuffer.getInt(Unknown Source)
at com.healthmarketscience.jackcess.ReferenceUsageMap.addOrRemovePageNumber(ReferenceUsageMap.java:82)
at com.healthmarketscience.jackcess.UsageMap.addPageNumber(UsageMap.java:201)
at com.healthmarketscience.jackcess.Table.newDataPage(Table.java:761)
at com.healthmarketscience.jackcess.Table.addRows(Table.java:714)
at com.healthmarketscience.jackcess.Table.addRow(Table.java:660)

From what I've seen online, it appears that the current version of Jackcess is 2.*, so is this something that can be solved by using the latest version, or is there a bigger problem, i.e., is 4 million too many rows for an Access table? Alternatively, is there a way to add rows via a Cursor object that would be less memory-intensive?


Solution

  • If the application in question is still important to your business then you need to update it to use a current version of Jackcess. You've gotten over 9 years out of it so far (Jackcess 1.1.8 was released in February 2007) and things have progressed a little bit since then.

    Notably, the current version of Jackcess supports DatabaseBuilder#setAutoSync(false) which disables the row-by-row flushing of updates to the database file. (I searched the Jackcess 1.1.8 source code for 'autosync' and found no matches, so I assume that it wasn't supported back then.)

    For a test insert of 100,000 rows using Jackcess 2.1.3, setAutoSync(true) (the default) took around 200 seconds, while the same code with setAutoSync(false) took 8 seconds.

    A test insert of 4,000,000 rows with setAutoSync(false) took around 220 seconds, or just over 3.5 minutes. Based on the numbers above, the same operation without setAutoSync(false) would be expected to take somewhere on the order of 5,500 seconds, or 1.5 HOURS, to accomplish the same thing.