androidsqliteandroid-sqliteandroid-cursorgreendao

greendao - android.database.CursorWindowAllocationException: Could not allocate CursorWindow <DBName> of size 104857600 due to error -12

So I basically parse a JSON and store ~3000 rows (no multimedia) in the greendao generated table and I am getting the above error from while querying data in this function:

    @Synchronized
    private fun checkGroupVisibility(
        dependencyList: ArrayList<Long>,
        dependencyType: String,
        mInspectionId: Long,
        isFromPhilosophyFlow: Boolean,
        daoSession: DaoSession?,
        string: String?
    ): Boolean {
        var show = true
//        Log.d("TAG", "checkGroupVisibility: lock testing 2"+string)
        if (!DataSanityUtils.isListEmpty(dependencyList)) {
            //check further in database if dependency question is answered
            val inspectionReportCarParList:List<InspectionReportCarPart>? = daoSession?.inspectionReportCarPartDao?.queryBuilder()
                ?.where(InspectionReportCarPartDao.Properties.InspectionID.eq(mInspectionId))
                ?.where(InspectionReportCarPartDao.Properties.QuestionId.`in`(dependencyList))
                ?.list()

            val inspectionReportCheckpointList:List<InspectionReportCheckpoint>? = daoSession?.inspectionReportCheckpointDao?.queryBuilder()
                ?.where(InspectionReportCheckpointDao.Properties.InspectionID.eq(mInspectionId))
                ?.where(InspectionReportCheckpointDao.Properties.ReportCheckpointVerdict.eq(CommonConstants.CHECKPOINT_VERDICT_NOT_OKAY))
                ?.list()

            for (checkPointId in dependencyList) {
                if (!isFromPhilosophyFlow && philosophyIdCheckPointMap.containsKey(checkPointId)) {
                    show = false
                    break
                }
                val isCarPartFound:InspectionReportCarPart?= inspectionReportCarParList?.find {
                    it.questionId==checkPointId
                }

                if (isCarPartFound==null) {
                    val isFound= inspectionReportCheckpointList?.find {
                        it.optionId==checkPointId
                    }
                    if (isFound == null) {
                        show = false
                        if (CommonConstants.DEPENDENCY_RELATION_AND.equals(
                                dependencyType,
                                ignoreCase = true
                            )
                        ) {
                            break
                        }
                    } else if (CommonConstants.DEPENDENCY_RELATION_OR.equals(
                            dependencyType,
                            ignoreCase = true
                        )
                    ) {
                        show = true
                        break
                    }
                } else if (CommonConstants.DEPENDENCY_RELATION_OR.equals(
                        dependencyType,
                        ignoreCase = true
                    )
                ) {
                    show = true
                    break
                }
            }
        }
//        Log.d("TAG", "checkGroupVisibility: lock testing 3:"+string)
        return show
    }

The exact error is coming when evaluating inspectionReportCarParList (i.e. the first query in the code). Here is the full stacktrace :

android.database.CursorWindowAllocationException: Could not allocate CursorWindow '/data/user/0/com.example.visor/databases/report.json-db' of size 104857600 due to error -12.
    at android.database.CursorWindow.nativeCreate(CursorWindow.java)
    at android.database.CursorWindow.<init>(CursorWindow.java:139)
    at android.database.CursorWindow.<init>(CursorWindow.java:120)
    at android.database.AbstractWindowedCursor.clearOrCreateWindow(AbstractWindowedCursor.java:202)
    at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:149)
    at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:142)
    at de.greenrobot.dao.AbstractDao.loadAllFromCursor(AbstractDao.java:371)
    at de.greenrobot.dao.AbstractDao.loadAllAndCloseCursor(AbstractDao.java:184)
    at de.greenrobot.dao.InternalQueryDaoAccess.loadAllAndCloseCursor(InternalQueryDaoAccess.java:21)
    at de.greenrobot.dao.query.Query.list(Query.java:130)
    at de.greenrobot.dao.query.QueryBuilder.list(QueryBuilder.java:353)
    at com.example.inspectionreport.utils.ReportUtils.checkGroupVisibility(ReportUtils.kt:31)

Also I am using de.greenrobot:greendao:2.0.0 this version of greendao.

I thought greendao is supposed to handle all the cusrsor actions and it closes the cursor after querying itself. But still cursorWindow size is getting very huge and the application crashes. Please help!

Solution

  • A Cursor Window has a restricted size (depends upon the Android Version but up to 4Mb). 104857600 is 100Mb, so way too large.

    A CursorWindow has to be able to accommodate 1 or more rows, if it cannot hold 1 row then you will get such an error.

    It is possible to store data that is too large to be retrieved (it is a common issue with images).

    The recommended way of handling such large amounts of data is to store the data, such as an image, in a file and store the path or part of the path in the database.

    Additional re the comment

    I got it but in this case I do not store any multimedia directly here

    The issue is not limited to multimedia, that is just an example of the data being too large. It could be String (TEXT) data.

    Here's an example that shows the case (just using SQLite which greenDao is basically just a wrapper around), the example also includes an example of an alternative that does not experience the issue as it stores managable sections of the details of the report in a separate related table.

    Consider the following code:-

    public class MainActivity extends AppCompatActivity {

    DBHelper mDBHelper;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        mDBHelper = DBHelper.getInstance(this);
        /* Generate the base string i.e. 1000K */
        String current_report_detail = generateReportDetail(1000);

        /* Insert Rows with 1000K report detail */
        mDBHelper.insertReportV1(System.currentTimeMillis() / 1000,current_report_detail);
        mDBHelper.insertReportV2(System.currentTimeMillis() / 1000, current_report_detail);

        /* Insert Rows with 2000K eport detail */
        mDBHelper.insertReportV1(System.currentTimeMillis()/1000,current_report_detail + current_report_detail);
        mDBHelper.insertReportV2(System.currentTimeMillis() / 1000, current_report_detail + current_report_detail);

        /* Insert Rows with 3000K eport detail */
        mDBHelper.insertReportV1(System.currentTimeMillis()/1000,current_report_detail + current_report_detail + current_report_detail);
        mDBHelper.insertReportV2(System.currentTimeMillis() / 1000, current_report_detail + current_report_detail + current_report_detail);

        /* Insert Rows with 4000K eport detail */
        mDBHelper.insertReportV1(System.currentTimeMillis()/1000,current_report_detail + current_report_detail + current_report_detail + current_report_detail);
        mDBHelper.insertReportV2(System.currentTimeMillis() / 1000, current_report_detail + current_report_detail + current_report_detail + current_report_detail);

        /* Insert Rows with 5000K eport detail */
        mDBHelper.insertReportV1(System.currentTimeMillis()/1000,current_report_detail + current_report_detail + current_report_detail + current_report_detail + current_report_detail);
        mDBHelper.insertReportV2(System.currentTimeMillis() / 1000, current_report_detail + current_report_detail + current_report_detail + current_report_detail + current_report_detail);

    }

    /* return a String whose length is the passed linecount * 1024 i.e. 1K per line */
    private String generateReportDetail(int linecount) {
        StringBuilder sb1 = new StringBuilder();
        /* Build base line  of 1K of characters */
        for(int i=0; i < 1024; i++) {
            sb1.append((char) ((i % 95) + 33));
        }
        StringBuilder sb2 = new StringBuilder();
        for (int i=0;i<linecount;i++) {
            sb2.append(sb1.toString());
        }
        return sb2.toString();
    }
}

    So there are 5 stages each progressively inserting close to 1Mb of textual data along the lines of :-

    !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghi

    ....

    Each stage inserts the same data (1Mb,2Mb .... 5Mb) twice.

    When run, using App Inspection, then:-

    If an attempt is made to read all of the data in the v1_report table then the CursorWindow size occurs:-

    enter image description here

    Now for v2_report, there is no issue :-

    enter image description here

    As the long textual data has been split and placed into the v2_details table :-

    enter image description here

    The issue can be clearly seeing (also proving that the data exists and has been inserted successfully into v1_report) by using the following:-

    enter image description here

    Although not accurate (the example has been quickly put together and it looks as though I've dropped a byte/character) the equivalent can be extracted from both the v2 tables as per:-

    enter image description here