phpcodeigniterpaginationsql-limitmysql-num-rows

Increasing LIMIT and OFFSET values with looped pagination queries is returning unexpected row counts in CodeIgniter


I am working with a very large data set (786,432 rows to be precise).

So, to prevent memory limits I want to loop over the data set in piles of 50,000 rows, so to test this out I thought I would try:

public function test()
{
    $start = 0;
    $end = 50000;

    $q = $this->db->select('uuid')->from('userRegionLink')->limit($end, $start)->get();
    $i = 0;
    while ($q->num_rows() != 0) {
        echo 'Round: ' . ++$i . '<br />';
        echo 'Rows: ' . $q->num_rows() . '<br />';
        echo 'Start: ' . $start . '<br />';
        echo 'End: ' . $end . '<hr />';

        $start = $end;
        $end = $end+50000;
        $q = $this->db->select('uuid')->from('userRegionLink')->limit($end, $start)->get();
    }
}

But my results are very strange; look at the result starting from round 9.

What is causing these strange row counts?

Round: 1
Rows: 50000
Start: 0
End: 50000


Round: 2
Rows: 100000
Start: 50000
End: 100000
Round: 3
Rows: 150000
Start: 100000
End: 150000
Round: 4
Rows: 200000
Start: 150000
End: 200000
Round: 5
Rows: 250000
Start: 200000
End: 250000
Round: 6
Rows: 300000
Start: 250000
End: 300000
Round: 7
Rows: 350000
Start: 300000
End: 350000
Round: 8
Rows: 400000
Start: 350000
End: 400000
Round: 9
Rows: 386432
Start: 400000
End: 450000
Round: 10
Rows: 336432
Start: 450000
End: 500000
Round: 11
Rows: 286432
Start: 500000
End: 550000
Round: 12
Rows: 236432
Start: 550000
End: 600000
Round: 13
Rows: 186432
Start: 600000
End: 650000
Round: 14
Rows: 136432
Start: 650000
End: 700000
Round: 15
Rows: 86432
Start: 700000
End: 750000
Round: 16
Rows: 36432
Start: 750000
End: 800000


Solution

  • It looks like $end is not a global offset just number of records to fetch (offset from $start). Try to set $end always for 50000 and changing only $start.