CodeIgniter Active Record: Load One Row at a Time

Question

The normal result() method described in the documentation appears to load all records immediately. My application needs to load about 30,000 rows, and one at a time, submit them to a third-party search index API. Obviously loading everything into memory at once doesn't work well (errors out because of too much memory).

So my question is, how can I achieve the effect of the conventional MySQLi API method, in which you load one row at a time in a loop?

Answer 1

Here is something you can do.

while ($row = $result->_fetch_object()) {
  $data = array(
    'id'         => $row->id
    'some_value' => $row->some_field_name
  );
  // send row data to whatever api
  $this->send_data_to_api($data);
}

This will get one row at the time. Check the CodeIgniter source code, and you will see that they will do this when you execute the result() method.

Answer 2

For those who want to save memory on large result-set:

Since CodeIgniter 3.0.0, There is a unbuffered_row function,

All the methods above will load the whole result into memory (prefetching). Use unbuffered_row() for processing large result sets.

 

This method returns a single result row without prefetching the whole result in memory as row() does. If your query has more than one row, it returns the current row and moves the internal data pointer ahead.

$query = $this->db->query("YOUR QUERY");

while ($row = $query->unbuffered_row())
{
    echo $row->title;
    echo $row->name;
    echo $row->body;
}

You can optionally pass ‘object’ (default) or ‘array’ in order to specify the returned value’s type:

$query->unbuffered_row();               // object
$query->unbuffered_row('object');       // object
$query->unbuffered_row('array');        // associative array

Official Document: https://www.codeigniter.com/userguide3/database/results.html#id2