phpmysqlcodeigniter-3

multiple connections in CodeIgniter are very slow - how to speed up


I'm using CodeIgniter 3, here is the code in question:

//now transfer data - this only works for very small to medium tables
$all = 'SELECT * FROM `'.$sourceServer['database'].'`.`'.$table.'`';
if($data = $this->source->cnx->query($all)){
    $sqls = [];
    foreach($data->result_array() as $v){
        $sql = 'INSERT INTO `'.$targetServer['database'] . '`.`' . $targetTable . '` SET ';
        foreach($v as $o => $w){
            $sql .= "\n\t`" . $o . '`=' .
                (is_null($w) ? '' : "'") .
                (is_null($w) ? 'NULL' : str_replace("'", "\\'", $w)) .
                (is_null($w) ? '' : "'") . ',';
        }
        $sqls[] = rtrim($sql, ',');
    }
    foreach($sqls as $sql){
        $this->target->cnx->query($sql);
    }
}

Note that $this->target->cnx.. and $this->source->cnx.. are instantiated CI connections but could well be on two different physical servers on a network.

Also, both connections are open the entire time of processing since I may well want to perform other cross-network operations later (I could close $this->source->cnx after I got the data but might have to open it again).

I find this is very slow; how can I speed it up?


Solution

  • $this->db->insert_batch('mytable', $data); note data is array of array [[],[],[]...[]] or form query like this

    INSERT INTO mytable (title, name, date) VALUES ('My title', 'My name', 'My date'), ('Another title', 'Another name', 'Another date')