My problem is I want to access json data from an API webpage, and there are only 10 calls per second allowed. After 10 calls added to the handler I exec them and then every result gets decoded, the data processed and the curl session will be removed from the handler.
In average it takes for one call about ~600 ms (results of 11600 calls). How to stop calling new ones after I called 10? I tried to use usleep (and it works) but it slows down everything really heavy. Somtimes about 20 seconds~ is this the only way to delay curl calls or how can I change this code to be more efficient?
$i = 0;
$run = NULL;
do {
curl_multi_add_handle($handler, $curls[$i++]);
while ($i < $curl_count && $i % 10 != 0) {
curl_multi_add_handle($handler, $curls[$i++]);
}
curl_multi_exec($handler, $running);
do {
if (curl_multi_select($handler) != -1) {
curl_multi_exec($handler, $running);
}
} while ($running > 0);
while ($msg = curl_multi_info_read($handler)) {
$exec_time[] = curl_getinfo($msg['handle'], CURLINFO_TOTAL_TIME);
$response = curl_multi_getcontent($msg['handle']);
$results = json_decode($response);
processData() // function to process data
curl_multi_remove_handle($handler, $msg['handle']);
}
$waiting_time = 1 - max($exec_time);
if ($waiting_time > 0) {
usleep(1000000 * $waiting_time);
}
$exec_time = NULL;
usleep(100 * 1000);
} while ($i < count($curls));
curl_multi_close($handler);
EDIT Updated code to solve errors and improving it. See edits to see what has changed.
You probably need to set up a timer and only sleep for the amount of time needed to get to 1 second interval, not a hard-coded 1 second time. So something like:
// start loop here
$start_time = microtime();
// run your 10 cURL requests here
$end_time = microtime();
$udelay = 1000000 * (1 - ($end_time - $start_time));
if ($udelay > 0) {
usleep($delay);
}
// end loop here
By the way, that seems to be a fairly rudimentary usage of curl_multi. If you like you can take a look at this lightweight curl-based REST client I wrote. It would make this more robust for you with Exception handling and such.