I'm trying to limit my cURL responses as suggested in these posts:Retrieve partial web page and PHP CURLOPT_WRITEFUNCTION doesn't appear to be working. The idea is to limit the response to 4000 characters as specified in the callback function.
I wrote the following function, but I already know that it doesn't make sense, because a parameter in the callback function definition doesn't vary within a loop as it would within a function call. By the time the functions are actually called, the value for $key is fixed, so my references to that index won't vary.
It seems that I need a new closure function for each of the loops, and each one needs to reference its own $full_length variable. However, I don't see how that's possible. In order to do that, it seems I would have to somehow make a reference the closure object in order to specify the specific $full_length variable.
Any help would be appreciated. Thanks.
function get_headers($urls){
$curly = array();
$result = array();
$mh = curl_multi_init();
$obj = $this;
foreach ($urls as $key => $url) {
$this->full_length[$key] = 0;
$callback = function ($ch, $string) use ($obj, $key){
$length = strlen($string);
$obj->full_length[$key] += $length;
if($obj->full_length[$key] >= 4000){
return -1;
}
return $length;
};
$curly[$key] = curl_init
curl_setopt($curly[$key], CURLOPT_URL, $url);
curl_setopt($curly[$key], CURLOPT_HEADER, 0);
curl_setopt($curly[$key], CURLOPT_WRITEFUNCTION, $callback);
curl_setopt($curly[$key], CURLOPT_RETURNTRANSFER, 1);
curl_multi_add_handle($mh, $curly[$key]);
}
$running = null;
do {
curl_multi_exec($mh, $running);
} while($running > 0);
foreach($curly as $key => $cnt) {
$content = curl_multi_getcontent($cnt);
curl_multi_remove_handle($mh, $cnt);
if (strlen($content) > 0){
$result[$key] = $content;
} else {
curl_multi_close($mh);
return FALSE;
}
}
curl_multi_close($mh);
return $result;
}
EDIT:
I found a post that does exactly what I'm trying to do, but it's in javascript: closure inside a for loop - callback with loop variable as parameter . I wrote the following function to try to do the same thing in PHP:
function get_write_function($key){
$this->full_length[$key] = 0;
$obj = $this;
$funky = function ($ch, $str) use ($obj, $key){
$length = strlen($str);
$obj->full_length[$key] += $length;
if($obj->full_length[$key] >= 4000){
return -1;
}
return $length;
};
return $funky;
}
The code ran without errors, but it still didn't do what I wanted. After closing my cURL handles I dumped the $full_length array, and it only showed:
array([0] => 0, [1] => 0)
That indicates that they were initialized by the get_write_function (since I didn't initialize anything in the class declaration), but that the values were never updated afterwards.
I finally got it figured out. The biggest problem was the fact that cURL was ignoring the WRITEFUNCTION until I placed it as the very last option specified, as I posted here: cURL WRITEFUNCTION not Being Called. I actually didn't need the return transfer, since I wrote the output to a class variable. That was necessary because when the callback returns -1, nothing gets returned. The following code works great:
var $full_length = array();
var $result = array();
function get_headers($urls){
$curly = array();
$mh = curl_multi_init();
foreach ($urls as $key => $url) {
$callback = $this->get_write_function($key);
$curly[$key] = curl_init
curl_setopt($curly[$key], CURLOPT_URL, $url);
curl_setopt($curly[$key], CURLOPT_HEADER, 0);
curl_setopt($curly[$key], CURLOPT_WRITEFUNCTION, $callback);
curl_multi_add_handle($mh, $curly[$key]);
}
$running = null;
do {
curl_multi_exec($mh, $running);
} while($running > 0);
foreach($curly as $key => $cnt) {
curl_multi_remove_handle($mh, $cnt);
}
curl_multi_close($mh);
return $this->result;
}
function get_write_function($key){
$this->full_length[$key] = 0;
$this->result[$key] = '';
$obj = $this;
$funky = function ($ch, $str) use ($obj, $key){
$obj->result[$key] .= $str;
$length = strlen($str);
$obj->full_length[$key] += $length;
if($obj->full_length[$key] >= 4000){
return -1;
}
return $length;
};
return $funky;
}