phprecursionforeachslim

Make repeated API calls until all results are collected


My script is designed with the following intention :

  1. I visit the route /postman-test-route
  2. It makes an API call to https://pro-sitemaps.com/api/ with some headers and parameters
  3. Each API result shows a maximum 20-entries
  4. When the result is shown, I iterate the result and count the entries. If entries == 20, then another API call is made, but change the 'from' parameters from 0 to 20, then 40 then 60 etc. until a batch of entries is LESS than 20.

The trouble is that the code only runs once. The code looks like this:

$app->map(['GET', 'POST'],'/postman-test-route', function (Request $request, Response $response) {
    function getPROsitemapsEntries($total_from)
    {
        $client = new Client([
            'sink' => 'C:\Users\****\Desktop\temp.txt'
        ]);

        $r = $client->request('POST', 'https://pro-sitemaps.com/api/', [
            'form_params' => [
                'api_key' => 'ps_UmTvDUda.***************',
                'method' => 'site_history',
                'site_id' => 3845****,
                'from' => $total_from, // From the entity's ID, can run a foreach for every 20 entries. Keep a counter on result, if less than 20 then continue, otherwise die.
            ]
        ]);
        return $r;
    }

    $function_call =   getPROsitemapsEntries(0);
    $responseData = json_decode($function_call->getBody(), true);

    $i = 0;
    $items = array(); // ALL entries should be saved here. 
    foreach ($responseData['result']['entries'] as $entries) {
        $items[] = $entries;
        $i++;
    }

    // Here it should call the API again with 'from' = 20, then 40, then 60
    if ($i > 20) {
        getPROsitemapsEntries($i);
    } else {
        die;
    }

So as you can see this code:

if ($i > 20) {
    getPROsitemapsEntries($i);
} else {
    die;
}

I thought this would call the API again and inside foreach new entries should be saved (not override).


Solution

  • So you are actually calling the API again, you're just not iterating over the results.

    $shouldProcess = true;
    $searchIndex = 0;
    $items = [];
    while ($shouldProcess) {
        $processedThisLoop = 0;
        $function_call = getPROsitemapsEntries($searchIndex);
        $responseData = json_decode($function_call->getBody(), true);
    
        foreach($responseData['result']['entries'] as $entries) {
            $items[] = $entries;
            $searchIndex++;
            $processedThisLoop++;
        }
    
        if($processedThisLoop == 0) {
            // Didn't find any so stop the loop
            $shouldProcess = false;
        }
    }
    
    var_dump($items);
    

    In the code above we are keeping track of the total amount of entries we have processed in $searchIndex. This will allow us to keep getting new items instead of the old ones.

    $shouldProcess is a bool that will dictate if we should keep trying to get new entries from the API.

    $items is an array that will hold all entries from the API.

    $processedThisLoop contains the amount of entries that we have processed in this loop, i.e. did this request to the API have any entries to process? If it didn't, then set $shouldProcess to false and this will stop the while loop.