I have a google task which gets all companies from my firebase database. I then go through each of those companies in a loop and call additional task for updating each specific company. My problem is that my companies count is increasing and when doing foreach like this i can get into memory limit issues. Here is the actual code for calling the tasks and subtasks:
$router->get('companies', function () use ($router) {
$slackDataHelpersService = new \App\Services\SlackDataHelpersService();
$companiesDocuments = $slackDataHelpersService->getCompanies();
foreach ($companiesDocuments->documents() as $document) {
$cid = $document->id();
createTask('companies', 'updateCompany', "{$cid}");
}
return res(200, 'Task done');
});
How can i separate my initial companies documents into chunks and call a task for each of those chunks? For example, a task that will go through every 100 documents instead of the whole list?
Here is what i tried without success(i used members in this case):
$router->get('test2', function () use ($router) {
$db = app('firebase.firestore')->database();
$membersRef = $db->collection('companies')->document('slack-T01L7H2NDPB')->collection('members');
$query = $membersRef->orderBy('created', 'desc')->limit(10);
$perPage = 10;
$batchCount = 10;
$lastCreated = null;
while ($batchCount == $perPage) {
$loopQuery = clone $query;
if ($lastCreated != null) {
$loopQuery->startAfter($lastCreated);
}
$docs = $loopQuery->documents();
$docsRows = $docs->rows();
$batchCount = count($docsRows);
if ($batchCount > 1) {
$lastCreated = $docsRows[$batchCount - 1];
}
echo $lastCreated['created'];
//createTasksByDocs($docs);
}
//return res(200, 'Task done');
});
I ended up making a function which uses a while loop and loops until it reaches the limit:
function paginateCollections($ref, $limit, $functionName)
{
$query = $ref->orderBy('created', 'desc')->limit($limit);
$perPage = $limit;
$batchCount = $limit;
$lastCreated = null;
while ($batchCount == $perPage) {
$loopQuery = clone $query;
if ($lastCreated != null) {
$loopQuery = $loopQuery->startAfter([$lastCreated]);
}
$docs = $loopQuery->documents();
$docsRows = $docs->rows();
$batchCount = count($docsRows);
if ($batchCount > 1) {
$lastCreated = $docsRows[$batchCount - 1]['created'];
}
if (function_exists($functionName)) {
$functionName($docs);
}
}
}