phpgoogle-apifreebase

Paginating over travel destinations in Freebase


I am trying to fetch all travel destinations from Freebase API. According to the documentation I am using cursor and limit parameters. I start with cursor at 0 and limit 100, I then add 100 to the cursor in every iteration. The problem is I am getting only 76 results this way. There should be cca 1500 travel destinations in Freebase database.

Documentation:

https://developers.google.com/freebase/v1/search

Here is my code:

<?php

require_once __DIR__ . '/vendor/autoload.php';

$apiKey = 'SUPER_SECRET_API_KEY';

use Guzzle\Http\Client;

//// GOOGLE FREEBASE
$client = new Client('https://www.googleapis.com');

$cursor = 0;
$limit = 100;
$results = array();

do {

    $request = $client->get('/freebase/v1/search');
    $request->getQuery()->set('key', $apiKey);
    $request->getQuery()->set('filter', '(all type:/travel/travel_destination)');
    $request->getQuery()->set('cursor', $cursor);
    $request->getQuery()->set('limit', $limit);
    $request->getQuery()->set('indent', 'true');
    $request->getQuery()->set('query', '');

    $response = $request->send()->json();

    foreach ($response['result'] as $result) {
        $results[] = $result;
    }

    $cursor += $limit;

} while (count($response['result']) > 0);

echo count($results);exit;

It prints 76.


Solution

  • Returning exhaustive results isn't an intended use case for the search API, so this would be better done using either MQL or the bulk data dumps.

    Since this very first query returns a full 100 results, I'm assuming there's a bug in your PHP. If I had to guess, I'd guess that you're either counting the number of iterations or the number of results returned in the last iteration, probably the latter.