javagoogle-analytics-apigoogle-reporting-api

Displaying more than 10000 rows using Core Reporting Google API v4 ( Java)


I'm fetching Google Analytics data using Core Reporting API v4. I'm able to capture at most 10,000 records for a given combination of Dimensions & Metrics. My question is that if my query can produce more than 10,000 search results then how can I fetch all those records? I have gone through the documentation and found that in a single request we can't access more than 10,000 records by setting the properties of ReportRequest object.

ReportRequest request = new ReportRequest()
    .setDateRanges(Arrays.asList(dateRange)) 
    .setViewId(VIEW_ID)
    .setDimensions(Arrays.asList(dimension))
    .setMetrics(Arrays.asList(metric))
    .setPageSize(10000); 

How can we enable multiple requests in a single run depending upon the number of search-results that can be obtained.

For example : If my query can return 35,000 records then there should be 4 requests (10,000,10,000, 10,000 & 3,500) managed internally.

Please look into this and facilitate me some guidance. Thanks in Advance.


Solution

  • Here's a stable and extensively tested solution in Java. It is a recursive solution that stores every 10000 results batch (if any) and recalls itself until finds a null nextToken. In this specific solution every 10000 results batch is saved into a csv and then a recursive call is performed! Note that the first time this function called from somewhere outside, the nextPageToken is also null!! Focus on the recursive rationale and the null value check!

    private static int getComplexReport(AnalyticsReporting service,int 
    reportIndex,java.lang.String startDate,String endDate,ArrayList<String>
    metricNames,ArrayList<String> dimensionNames,String pageToken)    
    throws IOException
    
    ReportRequest req = createComplexRequest(startDate,endDate,metricNames,dimensionNames,pageToken);
    
    ArrayList<ReportRequest> requests = new ArrayList<>();
    requests.add(req);
    
    // Create the GetReportsRequest object.
    GetReportsRequest getReport = new GetReportsRequest()
        .setReportRequests(requests);
    
    // Call the batchGet method.
    GetReportsResponse response = service.reports().batchGet(getReport).execute();
          //printResponse(response);
    
    
    saveBatchToCsvFile("dummy_"+startDate+"_"+endDate+"_"+Integer.toString(reportIndex)+".csv",startDate+"_"+endDate,response,metricNames,dimensionNames);
    String nextToken = response.getReports().get(0).getNextPageToken();
    //System.out.println(nextToken);
    if(nextToken!=null)
        return getComplexReport(service,reportIndex+1,"2016-06-21","2016-06-21",metricNames,dimensionNames,nextToken);
    
    return reportIndex; 
    }