phpbigtable

BigTable - getting total number of rows


I'm using PHP client for working with BigTable instance. Is there a way to fetch total number of rows from the table?

So far the solution I found is to use Filter::pass() and manually loop through all returned results. Wondering if there is an easier way.


Solution

  • BigTable does not maintain count of rows seprately that you can pull directly. Even the cbt tool which supports count reads all the rows to get to a count. Since you are doing count you should use Strip value filter $filter = Filter::value()->strip(). This removes all the cell values making it easy on the memory consumption.

    For larger datasets where you want to avoid scan all together you could use increment with your writes to maintain the count yourself.

    If you are sticking with scans, for larger tables you may want to use a system like dataflow which can distribute the count process to many workers and get you the results quickly. Here is the SourceRowCount example from Dataflow.