javascriptphpgoogle-mapsheatmapgoogle-fusion-tables

Generating heatmap layer for milions of points


I'm using heatmap layer from Google Maps to display a heatmap, however, I now have too many points and it stopped working, because browser cannot handle it anymore. I've found that they provide Fusion Tables, but they're also limited: to 100k rows, which is way too low. I need to render heatmap of milions or maybe even more points. I'd be perfect if my server can have some PHP script to render a heatmap, for example, once a day. And then a client from js will just download a preloaded heatmap (on the map like google maps, but may be different map too). Is this possible with some existing technology (can be commercial)?


Solution

  • All you need is to pre-clusterize your points into smaller number of points and pass these groups to Google Maps as if it were your original points. So, if you had 3 nearby points with values 3, 5 and 10, you create one point with value 18 at weighted average of their coordinates. Do it for all your dataset and you will reduce it 3 times. Change 3 to any appropriate value to further reduce your dataset.

    The easy way to cluster your data is to use geohashing[1]. Here is nice geohashing library for PHP[2]. You may calculate a set of geohashes of different precision for each of your points only once when adding, and then use desired precision to reduce your dataset using simple GROUP BY. Example (meta):

    use Lvht\GeoHash;
    
    class DataPoint extends ActiveRecord {
        public geo_hash_precision4, geo_hash_precision5, geo_hash_precision6;
        public function save() {
            $this->geo_hash_precision4 = GeoHash::encode($this->lat,$this->lon, 0.0001);
            $this->geo_hash_precision5 = GeoHash::encode($this->lat,$this->lon, 0.00001);
            $this->geo_hash_precision6 = GeoHash::encode($this->lat,$this->lon, 0.000001);
            parent::save();
        }
    }
    
    class DataSet extends ActiveQuery {
        /**
         * @param int $p Desired precision
         */
        public function getValues($p = 6) {
            $data = $this->rawQuery("SELECT geo_hash_precision{$p}, SUM(value) FROM {$this->table} GROUP BY geo_hash_precision{$p}");
            // Add bounding box WHERE to reduce set for map size
            foreach ($data as $row) {
                list($minLon, $maxLon, $minLat, $maxLat) = GeoHash::decode($row["geo_hash_precision{$p}"]);
                $row['lat'] = ($maxLat - $minLat) / 2;
                $row['lon'] = ($maxLon - $minLon) / 2;
                unset($row["geo_hash_precision{$p}"]);
            }
        }
    }
    
    1. https://en.wikipedia.org/wiki/Geohash
    2. https://github.com/lvht/geohash