javascriptfirebasegeolocationreal-timegeohashing

Realtime complex Geoquery within Firebase for 100K users


The use case:

The problem:

The suboptimal alternative: [Web Javascript Firebase V9 Modular] I would use Firestore, create a Geoquery for a given range(which is static, not real time) and get all docs from it. Then in the console the user would only see real time info by clicking into a specific truck(document), so I could attach onSnapshot to that document.

Is this use case simply not supported or is there a way to model data to accommodate it at a reasonable cost ?


Solution

  • GeoFire is only fully supported on the Realtime Database. GeoFire's data is intended to be kept separate from your own data and linked to it by a unique key/id. This minimises the indexing required to execute queries and minimises the data handed around while querying.

    There is a workaround for Cloud Firestore but it's not ideal for your use case because it grabs all the nearby records then filters the rest on the client.

    If you are curious, a GeoFire entry looks similar to the following when uploaded (this is formatted for readability):

    "path/to/geofire/": {
      "<key>/": {
        ".priority": string, // a geohash, hidden
        "g": string, // a geohash
        "l": {
          "0": number, // latitude, number
          "1": number, // longitude, number
        }
      }
    }
    

    As can be seen above, there isn't any user-provided data here only a "key". This key can have any meaning such as the code of an airport in a registry, a push ID under a Realtime Database location, a Cloud Firestore document ID or some base64-encoded data.

    With the introduction of Firestore, a number of users store the GeoFire data in the Realtime Database and link it back to Cloud Firestore documents using the key stored in GeoFire.

    In the below example, a truckInfo object looks like this:

    interface TruckInfo {
      key: string;
      location: Geopoint; // [number, number]
      distance: number;
      snapshot: DataSnapshot | undefined;
      cancelling?: number; // an ID from a setTimeout call
      errorCount: number; // number of errors trying to attach data listener
      hidden?: true; // indicates whether the vehicle is currently out of range
      listening?: boolean; // indicates whether the listener was attached successfully
    }
    

    For the below code to work, you must define two callback methods:

    const truckUpdatedCallback = (truckInfo, snapshot) => { // or you can use ({ key, location, snapshot }) => { ... }
      // TODO: handle new trucks, updated locations and/or updated truck info
      // truckInfo.hidden may be true!
    }
    
    const truckRemovedCallback = (truckInfo, snapshot) => {
      // TODO: remove completely (deleted/out-of-range)
    }
    

    These callbacks are then invoked by the following "engine":

    const firebaseRef = ref(getDatabase(), "gf"), // RTDB location for GeoFire data
      trucksColRef = collection(getFirestore(), "trucks"), // Firestore location for truck-related data
      geoFireInstance = new geofire.GeoFire(firebaseRef),
      trackedTrucks = new Map(), // holds all the tracked trucks
      listenToTruck = (truckInfo) => { // attaches the Firestore listeners for the trucks
        if (truckInfo.cancelling !== void 0) {
          clearTimeout(truckInfo.cancelling);
          delete truckInfo.cancelling;
        }
       
        if (truckInfo.listening || truckInfo.errorCount >= 3)
          return; // do nothing.
    
        truckInfo.unsub = onSnapshot(
          truckInfo.docRef,
          (snapshot) => {
            truckInfo.listening = true;
            truckInfo.errorCount = 0;
            truckInfo.snapshot = snapshot;
            truckUpdatedCallback(truckInfo, snapshot); // fire callback
          },
          (err) => {
            truckInfo.listening = false;
            truckInfo.errorCount++;
            console.error("Failed to track truck #" + truckInfo.key, err);
          }
        )
      },
      cancelQuery = () => { // removes the listeners for all trucks and disables query
        // prevents all future updates
        geoQuery.cancel();
        trackedTrucks.forEach(({unsub}) => {
          unsub && unsub();
        });
      };
    
    const geoQuery = geoFireInstance.query({ center, radius });
    
    geoQuery.on("key_entered", function(key, location, distance) {
      let truckInfo = trackedTrucks.get(key);
    
      if (!truckInfo) {
        // new truck to track
        const docRef = document(trucksColRef, key);
        truckInfo = { key, location, distance, docRef, errorCount: 0 };
        trackedTrucks.set(key, truckInfo);
      } else {
        // truck has re-entered watch area, update position
        Object.assign(truckInfo, { location, distance });
        delete truckInfo.hidden;
      }
    
      listenToTruck(truckInfo);
    });
    
    geoQuery.on("key_moved", function(key, location, distance) {
      const truckInfo = trackedTrucks.get(key);
      if (!truckInfo) return; // not being tracked?
      Object.assign(truckInfo, { location, distance });
      truckUpdatedCallback(truckInfo, snapshot); // fire callback
    });
    
    geoQuery.on("key_exited", function(key, location, distance) {
      const truckInfo = trackedTrucks.get(key);
      if (!truckInfo) return; // not being tracked?
    
      truckInfo.hidden = true;
    
      const unsub = truckInfo.unsub,
        cleanup = () => { // removes any listeners for this truck and removes it from tracking
          unsub && unsub();
          truckInfo.listening = false;
          trackedTrucks.delete(key);
          truckRemovedCallback(truckInfo, snapshot); // fire removed callback
        };
    
      if (location === null && distance === null) {
        // removed from database, immediately remove from view
        return cleanup();
      }
    
      // keep getting updates for at least 60s after going out of
      // range in case vehicle returns. Afterwards, remove from view
      truckInfo.cancelling = setTimeout(cleanup, 60000);
    
      // fire callback (to hide the truck)
      truckUpdatedCallback(truckInfo, snapshot);
    });