firebasegoogle-cloud-functions

How to intentionally test idempotency of Firebase Functions


I'm testing how can I trace and track the idempotent of Firebase Functions so I'm intentionally failing my function in order to retry the event to be delivered again, and I'm able to get the expected results, however would like to know some questions provided below.

My sample code :

import { onDocumentWritten } from "firebase-functions/v2/firestore";
import * as logger from "firebase-functions/logger";
import * as admin from 'firebase-admin';

admin.initializeApp();
const db = admin.firestore();

/**
 * Checks if an event has already been processed.
 *
 * @param {string} event_id - The ID of the event to check.
 * @returns {Promise<boolean>} - A promise that resolves to true if the event has been processed, false otherwise.
 */
async function CheckForProcessed(event_id: string) {
    const docRef = db.collection("processedEvents").doc(event_id);
    const snapshot = await db.runTransaction(t => {
        return t.get(docRef);
    });
    return snapshot.exists;
}

/**
 * Marks an event as processed.
 *
 * @param {string} event_id - The ID of the event to mark as processed.
 * @returns {Promise<FirebaseFirestore.WriteResult>} - A promise that resolves to the write result.
 */
async function MarkEventProcessed(event_id: string) {
    const docRef = db.collection("processedEvents").doc(event_id);
    var data =
    {
        processedAt: admin.firestore.FieldValue.serverTimestamp(),
    }
    return docRef.set(data);
}

/**
 * Idempotency check for firestore events.
 *
 * @param {any} event - The firestore event data
 * @returns {Promise<any>} - A promise that resolves to null or throws an error
 */
export const idempotency_check = onDocumentWritten({
    document: "mocks/{mockId}",
    retry: true
}, async (event) => {
    const eventId = event.id;
    logger.info(`Event ID Found : ${eventId}`);
    logger.info(`Event Type Found : ${event.type}`);
    const mockId = event.params.mockId;
    logger.info(`Mock ID Found : ${mockId}`);
    const isProcessed = await CheckForProcessed(eventId);
    if (isProcessed) {
        logger.info('Event already processed:', eventId);
        return null;
    }
    logger.info('Processing event:', eventId);
    await MarkEventProcessed(eventId);

    const newValue = event.data?.after.exists ? event.data.after.data() : null;
    const oldValue = event.data?.before.exists ? event.data.before.data() : null;

    logger.info('New Data:', newValue);
    logger.info('Old Data:', oldValue);

    if (newValue?.name === "Admin") {
        logger.debug('Error processing event:', eventId);
        throw new Error("Invalid Name");
    }

    return null;
});

So when I try to create a document with the name : Admin it is logging following in cloud logging :


Event ID Found : b1f66fac-57dc-48bd-99b6-dac11c3e5631
Event Type Found : google.cloud.firestore.document.v1.written
Mock ID Found : 7B6JfVEJFO3hDIDzO8MJ
Processing event: b1f66fac-57dc-48bd-99b6-dac11c3e5631
New Data:
Old Data: null
Error processing event: b1f66fac-57dc-48bd-99b6-dac11c3e5631
Error: Invalid Name at /workspace/lib/index.js:64:15@

2nd Attempt : 
Event ID Found : b1f66fac-57dc-48bd-99b6-dac11c3e5631
Event Type Found : google.cloud.firestore.document.v1.written
Mock ID Found : 7B6JfVEJFO3hDIDzO8MJ
Event already processed: b1f66fac-57dc-48bd-99b6-dac11c3e5631

My function setting when deployed : 
Timeout : 60 seconds
Minimum instances : 0
Maximum instances : 100
Concurrency : 80

Question :

  1. I like to know whether the retried event will be handled by the same Cloud Run instance as I'm using gen2 firebase functions on which we can set the concurrency and this was not gonna happen in case of gen1 functions since they have no concurrency.
  2. If a retried event gets handled by the same instance how can I identify ?
  3. Is this the way to test idempotency of Firebase Functions or is there any better way ?

Thank you for your help in advance.


Solution

  • I like to know whether the retried event will be handled by the same Cloud Run instance

    The system doesn't guarantee this at all. It might or might not happen, and you shouldn't depend on it in order to operate effectively. Server instances can come and go whenever the system decides it's appropriate to do so. Your code should be prepared to work in any circumstance (serverless products like Cloud Functions and Cloud Run are said to be "stateless").

    If a retried event gets handled by the same instance how can I identify ?

    You'd have to write code to store and check some global state on the instance. That's entirely up to you to figure out. The system doesn't give you stateful processing of events. If you need to check state of any kind, that's up to you. Typically you use some sort of database or other independent storage mechanism to persist state that can be shared across any number of instances that might be running at any given time.

    Is this the way to test idempotency of Firebase Functions or is there any better way ?

    Strictly speaking, this is a matter of opinion and therefore off-topic for Stack Overflow. But there is nothing in the system or its documentation that suggest any way at all to formally or even "correctly" test this.