Firestore's export and import services enable you to recover a corrupt database to some point prior to your desired recovery time. But how would you roll the database forward from there to the actual point of failure?
Would you have to develop your own recovery process, together with transaction-logging arrangements to drive this?
As an example of why you might need this, imagine you've just been informed that a Junior member of your IT staff, thinking they were deleting a collection of test data from the company's Firestore database has actually deleted the company's Customer_Details collection.
You have a copy of the collection from the last time your automated Export ran - but this is now a fortnight old and your company has been acquiring new customers through its online transaction site at a rapid rate. There are no paper records for these customers. You are sunk!
In your previous employment, you'd used a SQL database that automatically created log files that would allow you to recover a corrupted database to its point of failure by reloading it from a dump and then rolling it forward. Firestore doesn't appear to have an equivalent facility. If so, how could you have designed your system to replicate this functionality?
There is what you called Cloud Firestore triggers which handles events in Cloud Firestore with no need to update client code.
You can attempt to write the new update to Firestore. If it succeeds, your .then can resolve the promise and the function can end. If it fails, add some lines to the .catch to update the document with change.before.data()
Below is the sample code:
const admin = require('firebase-admin');
const functions = require('firebase-functions');
admin.initializeApp();
const firestore = admin.firestore();
exports.myFunction = functions.firestore
.document('myCollection/{myDocumentId}')
.onUpdate((change, context) => {
const newValue = change.after.data();
const oldValue = change.before.data();
let myChanges = {
// Update a value
};
return change.after.ref.update(myChanges)
.then(() => {
console.log('Document updated successfully');
return Promise.resolve();
})
.catch(() => {
// The update failed and the document needs to be rolled back
return change.before.ref.set(oldValue);
})
.then(() => {
console.log('Document was rolled back');
return Promise.resolve();
})
.catch(err => {
console.error(err);
return Promise.reject(err);
});
});
The document will then return to its previous value.
For Firestore Backups:
There are various ways to trigger Cloud Firestore backups:
You can also develop your own recovery process by using Cloud Workflows.
Steps to create your own recovery process:
Create Cloud Storage Bucket. (You can skip these if you already have.)
Create the Cloud Workflow definition to execute Firestore export API call.
firestoreExportDocumentsus-central1. Hit Next.- initialize:
assign:
- project: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
- firestoreDatabaseId: (default)
- firestoreBackupBucket: gs://my_bucket_for_backups
- exportFirestoreDatabaseAll:
call: http.post
args:
url: ${"https://firestore.googleapis.com/v1/projects/"+project+"/databases/"+firestoreDatabaseId+":exportDocuments"}
auth:
type: OAuth2
body:
outputUriPrefix: ${firestoreBackupBucket}
result: result
- returnResult:
return: ${result}
Setup IAM permissions to execute the Cloud Workflow.
At this time, you can execute your workflow. The workflow status will show succeeded status when the workflow was able to trigger the Firestore export/backup process.
Setup the invocation via Cloud Scheduler
workflows-datastore-export-backup0 0 * * *. You can generate cron schedules here.HTTP as method choose POSThttps://workflowexecutions.googleapis.com/v1beta/projects/<PROJECT-ID>/locations/us-central1/workflows/<WORKFLOW_NAME>/executionsOAuth, and add your service account previously used, and for Scope use:
https://www.googleapis.com/auth/cloud-platformRun the scheduler.
The output is on your selected Cloud Storage bucket and the format is not self readable, files are packet into many parts and there is a manifest file that resembles the schema and format. To import/restore backups to a Firestore/Datastore instance, you can do that manually from the Datastore Import/Export UI.
UPDATE:
You could also use Scheduled data exports as you mentioned. I included it on the options above.
Based on the scenarios given, once you delete a document in Firestore, the documents are gone for good. Currently, there is no undelete feature but there's a way you can implement an undo feature. If you have a function that deletes a document, you have to modify the delete function to not actually delete the document, and instead just update the UI to remove the view. You would have to schedule the deletion to happen some time later. The undo option would then just restore the visibility of the deleted document and cancel the delayed deletion.
Also, it is highly recommended to consider adding a new boolean field to the document to mark that it's deleted or not, and use that in a filter for the query you pass to FirebaseUI. The field would have to be present on every document in order for it to filter correctly. After the UI is done, you would then have to figure out how to actually delete all the documents you marked for deletion in the UI.
I'd also recommend to create a Feature Request if you want to have this kind of feature.