database-designcouchdbpouchdb

PouchDB / CouchDB synchronization and database design upgrade tools andgood practices


I am developing an ionic application using PouchDB to store data that I plan to sync with CouchDB and I am wondering what are the tools and/or good practices to manage database design upgrades.

By database design upgrade I mean upgrading the structure of the documents or the algorithm to calculate their ids (I am using DocUri and allDocs queries and may have to update the structure of the ids to meet new requirements) but that could also be indexes upgrades.

Without database synchronization I would just store a design version number in the database, check this version number at application start time and trigger upgrade actions when needed.

Adding database synchronization seems to make things much more complex. Even when using the "one database per user" paradigm, an user may have several instance of the application running in several browser instances or hybrid apps on several devices. If one of these instances performs a database upgrade that may render the data unusable for the other instances. If several instances start a database upgrade at the same time this might lead to conflicts and in the best case to a lot unnecessary traffic and duplication of work.

I think that some kind of orchestration is needed and the basics for a possible scenario could be:

Of course the devil is in the details and that raises more questions:

That seems complex and messy but everyone syncing data must have had the same issue...

How do you handle these upgrades in you applications ?


Solution

  • First off, I'd make updates in the document structure as backwards compatible as possible:

    It's also useful to have a versioning logic:

    So if a schema upgrade is necessary, I'll usually:

    1. Release a client version that supports both the old and the new document structure. Notify the user that he should upgrade.
    2. Once that version has enough penetration, enforce it as required version to prevent modifications by clients with the old version.
    3. Perform a server side migration, if needed.
    4. Release a client version that supports only the new schema.

    In general, I'd try to avoid modifying data directly in the client's PouchDB. Instead, the server should have the central authority and the PouchDBs are as read-only as possible.