I am using incremental backups using rsnapshot combined with a custom
cmd_cp and cmd_rm to make use of btrfs snapshots, this procudes multiple
daily btrfs subvolumes:
.sync
daily.0
daily.1
daily.2
.sync is the folder that gets synced to using SSH from the server I backup.
When completed this happens:
mv daily.2/ daily.3/
mv daily.1/ daily.2/
mv daily.0/ daily.1/
rsnapshot_cp_btrfs -al .sync daily.0
The cp command translates into btrfs subvolume snapshot -r .sync daily.0.
This all works great. But now I want to sync all backups to an other server too, so I have a full mirror of all backups. This sync should always work, even if it would be synced a week after (maybe due SSH connection issues).
Normally syncing would be easy using btrfs send and receive using parent snapshots as described on the wiki: https://btrfs.wiki.kernel.org/index.php/Incremental_Backup#Doing_it_by_hand.2C_step_by_step
I imagined a loop that just sends all daily folders and maintaining the old backup for parent reference.
But in this case daily.0 moves to daily.1, 1 to 2, and so on. So this would not work.
I could send a simple mv to the remote server, but I can't trust this since
on any errors, the folder structures would not be proper a day later. I want a true mirror, but making use of the btrfs tools.
Did anyone managed a similar situation or know the best way to clone all subvolumes to the other server?
Big thanks!
I solved it! I created a bash script that syncs all snapshots with a date in the name to the remote server. The date is subtracted from btrfs subvolume show.
So daily.0 can become 2021-09-20-08-44-46 on the remote.
I sync backwards. daily.30 first. daily.0 last. This way I can pass the
proper parent to btrfs send. E.g.: btrfs send -p daily.30 daily.29.
If the date named snapshot exists on the remote, I check with btrfs subvolume show
whether it was properly synced. If not, I delete the remote subvolume/snapshot and
re-sync. If it was already synced, I will skip the sync. A proper synced subvolume/snapshot has a Received UUID and readonly flag.
After syncing I compare all snapshot names of the remote to what was just synced. The difference will deleted (thus old snapshots).
I might share the code in the future when it's all been stable for a long run. For now I hope the above information will help others!