Automate sync via git between repo and server, ignoring but not removing files in .gitignore from source

I would like to push and pull from a GitHub repository in a slightly complicated setup.

CMS/Server ⇄ GitHub ⇄ Translation Service

I'm using a flat file CMS which stores content as .txt files, one per language: *.en.txt, *.de.txt, *.es.txt. It also stores the multimedia files in the same directories, *.png, .mp4, *.jpg, etc.

My .gitignore excludes all the media files from the repo as they are not relevant to the Translation Service.

The *.txt files need to be pushed to the GitHub repo when changes are made via the CMS. The Translation Service reads the *.en.txt files and deposits/updates the other language files (*.en.txt, *.de.txt, *.es.txt) once per 24hrs back into the repo. These then need to be pulled by the server back into the CMS, whilst not removing the files that aren't in the repo, like multimedia files.

I would like to script and automate the push/pull process between the server/CMS and the GitHub repo making sure that the media files are not removed from the server/CMS when fetching/pulling from the repo, even though they are not in the repo.

I'm currently handling this process manually. Could someone outline the approach that one might use to set this relationship up to be automated?

Let me know if you need any more information.

Solution

Given that I have a ci user and sudo running different parts of this process, the thing that was causing the most issues was having the incorrect options --flags on the various steps, i.e.

--recursive and --delete on the initial backup, so that recently deleted items are removed from the previous backup
---no-group when merging the latest changes from Github to prevent rsync errors from file permissions
--no-group and delete when merging changes from backup to Github, so that deleted items are removed from Github and errors from file permissions from users.

My solution:

Make a copy of current web folder:

rsync --recursive --links --delete /directory/web /directory/backup

Merge new changes from Git folder to web folder:

rsync -aO --no-group --exclude='.git/' --exclude='.gitignore' --exclude='*.en.txt' /directory/git /directory/web

Merge new changes from backup folder to Git folder:

rsync -aO --no-group --delete --filter='P .gitignore' --exclude='.git/' --include='*/' --include='*.en.txt' --exclude='*' /directory/backup/ /directory/web

Hope this answer helps someone else in this predicament.