LFTP in gitlab CI: files are not updated on FTP server even if they are changed in the last commit

we have an issue with GITLAB-CI pipelines when we commit a new modification on the server. We want to update the modification on the FTP server automatically.

To do that, we are using a simple CI configuration in gitlaci.yaml file:

deploy:
  script:
    - bash ./deploy.sh
  only:
    - develop
    - master

The content of deploy.sh is very simple too:

#!/bin/bash

echo "Deployment start";

echo "*** Target branch test:"
target="null"
case "$CI_COMMIT_REF_NAME" in
"develop")
    target=$DEV
    ;;
"master")
    target=$PROD
    ;;
*)
    echo "Pas de deployment sur cette branch"
    exit 1;
    ;;
esac
echo "*** $target"


echo "*** Install client FTP"
apt-get update -qq && apt-get install -y -qq lftp

echo "*** Copy file to target"
lftp -c "set ftp:ssl-allow no; open -u $USERNAME,$PASSWORD $HOST; mirror -Rnev ./ $target --ignore-time --parallel=10 --exclude-glob .git* --exclude .git/"

(the variables are will setup in CI variable, and most of the time it is working)

When we try to deploy changes by pipeline we have

sometimes he sends the files
sometimes don't send anything, zero files, even the ones changed in the last commit

(Reading database ... 29705 files and directories currently installed.)
Preparing to unpack .../lftp_4.7.4-1_amd64.deb ...
Unpacking lftp (4.7.4-1) ...
Setting up lftp (4.7.4-1) ...
*** Copy file to the target
Job succeeded

Any ideas?

The job is succeeded, but the commited file are not send on the FTP...
Which is a bit strange... How can I have a pipeline if no file is changed ^^ ??

Do you know what's happened?

Thank you,
Regards,
Nicolas

Additionnal log during my last issue. The related file updated in gitLab was trombi/index.php, it does not appear on the log... And on the server, it did not change at all:

 $ bash ./deploy.sh
 *** Déploiement de xxx - start:
 *** Target branche:
 *** ./www/xxxDev
 *** voir le resultat sur: xxx/Dev
 *** Install client FTP
 debconf: delaying package configuration, since apt-utils is not installed
 Selecting previously unselected package libtcl8.6:amd64.
 (Reading database ... 23968 files and directories currently installed.)
 Preparing to unpack .../libtcl8.6_8.6.9+dfsg-2_amd64.deb ...
 Unpacking libtcl8.6:amd64 (8.6.9+dfsg-2) ...
 Selecting previously unselected package tcl8.6.
 Preparing to unpack .../tcl8.6_8.6.9+dfsg-2_amd64.deb ...
 Unpacking tcl8.6 (8.6.9+dfsg-2) ...
 Selecting previously unselected package tcl-expect:amd64.
 Preparing to unpack .../tcl-expect_5.45.4-2_amd64.deb ...
 Unpacking tcl-expect:amd64 (5.45.4-2) ...
 Selecting previously unselected package expect.
 Preparing to unpack .../expect_5.45.4-2_amd64.deb ...
 Unpacking expect (5.45.4-2) ...
 Selecting previously unselected package lftp.
 Preparing to unpack .../lftp_4.8.4-2_amd64.deb ...
 Unpacking lftp (4.8.4-2) ...
 Setting up lftp (4.8.4-2) ...
 Setting up libtcl8.6:amd64 (8.6.9+dfsg-2) ...
 Setting up tcl8.6 (8.6.9+dfsg-2) ...
 Setting up tcl-expect:amd64 (5.45.4-2) ...
 Setting up expect (5.45.4-2) ...
 Processing triggers for mime-support (3.62) ...
 Processing triggers for hicolor-icon-theme (0.17-2) ...
 Processing triggers for libc-bin (2.28-10) ...
 *** Copy file to target
 Mirroring directory `.gitlab'
 Mirroring directory `adm'
 Mirroring directory `xxx'
 Mirroring directory `communaute'
 Mirroring directory `conso'
 Mirroring directory `cron'
 Mirroring directory `css'
 Mirroring directory `doc'
 Mirroring directory `eve'
 Mirroring directory `img'
 Finished mirror `img'
 Mirroring directory `js'
 Mirroring directory `ad/comi'
 Mirroring directory `comm/img'
 Finished mirror `conso'
 Mirroring directory `materiel'
 Finished mirror `js'
 Mirroring directory `monCompte'
 Mirroring directory `.gitlab/issue_templates'
 Finished mirror `association'
 Mirroring directory `money'
 Finished mirror `css'
 Mirroring directory `xxx'
 Mirroring directory `documents/parsedown-contents'
 Mirroring directory `events/championnats'
 Finished mirror `cron'
 Mirroring directory `proto'
 Finished mirror `adm/comite'
 Mirroring directory `adm/doc-legaux'
 Finished mirror `communaute/img'
 Mirroring directory `communaute/jou'
 Finished mirror `materiel'
 Mirroring directory `public'
 Finished mirror `.gitlab/issue_templates'
 Finished mirror `.gitlab'
 Mirroring directory `te'
 Finished mirror `monCompte'
 Mirroring directory `webServices'
 Finished mirror `xxx'
 Mirroring directory `admin/emails'
 Finished mirror `documents/parsedown-contents'
 Mirroring directory `documents/parsedown-master'
 Mirroring directory `money/cotisations'
 Finished mirror `events/championnats'
 Mirroring directory `events/festivals'
 Mirroring directory `proto/auth'
 Finished mirror `adm/do'
 Mirroring directory `adn/jeux'
 Finished mirror `communaute/jou'
 Mirroring directory `communaute/mem'
 Mirroring directory `public/noel'
 Mirroring directory `te/con'
 Finished mirror `adm/emails'
 Mirroring directory `adm/ppp'
 Finished mirror `events/fest'
 Mirroring directory `events/tour'
 Finished mirror `proto/auth'
 Finished mirror `proto'
 Mirroring directory `money/devis'
 Mirroring directory `communaute/membres/Ajouter'
 Finished mirror `te/cont'
 Finished mirror `te'
 Mirroring directory `communaute/membres/messageCollectif'
 Finished mirror `adm/popp'
 Mirroring directory `adm/reglement-interieur'
 Finished mirror `events/tour'
 Finished mirror `events'
 Mirroring directory `communaute/membres/trombi'
 Finished mirror `communaute/membres/Ajouter'
 Mirroring directory `communaute/membres/urgence'
 Finished mirror `communaute/membres/messageCollectif'
 Mirroring directory `communaute/outils'
 Finished mirror `adm/reglement-interieur'
 Mirroring directory `admin/roles-engagements-avantages'
 Finished mirror `communaute/membres/trombi'
 Mirroring directory `communaute/partenaires'
 Mirroring directory `public/noel/css'
 Finished mirror `communaute/membres/urgence'
 Finished mirror `communaute/membres'
 Mirroring directory `communaute/sponsors'
 Mirroring directory `communaute/outils/association'
 Finished mirror `admin/roles-engagements-avantages'
 Mirroring directory `admin/settings'
 Finished mirror `communaute/partenaires'
 Mirroring directory `communaute/outils/classements'
 Finished mirror `public/noel/css'
 Mirroring directory `public/noel/fonts'
 Finished mirror `communaute/sponsors'
 Mirroring directory `communaute/outils/smash-rules-sets'
 Finished mirror `communaute/outils/association'
 Mirroring directory `communaute/outils/teams'
 Finished mirror `admin/settings'
 Mirroring directory `public/noel/img'
 Finished mirror `communaute/outils/classements'
 Mirroring directory `public/noel/js'
 Mirroring directory `wwws/compte'
 Mirroring directory `money/cotisations/fiches'
 Finished mirror `admin/jeux'
 Finished mirror `documents/parsedown-master'
 Finished mirror `admin'
 Finished mirror `documents'
 Mirroring directory `money/img'
 Mirroring directory `money/note2Frais'
 Finished mirror `public/noel/fonts'
 Mirroring directory `money/tresorerie'
 Finished mirror `communaute/outils/smash-rules-sets'
 Mirroring directory `wwws/database'
 Finished mirror `communaute/outils/teams'
 Finished mirror `communaute/outils'
 Finished mirror `communaute'
 Mirroring directory `wwws/devis'
 Finished mirror `public/noel/img'
 Mirroring directory `wwws/discord'
 Finished mirror `public/noel/js'
 Finished mirror `public/noel'
 Finished mirror `public'
 Mirroring directory `wwws/email'
 Finished mirror `money/cotisations/fiches'
 Finished mirror `money/cotisations'
 Mirroring directory `wwws/emoji'
 Finished mirror `wwws/compte'
 Mirroring directory `wwws/files'
 Finished mirror `money/img'
 Mirroring directory `wwws/googleMaps'
 Mirroring directory `money/note2Frais/min'
 Finished mirror `money/tresorerie'
 Mirroring directory `money/note2Frais/dons'
 Finished mirror `wwws/devis'
 Mirroring directory `wwws/jou'
 Finished mirror `wwws/discord'
 Mirroring directory `wwws/membres'
 Finished mirror `money/devis'
 Mirroring directory `wwws/permissions'
 Finished mirror `wwws/dda'
 Mirroring directory `wwws/public'
 Finished mirror `wwws/email'
 Mirroring directory `wwws/resssppp'
 Finished mirror `wwws/emoji'
 Finished mirror `wwws/googleMaps'
 Finished mirror `wwws/files'
 Finished mirror `mon/note2Frais/aport'
 Finished mirror `mon/note2Frais/dons'
 Finished mirror `mon/note2Frais'
 Finished mirror `mon'
 Finished mirror `wwws/jou'
 Finished mirror `wwws/membres'
 Finished mirror `wwws/public'
 Finished mirror `wwws/permissions'
 Finished mirror `wwws/resetPasswords'
 Finished mirror `wwws'
 *** End
 ********* ATTENTION si pas de ligne entre 'End' et 'Copy file to target' alors il faut re lancer le déploiement
Running after script
00:02
Saving cache
00:02
Uploading artifacts for successful job
00:01
 Job succeeded

I confirm, the pipeline is running for a commit which has updated file:

Running with gitlab-runner 12.9.0-rc1 (a350f628)
   on docker-auto-scale ed2dce3a
Preparing the "docker+machine" executor
00:32
 Using Docker executor with image ruby:2.5 ...
 Pulling docker image ruby:2.5 ...
 Using docker image sha256:5a76bd71024b1c46b1e8871b205d025b03abe85ea for ruby:2.5 ...
Preparing environment
00:05
 Running on runner-ed2dce3a-project-10794060-concurrent-0 via runner-ed2dce3a-srm-1584724660-6891f84d...
Getting source from Git repository
00:03
 $ eval "$CI_PRE_CLONE_SCRIPT"
 Fetching changes...
 Initialized empty Git repository in /builds/xxx/prod/.git/
 Created fresh repository.
 From https://gitlab.com/xxx/prod
  * [new ref]         refs/pipelines/128249282 -> refs/pipelines/128249282
  * [new branch]      develop                  -> origin/develop
  * [new branch]      master                   -> origin/master
 Checking out 0f5997df as develop...

And the commit 0f5997df contains: Showing 1 changed file with 3 additions and 3 deletions

Solution

At the end, I finaly decide to move my projet from gitlab to github. The CI/CD is pretty the same, except the need/possibility to split the yaml files into each target env.

This is what I did and one file for dev, an other for prod. The contents of the two files are pretty the same, I show you the content of DEV file, the only diff is for URL_DEV secret value

on:
  push:
    branches:
      - develop
name: Publish on https://www.mydomain.fr/myprojectname
jobs:
  FTP-Deploy-Action:
    name: FTP-Deploy-Action
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2.1.0
      with:
        fetch-depth: 2
    - name: FTP-Deploy-Action
      uses: SamKirkland/FTP-Deploy-Action@3.1.1
      with:
        ftp-server: ${{ secrets.FTP_HOSTNAME }}/${{ secrets.URL_DEV }}/
        ftp-username: ${{ secrets.FTP_USERNAME }}
        ftp-password: ${{ secrets.FTP_PASSWORD }}

All the secrets variables are the same as I had in gitlab;

By doing this change, I solved the issue related to this ticket + I removed the limitation due to gitlab shared runner minute

ByeBye GitLab, Hello GitHub...And thank to everybody who help me to find a fix