ftpftp-client

use FTP (only option) to sync files in a directory with a single line command in linux


I need to synchronize files in my computer with server to which I have only FTP access and I would prefer a single line to add to crontab which checks every day that new or updated files are transferred (overwritten).

The question has been asked years ago but no simple answer was forthcoming and I want to know if there are better solutions today than ncftput, wput etc. which do not allow to

ncftpput -R -z -u "USER" -p "PASS" webxx.at /dir/ /source/

is rumored to work, but the -z switch seems "of label" use. My experiments seem to indicate that times are not reliable checked.


Solution

  • Just put this code into a file FtpSync.sh and add this file into your crontab.
    You can adjust the parameters in the file. I tried to create "speaking" parameters so that they explain themselves.
    A call of this script will either download or upload files (depending on the parameter CopyServerdataToLocal). If you want to do both, simply start the script twice with different parameters (or create two script files).

    FtpSync.sh

    #!/bin/bash
    # Michael Hutter, 20.11.2021
    # This Script can be used to synchronize a remote FTP directory and a local directory
    HOST='ftp.mywebspace.de'
    USER='web1234567f1'
    PASS='YourSecretPwd'
    SERVERFOLDER='/FolderOnWebspace'
    PCFOLDER='/home/michael/ftpsync/MyLocalFolder'
    CopyMoreThanOneFileSimultaneously="--parallel=10"
    
    CopyServerdataToLocal=1 # 0=Upload, 1=Download
    IgnoreSubdirectories=1
    ContinuePartiallyTransmittedFiles=0
    DontOverwriteNewerExistingFiles=0 # If this is used ContinuePartiallyTransmittedFiles will not work!
    
    DeleteAdditionalFilesInDestinationDir=0 # Deletes files in DestDir which are not present in SourceDir
    RemoveSourceFilesAfterTransfer=0 # Moves the files instead of copying them
    
    ExcludeParams='--exclude-glob .* --exclude-glob .*/' # Exclude (hidden) files and direcories -> starting with a dot
    
    OnlyShowChangesButDontChangeFiles=0 # DryRun mode
    OutputAsMuchInfoAsPossible=1 # Verbose mode
    
    ################################################################################
    
    if [ $CopyServerdataToLocal -eq 1 ]; then
       if [ $OutputAsMuchInfoAsPossible -eq 1 ]; then
          echo "Modus=Download"
       fi
       DIRECTORIES="$SERVERFOLDER $PCFOLDER"
    else
       if [ $OutputAsMuchInfoAsPossible -eq 1 ]; then
          echo "Modus=Upload"
       fi
       REVERSE='--reverse'
       DIRECTORIES="$PCFOLDER $SERVERFOLDER"
    fi
    if [ $IgnoreSubdirectories -eq 1 ]; then
       IGNORESUBDIRS='--no-recursion'
    fi
    
    if [ $ContinuePartiallyTransmittedFiles -eq 1 ]; then
       CONTINUEFILES='--continue'
    fi
    if [ $DontOverwriteNewerExistingFiles -eq 1 ]; then
       ONLYNEWER='--only-newer'
    fi
    if [ $DeleteAdditionalFilesInDestinationDir -eq 1 ]; then
       DELETE='--delete'
    fi
    if [ $RemoveSourceFilesAfterTransfer -eq 1 ]; then
       REMOVE='--Remove-source-files'
    fi
    if [ $OnlyShowChangesButDontChangeFiles -eq 1 ]; then
       DRYRUN='--dry-run'
    fi
    if [ $OutputAsMuchInfoAsPossible -eq 1 ]; then
       VERBOSE='--verbose'
    fi
    
    lftp -f "
    open $HOST
    user $USER $PASS
    lcd $PCFOLDER
    mirror $DRYRUN $REVERSE $IGNORESUBDIRS $DELETE $REMOVE $CONTINUEFILES $ONLYNEWER $CopyMoreThanOneFileSimultaneously --use-cache $ExcludeParams $VERBOSE $DIRECTORIES
    bye
    "