bashcp

find and cp command don't work in my script but they work in my terminal


In my current directory, I have a list (list.txt) file like this

Container ID
W1412886
W1402878
W1402821
W1402789
W1402792
...

I also have folders that contains files whose file name start with the values shown in the previous list.

I also have a script with which I want to loop over each element of my list and copy the found files into a directory called FINAL. This script is pretty simple but for a reason I don't know is not working

#!/bin/bash

# Path to your file containing container IDs
container_file="list.txt"
# Destination folder
destination_folder="FINAL"

# Create destination folder if it doesn't exist
mkdir -p "$destination_folder"

# Skip the header and process each container ID in the file
tail -n +2 "$container_file" | while IFS= read -r container_id || [[ -n "$container_id" ]]; do
    # Check if the line is not empty
    if [[ -n "$container_id" ]]; then
        echo "Processing Container ID: $container_id"  # Debugging line
        # Find files that start with the container ID and copy them to the FINAL folder
        find . -type f -name ${container_id}* -exec cp {} "$destination_folder" \;
    fi
done

I get the Container ID from the echo but it cannot copy. Interestingly if I run the following command, this works

 find . -type f -name W1412886* -exec cp {} FINAL \;

Do you have any idea why it is working in a single command but the loop (that seems to work) is not copying? I am using git bash


Solution

  • A reworked (minor changes,comments removed) of your script, using a 'dos' formatted list.txt file.

    As others mentioned, list.txt is probably CRLF (aka windows aka dos formatted records) is the root issue, also added -print to 'find....', cater for dos formatted records - works without change on standard 'unix' text files

    $ ls -l W*
    -rw-rw-r-- 1 tester tester 0 Nov  3 21:49 W1402789.0
    -rw-rw-r-- 1 tester tester 0 Nov  3 21:49 W1402789.1
    ---------- 1 tester tester 0 Nov  3 22:14 W1402792
    -rw-rw-r-- 1 tester tester 0 Nov  3 21:49 W1402878.1
    -rw-rw-r-- 1 tester tester 0 Nov  3 21:49 W1402878.2
    -rw-rw-r-- 1 tester tester 0 Nov  3 21:49 W1402878.3
    
    $ file list.txt
    list.txt: ASCII text, with CRLF line terminators
    
    $ cat list.txt
    Container ID
    W1412886
    W1402878
    
    W1402821
    W1402789
    W1402792
    
    $ cat script.sh 
    #!/bin/bash
    
    container_file="list.txt"
    destination_folder="FINAL"
    
    mkdir -p "$destination_folder"
    
    while IFS= read -r container_id
    do
        if [[ -n "$container_id" ]]; then
            echo "Processing Container ID: $container_id"
            find . -type f -name "${container_id}*" -print -exec cp -f {} "$destination_folder" \;
        fi
    done <<< $(tail -n +2 "$container_file" | tr -d '\r') # cater for 'dos' formatted lines
    
    $ ./script.sh
    Processing Container ID: W1412886
    Processing Container ID: W1402878
    ./W1402878.2
    ./W1402878.1
    ./W1402878.3
    Processing Container ID: W1402821
    Processing Container ID: W1402789
    ./W1402789.0
    ./W1402789.1
    Processing Container ID: W1402792
    ./W1402792
    cp: cannot open './W1402792' for reading: Permission denied
    
    $ ls FINAL/
    W1402789.0  W1402789.1  W1402878.1  W1402878.2  W1402878.3
    

    hope this helps, or let the community know if you've resolved your issue.