The following is my shell script to transfer TWO files using scp protocol.
The second scp statement shows the source file directory containing spaces (see line 10 where I added a backslash in Badge\ Docs to denote the subfolder name is "Badge Docs" (it has a space between the two words).
# Copy revenue file
spawn scp ftpaccess@prod:/storage/revenue_bo/*Revenue.xlsx /data/revenue_bo/
expect "ftpaccess@prod's password:\n"
send "mypass\n"
send "exit\n"
expect eof
# Copy PO_REPORT.xlsx
cd /data/storage/exp/Badge_Docs/
spawn scp -T ftpaccess@prod:"/storage/data/Badge\ Docs/PO_REPORT.xlsx" /data/storage/exp/Badge_Docs/
expect "ftpaccess@prod's password:\n"
send "mypass\n"
send "exit\n"
interact
The script works when I ran it in the Linux Server. Both Revenue.xlsx and PO_REPORT.xlsx are picked up/refreshed in their corresponding destination folder.
[linux_prd]$ ./copy_data_reports.sh
spawn scp ftpaccess@prod:/storage/revenue_bo/*Revenue.xlsx /data/revenue_bo/
ftpaccess@prod's password:
Revenue.xlsx 100% 130KB 1.5MB/s 00:00
spawn scp -T ftpaccess@prod:"/storage/data//Badge Docs/PO_REPORT.xlsx" /data/storage/exp/Badge_Docs/
ftpaccess@prod's password:
PO_REPORT.xlsx 100% 20MB 27.5MB/s 00:00
We have set up the copy_data_reports.sh to run via Control-M. Control-M shows this script is executed without issues, BUT I see that only the 'Revenue.xlsx' is being transmitted, while the second file (PO_REPORT.xlsx) is not being transmitted. I am suspecting it might have something to do with the presence of the space in the originating path for the second scp statement (the originating folder path for the first scp does NOT contain spaces, and it worked).
I made sure to enclose the originating folder path name in DOUBLE QUOTES (") AND I kept the backslash to indicate that the path has a space in between ('Badge Docs). Is there something I could have missed?
---- UPDATE -Below is the log file generated by running the copy_data_reports.sh via CONTROL-M:
spawn scp ftpaccess@prod:/storage/revenue_bo/*Revenue.xlsx /data/revenue_bo/
ftpaccess@prod's password:
Revenue.xlsx 100% 130KB 1.5MB/s 00:00
spawn scp -T ftpaccess@prod:"/storage/data//Badge Docs/PO_REPORT.xlsx" /data/storage/exp/Badge_Docs/
ftpaccess@prod's password:<eof>
Interestingly the script job triggered by CONTROL-M seems to end abruptly while running the password execution for the second scp statement.
Is there any difference running the scp shell script from Linux versus the same script being triggered via the CONTROL-M? I dont understand why the job triggered by CONTROL-M terminated prematurely, as the log file from the Control-M run does not show any error messages. Appreciate all helpful inputs.
Someone suggested in the comment section above to replace 'interact' with 'expect eof' and Control-M seems to like it - it finally worked.
# Copy revenue file
spawn scp ftpaccess@prod:/storage/revenue_bo/*Revenue.xlsx /data/revenue_bo/
expect "ftpaccess@prod's password:\n"
send "mypass\n"
send "exit\n"
expect eof
# Copy PO_REPORT.xlsx
cd /data/storage/exp/Badge_Docs/
spawn scp -T ftpaccess@prod:"/storage/data/Badge\ Docs/PO_REPORT.xlsx" /data/storage/exp/Badge_Docs/
expect "ftpaccess@prod's password:\n"
send "mypass\n"
send "exit\n"
expect eof # replacing "interact" -> works with Control-M
The output log generated when Control-M executed the script - it now shows PO_REPORT.xls being transmitted successfully:
spawn scp ftpaccess@prod:/storage/revenue_bo/*Revenue.xlsx /data/revenue_bo/
ftpaccess@prod's password:
Revenue.xlsx 100% 130KB 1.5MB/s 00:00
spawn scp -T ftpaccess@prod:"/storage/data//Badge Docs/PO_REPORT.xlsx" /data/storage/exp/Badge_Docs/
ftpaccess@prod's password:
PO_REPORT.xlsx 100% 20MB 27.5MB/s 00:00
Belatedly realizing, too, that looking at the last line of the log file I posted above of the CONTROL-M triggerd job showing ftpaccess@prod's password:<eof> already giving a hint that I should have used expect eof in the second “spawn scp” statement just as I did in the first “spawn scp” statement. Lesson learned.