I have a Docker Compose system for testing, in which I am doing end-to-end testing of a single-page web app. Several buttons in the web site will result in an FTP connection being initiated in one container (missive-transmitter), going to a test FTP server in another container (missive-testbox).
My FTP logic in PHP always uses "passive" mode, and I think this is causing the problem. I have created a script to run in missive-transmitter, which is a simplified version of the real thing. It is as follows, and is run directly from the console:
<?php
# ftptest.php
error_reporting(-1);
ini_set('display_errors', true);
$conn = ftp_connect('missive-testbox', 21);
$ok1 = ftp_login($conn, 'missive_test', 'password');
if (!$ok1)
{
die("Cannot log in\n");
}
// *** Start problem section
$ok2 = ftp_pasv($conn, true);
if (!$ok2)
{
die("Cannot switch to passive mode\n");
}
// *** End problem section
$info = ftp_systype($conn);
echo "Info: $info\n";
$ok3 = ftp_put($conn, 'ftptest.php', 'ftptest.php', FTP_ASCII);
if (!$ok3)
{
die("Cannot send a file\n");
}
Now, if I remove the *** section (enabling passive mode) then the script will work. If I leave it in, I get this:
Info: UNIX
Warning: ftp_put(): php_connect_nonb() failed: Operation in progress (115) in /root/src/ftptest.php on line 23
Warning: ftp_put(): TYPE is now ASCII in /root/src/ftptest.php on line 23
Cannot send a file
I would like my FTP operation to work in PASV mode.
Oddly, if I install an FTP client then it seems to work in either active or passive modes, which is what I don't understand. On the missive-transmitter side:
~/src $ # This is the `sh` shell in `missive-transmitter`
~/src $ #
~/src $ # Install LFTP in Alpine environment
~/src $ apk add lftp
~/src $ lftp missive_test@missive-testbox
Password:
lftp missive_test@missive-testbox:~> set ftp:passive-mode off
lftp missive_test@missive-testbox:~> put ftptest.php
457 bytes transferred
lftp missive_test@missive-testbox:/> set ftp:passive-mode on
lftp missive_test@missive-testbox:/> put ftptest.php
457 bytes transferred
lftp missive_test@missive-testbox:/>
Is PHP doing something differently, or am I not actually using PASV mode in the console client?
I have confirmed that both containers can ping each other from their respective sh consoles. They are on the same (custom) Docker network.
The missive-testbox Docker container is based on gists/pure-ftpd, so it should be configured correctly as far as I know.
A useful point in an answer below is about how NAT might be making one side make a connection using the wrong IP address. However, the IP addresses appear to be on the same subnet, though I am no networking expert.
From missive-transmitter:
~ # ping missive-testbox
PING missive-testbox (172.19.0.2): 56 data bytes
64 bytes from 172.19.0.2: seq=0 ttl=64 time=0.076 ms
And from missive-testbox:
~ # ping missive-transmitter
PING missive-transmitter (172.19.0.4): 56 data bytes
64 bytes from 172.19.0.4: seq=0 ttl=64 time=0.119 ms
I think the fact they are both 172.19.0.x addresses means they should be able to see each other fully, though I am open to correction on that assumption.
It has been suggested that getting some FTP client or server logs would be a good way to debug this. The client is pretty easy. Here are the same ops as above, but in LFTP's debug mode.
Active mode is first:
~/src # lftp -d missive_test@missive-testbox
Password:
---- Resolving host address...
---- 1 address found: 172.19.0.2
lftp missive_test@missive-testbox:~> set ftp:passive-mode off
lftp missive_test@missive-testbox:~> put ftptest.php
---- Connecting to missive-testbox (172.19.0.2) port 21
<--- 220-Welcome to Pure-FTPd.
<--- 220-You are user number 1 of 5 allowed.
<--- 220-Local time is now 17:54. Server port: 21.
<--- 220-This is a private system - No anonymous login
<--- 220-IPv6 connections are also welcome on this server.
<--- 220 You will be disconnected after 15 minutes of inactivity.
---> FEAT
<--- 530 You aren't logged in
---> AUTH TLS
<--- 500 This security scheme is not implemented
---> USER missive_test
<--- 331 User missive_test OK. Password required
---> PASS XXXX
<--- 230 OK. Current directory is /
---> FEAT
<--- 500 Unknown command
---> PWD
<--- 257 "/" is your current location
---> TYPE I
<--- 200 TYPE is now 8-bit binary
---> PORT 172,19,0,4,159,62
<--- 200 PORT command successful
---> ALLO 457
<--- 500 Unknown command
---> STOR ftptest.php
---- Accepted data connection from (172.19.0.2) port 20
<--- 150 Connecting to port 40766
---- Closing data socket
<--- 226-File successfully transferred
<--- 226 0.000 seconds (measured here), 3.16 Mbytes per second
---> SITE UTIME 20171030154823 ftptest.php
<--- 500 Unknown command
---> SITE UTIME ftptest.php 20171030154823 20171030154823 20171030154823 UTC
<--- 500 Unknown command
457 bytes transferred
OK, that was successful. Here is the passive version in LFTP, again successful.
I notice the warning at the start, about an address needing to be fixed - could that be relevant? If either side advertises itself to the other as "localhost", that might be a problem :-):
lftp missive_test@missive-testbox:/> set ftp:passive-mode on
lftp missive_test@missive-testbox:/> put ftptest.php
---> PASV
<--- 227 Entering Passive Mode (127,0,0,1,117,54)
---- Address returned by PASV seemed to be incorrect and has been fixed
---- Connecting data socket to (172.19.0.2) port 30006
---- Data connection established
---> STOR ftptest.php
<--- 150 Accepted data connection
---- Closing data socket
<--- 226-File successfully transferred
<--- 226 0.000 seconds (measured here), 1.79 Mbytes per second
457 bytes transferred
It is hard to say which FTP operations are done here. But it might be that PHP is using PASV while lftp is using EPSV to set the passive mode.
In case of PASV the server sends both IP address and port number where it will await the connection. With EPSV the server instead only provides the port number and the target IP address is the one from the current FTP control connection. If NAT (network address translation) is involved (which is not unlikely within Docker setups) the server might see a different internal IP address as its own compared to the one which is externally visible from the FTP client, which means that the client cannot connect to the (wrong) IP address given in the response to the PASV command. With EPSV this problem does not exists since the client does not use a server provided IP address as target.