I'm working with FTPClient against an FTP server using Testcontainers.
A reproducible code sample is here:
import org.apache.commons.net.PrintCommandListener;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPReply;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.images.builder.ImageFromDockerfile;
import org.testcontainers.junit.jupiter.Testcontainers;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
import static org.assertj.core.api.Assertions.assertThat;
@Testcontainers
class FtpUtilsTest {
private static final int PORT = 21;
private static final String USER = "user";
private static final String PASSWORD = "password";
private static final int FTP_TIMEOUT_IN_MILLISECONDS = 1000 * 60;
private static final GenericContainer ftp = new GenericContainer(
new ImageFromDockerfile()
.withDockerfileFromBuilder(builder ->
builder
.from("delfer/alpine-ftp-server:latest")
.build()
)
)
.withExposedPorts(PORT)
.withEnv("USERS", USER + "|" + PASSWORD);
@BeforeAll
public static void staticSetup() throws IOException {
ftp.start();
}
@AfterAll
static void afterAll() {
ftp.stop();
}
@Test
void test() throws IOException {
FTPClient ftpClient = new FTPClient();
ftpClient.setDataTimeout(FTP_TIMEOUT_IN_MILLISECONDS);
ftpClient.setConnectTimeout(FTP_TIMEOUT_IN_MILLISECONDS);
ftpClient.setDefaultTimeout(FTP_TIMEOUT_IN_MILLISECONDS);
// Log
ftpClient.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
// Connect
try {
ftpClient.connect("localhost", ftp.getMappedPort(PORT));
ftpClient.setSoTimeout(FTP_TIMEOUT_IN_MILLISECONDS);
int reply = ftpClient.getReplyCode();
if (!FTPReply.isPositiveCompletion(reply)) {
ftpClient.disconnect();
throw new AssertionError();
}
// Login
boolean loginSuccess = ftpClient.login(USER, PASSWORD);
if (!loginSuccess) {
throw new AssertionError();
}
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
ftpClient.enterLocalPassiveMode();
} catch (IOException e) {
throw new AssertionError(e);
}
String remoteFile = "fileonftp";
try (InputStream targetStream = new ByteArrayInputStream("Hello FTP".getBytes())) {
assertThat(ftpClient.isConnected()).isTrue();
ftpClient.storeFile(remoteFile, targetStream);
}
}
}
This prints:
220 Welcome Alpine ftp server https://hub.docker.com/r/delfer/alpine-ftp-server/
USER *******
331 Please specify the password.
PASS *******
230 Login successful.
TYPE I
200 Switching to Binary mode.
PASV
227 Entering Passive Mode (172,17,0,3,82,15).
[Replacing PASV mode reply address 172.17.0.3 with 127.0.0.1]
then fails with:
Connection refused (Connection refused)
java.net.ConnectException: Connection refused (Connection refused)
...
at java.base/java.net.Socket.connect(Socket.java:609)
at org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:866)
at org.apache.commons.net.ftp.FTPClient._storeFile(FTPClient.java:1053)
at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:3816)
at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:3846)
What I don't understand is that it fails after successfully connecting and logging in, and returning true for isConnected.
Turns out that when removing the ftpClient.enterLocalPassiveMode(); it works, but I need it to work with the passive mode.
I guess the failure is related when switching to a different port for the passive call.
but when trying to add the ports to the withExposedPorts the container fails to start with:
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [55600, 55601, 55602, 55603, 55604, 55605, 55606, 55607, 55608, 55609, 55598, 55599] should be listening)
Running against a docker (docker run -d -p 21:21 -p 21000-21010:21000-21010 -e USERS="user|password" delfer/alpine-ftp-server) works.
Local docker versions:
Testcontainers - appears to behave the same both on 1.16.2 and 1.15.3
Link to testcontainers discussion
As you already figured out in the comments, the tricky part about FTP passive mode is that the server uses another port (not 21) for communication.
In the docker image you're using, it's a port from the 21000-21010 range by default. So you need to publish (expose) these additional container ports. In docker run command you used -p 21000-21010:21000-21010 for that.
However, Testcontainers library is designed to publish to random host ports to avoid the problem, when a desired fixed port (or a range of ports) is already occupied on the host side.
In case of FTP passive mode random ports on the host side cause problems, because afaik you can't instruct the ftp client to override the port, which FTP server returned for the passive mode. You'd need something like ftpClient.connect("localhost", ftp.getMappedPort(PORT)); but for passive mode ports as well.
Therefore the only solution I see here is to use a FixedHostPortContainer. Even though it's marked as deprecated and not recommended to use because of the mentioned issues with occupied ports, I think this is a valid use case for it here. FixedHostPortGenericContainer allows to publish fixed ports on the host side. Something like:
private static final int PASSIVE_MODE_PORT = 21000;
...
private static final FixedHostPortGenericContainer ftp = new FixedHostPortGenericContainer<>(
"delfer/alpine-ftp-server:latest")
.withFixedExposedPort(PASSIVE_MODE_PORT, PASSIVE_MODE_PORT)
.withExposedPorts(PORT)
.withEnv("USERS", USER + "|" + PASSWORD)
.withEnv("MIN_PORT", String.valueOf(PASSIVE_MODE_PORT))
.withEnv("MAX_PORT", String.valueOf(PASSIVE_MODE_PORT));
Keep in mind that this solution relies on the assumption that 21000 port is always free. If you're going to run this in the environment where it's not guaranteed, then you need to tweak it to find a free host port first. Like:
private static FixedHostPortGenericContainer ftp = new FixedHostPortGenericContainer<>(
"delfer/alpine-ftp-server:latest")
.withExposedPorts(PORT)
.withEnv("USERS", USER + "|" + PASSWORD);
@BeforeAll
public static void staticSetup() throws Exception {
Integer freePort = 0;
try (ServerSocket socket = new ServerSocket(0)) {
freePort = socket.getLocalPort();
}
ftp = (FixedHostPortGenericContainer)ftp.withFixedExposedPort(freePort, freePort)
.withEnv("MIN_PORT", String.valueOf(freePort))
.withEnv("MAX_PORT", String.valueOf(freePort));
ftp.start();
}