I have three physical node with docker installed on each of them. I configured Mesos, Marathon, Zookeeper, Hadoop and Flink on each docker. I can run Mesos,Marathon,Zookeeper and hadoop without any problems; also I made a folder flink in hdfs. By the way I configured high available hadoop cluster and run without any issue. But when I use this JSON file to run flink on Marathon, I receive failed task in Mesos UI. The error is in following:
Shutting MesosSessionClusterEntrypoint down with application status FAILED. Diagnostics java.io.IOException: Could not create FileSystem for highly available storage (high-availability.storageDir)
My flink configuration is like this:
env.java.home: /opt/java
jobmanager.rpc.address: 10.32.0.1
high-availability: zookeeper
high-availability.storageDir: hdfs:///flink/ha/
high-availability.zookeeper.quorum: 10.32.0.1:2181,10.32.0.2:2181
recovery.zookeeper.path.mesos-workers: /mesos-workers
mesos.master: 10.32.0.1:5050,10.32.0.2:5050
Would you please tell me how to solve the issue? Do I have to connect flink to mesos?
Thank you in advance.
Problem solved with this configuration:
jobmanager.rpc.address: localhost
high-availability.zookeeper.quorum:
10.32.0.1:2181,10.32.0.2:2181,10.32.0.3:2181
mesos.master: 10.32.0.1:5050,10.32.0.2:5050,10.32.0.3:5050