AWS managed Flink does not support setting entry class on starting a Flink app?

Read through the documentation of AWS Managed Flink, especially the "Runtime Properties" part, I do not find any mentioning about setting the main class for a given Flink jar.

The example it gives fixates the main class inside the POM.xml.

My case is that I have a multi-purpose Flink App that has multiple entrypoints - represented by multiple main classes - each carries out one specialized kind of job. These jobs, however, are similar in nature so it is reasonable to share a collection of utility classes, and coexist in the same code repository and be shipped together in one single jar file.

Unable to set entryclass on app start means I have to duplicate the shared utility classes and separate jobs into individual jars.

I maybe missed something, so I would like to consult more experienced professionals: Can I conclude that, for the current AWS Managed Flink, it is impossible to set the main class on starting an app, but must be hardcoded into the executable jar?

Solution

That's right! You cannot define an entry point in AWS Managed Flink from the AWS console. However, if you are using maven/gradle, a common workaround is to use profiles and set different mainClass for each profile.

Here's an example:

<profile>
    <id>job-one</id>
    <activation>
        <activeByDefault>true</activeByDefault>
    </activation>
    <properties>
        <project.mainClass>org.example.JobOne</project.mainClass>
    </properties>
    <build>
        <finalName>job-one-${project.version}</finalName>
    </build>
</profile>
<profile>
    <id>job-two</id>
    <properties>
        <project.mainClass>org.example.JobTwo</project.mainClass>
    </properties>
    <build>
        <finalName>job-two-${project.version}</finalName>
    </build>
</profile>

Reference to mainClass: <mainClass>${project.mainClass}</mainClass>

and then build multiple jars: mvn package -Pjob-one and mvn package -Pjob-two

If there are too many jobs and creating multiple jars is a hassle, consider using AWS EMR instead, which offers a lot more flexibility.