[SOLVED] Apache Flink Create Table

Apache Flink Create Table

I'm trying to create table in flink using Table API in Java using eclipse using the following code.

EnvironmentSettings settings = EnvironmentSettings
            .newInstance()
            .inStreamingMode()
            .build();

TableEnvironment tEnv = TableEnvironment.create(settings);

tEnv.executeSql("CREATE TABLE sink_table (" +
            "    `user_a` BIGINT," +
            "    `product` VARCHAR," +
            "    `amount` BIGINT," +
            "    `name_list` ARRAY<STRING>," +
            "    `id_list` ARRAY<INT>," +
            "    PRIMARY KEY (user_a) NOT ENFORCED " +
            ") WITH (" +
            ")");

However program exits without creating any table in Flink SQL Client. How do I create table?

Note: It's getting created directly through SQL client. Also my environment is running Word Count Program through same setup of eclipse.

Also check https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/create/ for similar code.

Solution

In Apache Flink SQL, a table is nothing more than a description of how to interpret data stored (or to be stored) somewhere else. When you create such a table it's necessary to specify where the data actually is (or is to be written): e.g., a Kafka topic, a file, a PostgreSQL table, etc. This information about the data storage location goes in the WITH (...) part of the CREATE TABLE statement.

If you want to work with this table in more than one application, and/or in the SQL Client, you have a couple of options. You can use a copy of the same "CREATE TABLE ..." in each place you want to work with the same table, or you can save the table (not the underlying data, but just the table metadata) in a catalog.

In the Java code you shared in your question, you have created a table in the temporary catalog for that one job. You will need to save it in a persistent catalog if you want the SQL Client to be able to use it directly, or you can create a duplicate table in the SQL Client (this "duplicate" will refer to the same underlying data; only the table metadata would be duplicated).

Since this table is not associated with any data storage, it's not very usable.