Kotlin Native, problem while compiling for multiple targets (only for linux targets)

I have a problem while compiling for androidNativeX64, androidNativeArm64, linuxArm64, linuxX64 targets.

I have a quite complex setup (multi-module gradle project).

I have the core module sqlx4k and 3 client modules sqlx4k-mysql, sqlx4k-postgres and sqlx4k-sqlite (let's call them impl modules).

Each one of those modules "wraps" a Rust library that creates a static library (.a)

.
├── build-logic
│   └── multiplatform-convention
├── sqlx4k
│   ├── rust_lib -> produces static lib libsqlx4k.a
│   └── src
├── sqlx4k-mysql
│   ├── examples
│   ├── rust_lib -> produces static lib libsqlx4k_mysql.a
│   └── src
├── sqlx4k-postgres
│   ├── examples
│   ├── rust_lib -> produces static lib libsqlx4k_postgres.a
│   └── src
└── sqlx4k-sqlite
    ├── examples
    ├── rust_lib -> produces static lib libsqlx4k_sqlite.a
    └── src

In each impl module (eg sqlx4k-mysql) the rust package includes the sqlx4k/rust_lib as dependency. In general the sqlx4k (core module) contains common code for both rust and kotlin codebases.

Of course, each impl module includes the core kotlin module sqlx4k.

Each module has several .def files per target. Here is an example for macosArm64 target:

package = sqlx4k.impl
headers = sqlx4k_postgres.h
compilerOpts = -I./rust_lib/target

staticLibraries = libsqlx4k_postgres.a
libraryPaths = ./rust_lib/target/aarch64-apple-darwin/release

In general, in each impl module, I only link the static rust lib from the same module (I don't link the libsqlx4k.a), since it's already packaged within each one of the impl static libs.

The following setup fails only for linux (also android) targets. For apple targets works totally fine.

The error is:

> Task :sqlx4k-mysql:compileNativeMainKotlinMetadata FAILED
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:12:20 Unresolved reference 'sqlx4k_fetch_all'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:13:20 Unresolved reference 'sqlx4k_of'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:16:20 Unresolved reference 'sqlx4k_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:17:20 Unresolved reference 'sqlx4k_tx_begin'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:18:20 Unresolved reference 'sqlx4k_tx_commit'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:19:20 Unresolved reference 'sqlx4k_tx_fetch_all'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:20:20 Unresolved reference 'sqlx4k_tx_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:21:20 Unresolved reference 'sqlx4k_tx_rollback'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:34:9 Unresolved reference 'sqlx4k_of'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:48:21 Unresolved reference 'sqlx4k_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:52:21 Unresolved reference 'sqlx4k_fetch_all'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:56:30 Unresolved reference 'sqlx4k_tx_begin'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:65:29 Unresolved reference 'sqlx4k_tx_commit'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:71:29 Unresolved reference 'sqlx4k_tx_rollback'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:78:33 Unresolved reference 'sqlx4k_tx_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-mysql/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/mysql/MySQL.kt:86:29 Unresolved reference 'sqlx4k_tx_fetch_all'.

> Task :sqlx4k-sqlite:compileNativeMainKotlinMetadata FAILED
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:12:20 Unresolved reference 'sqlx4k_fetch_all'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:13:20 Unresolved reference 'sqlx4k_of'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:16:20 Unresolved reference 'sqlx4k_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:17:20 Unresolved reference 'sqlx4k_tx_begin'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:18:20 Unresolved reference 'sqlx4k_tx_commit'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:19:20 Unresolved reference 'sqlx4k_tx_fetch_all'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:20:20 Unresolved reference 'sqlx4k_tx_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:21:20 Unresolved reference 'sqlx4k_tx_rollback'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:30:9 Unresolved reference 'sqlx4k_of'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:40:21 Unresolved reference 'sqlx4k_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:44:21 Unresolved reference 'sqlx4k_fetch_all'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:48:30 Unresolved reference 'sqlx4k_tx_begin'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:57:29 Unresolved reference 'sqlx4k_tx_commit'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:63:29 Unresolved reference 'sqlx4k_tx_rollback'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:70:33 Unresolved reference 'sqlx4k_tx_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-sqlite/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/sqlite/SQLite.kt:78:29 Unresolved reference 'sqlx4k_tx_fetch_all'.

> Task :sqlx4k-postgres:compileNativeMainKotlinMetadata FAILED
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:26:20 Unresolved reference 'sqlx4k_fetch_all'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:28:20 Unresolved reference 'sqlx4k_listen'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:29:20 Unresolved reference 'sqlx4k_of'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:32:20 Unresolved reference 'sqlx4k_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:33:20 Unresolved reference 'sqlx4k_tx_begin'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:34:20 Unresolved reference 'sqlx4k_tx_commit'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:35:20 Unresolved reference 'sqlx4k_tx_fetch_all'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:36:20 Unresolved reference 'sqlx4k_tx_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:37:20 Unresolved reference 'sqlx4k_tx_rollback'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:51:9 Unresolved reference 'sqlx4k_of'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:65:21 Unresolved reference 'sqlx4k_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:69:21 Unresolved reference 'sqlx4k_fetch_all'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:73:30 Unresolved reference 'sqlx4k_tx_begin'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:96:13 Unresolved reference 'sqlx4k_listen'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:121:29 Unresolved reference 'sqlx4k_tx_commit'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:127:29 Unresolved reference 'sqlx4k_tx_rollback'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:134:33 Unresolved reference 'sqlx4k_tx_query'.
e: file:///Users/smyrgeorge/dev/projects/test/sqlx4k/sqlx4k-postgres/src/nativeMain/kotlin/io/github/smyrgeorge/sqlx4k/postgres/PostgreSQL.kt:142:29 Unresolved reference 'sqlx4k_tx_fetch_all'.

Which is very weird I think. Because those symbols are the bindings that each impl module creates.

The full project can be found here: https://github.com/smyrgeorge/sqlx4k

UPDATE:

I just saw that if I run

./gradlew clean :sqlx4k-sqlite:build -Ptargets=linuxArm64

Which will build only for linuxArm64 it compiles perfectly.

But, if I run:

./gradlew clean :sqlx4k-sqlite:build -Ptargets=linuxArm64,linuxX64

Which will build for linuxArm64 and linuxX64 fails with the error mentioned above.

The error appears on :sqlx4k-sqlite:compileNativeMainKotlinMetadata task. Maybe there is a common file(s) that is being written during the compilation process and because is happening in parallel it fails. Although for apple targets there is no problem.

I managed to find a solution (although I couldn't find the source of the problem).

The solution was to provide a header file with all the bindings to the core sqlx4k module (although it is not using them all).