androidunity-game-enginegradlebuildandroid-multidex

How to find and fix source of android build failure ':transformClassesWithMultidexlistForRelease' in Unity project


This game has been out for several years, and has accumulated lots of old dependencies, like old versions of Swrve, IronSource, Crashlytics (using Fabric), Facebook, and others. I'm in the process of updating them, which caused all sorts of conflicts and build failures.

I am stuck on this latest build failure:

Execution failed for task ':transformClassesWithMultidexlistForRelease'.
> com.android.build.api.transform.TransformException: Error while generating the main dex list.

This is not very specific, but I did look into ensuring that multidex is enabled for the project (it is, and if I turn it off I get a different error that is clear about exceeding the number of methods for a single DEX).

I exported a gradle project and attempted to build from the command line, using the --debug option for more details. Note, I couldn't use system gradle (v6.6) because the version of the project that Unity generates uses deprecated features (v5.1.1). Instead, I made an alias to the internal gradle command Unity uses. I extracted the part that fails here.

01:38:01.777 [INFO] [com.android.build.gradle.internal.transforms.D8MainDexListTransform] Generating the main dex list using D8.
01:38:01.778 [INFO] [com.android.build.gradle.internal.transforms.D8MainDexListTransform] Program files:

 /Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/unity-android-resources/build/intermediates/intermediate-jars/release/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/f1a15ce9a3a238f913b1713c5b7eef8e/jars/classes.jar
/Users/turkeybash/.gradle/caches/modules-2/files-2.1/androidx.arch.core/core-common/2.0.0/bb21b9a11761451b51624ac428d1f1bb5deeac38/core-common-2.0.0.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/eafba0cd0d7e84442d378b6269357dc1/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/IronSource/libs/hyprmxadapter.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/6673f931a57b5dd7ba35e730131945d3/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/f723d303fde323253ef3670a7230198e/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/5315d1a553466e092ce2c4adb36c70ed/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/4da13394b8dfee0f81ebf3ba0d5cf1e1/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/3b6be90f862d2551f8ea28852889473f/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/93d642daf13ca919edc2114bcf171b98/jars/classes.jar
/Users/turkeybash/.gradle/caches/modules-2/files-2.1/androidx.annotation/annotation/1.1.0/e3a6fb2f40e3a3842e6b7472628ba4ce416ea4c8/annotation-1.1.0.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/b3224239e209e5cae589d6955f71e794/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/8012f241b1a94c937ae18da3cafcaccc/jetified-gson-2.8.5.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/061305bdad448d4ef850486898ce319e/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/a38b28179d7ba502e2bc5420005e1c81/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/d7033b64e7b421d923c873c8e1204c36/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/ee22c2bc5bb5d17778d30fd2f0d75f25/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/libs/AppsFlyerAndroidPlugin.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/28a19cfa10dc0aa1eb2571ec47520027/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/06c285be385b1e39ea6786bcb5fc9220/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/09ccabea0402ffea9ca87f157a761f6e/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/GooglePlayGamesManifest.plugin/build/intermediates/intermediate-jars/release/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/78542fe719422e18a588a7947da0926f/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/dd2c404be1e461c8a1f58455b6604c5d/jars/classes.jar
/Users/turkeybash/.gradle/caches/modules-2/files-2.1/com.parse.bolts/bolts-android/1.4.0/cc174c559b5177982887bf6e1b76003aebad9516/bolts-android-1.4.0.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/549f8a7eb2e15c3019a92607210809fa/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/073d4008289eb1e150f9b3f369e185b4/jetified-DisruptorBeamAndroid.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/libs/kotlin-stdlib-1.2.31.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/75cf5c04b9976659a3ef67ac8d77f145/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/Firebase/build/intermediates/intermediate-jars/release/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/3dc95681308d227c6e5bc752ff93fc3c/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/libs/uniClipboard.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/c3d32deac353d5f1fb471db3a02a1131/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/1fbd554b105b4246e31c79a86b00cc7e/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/e49f42c281104d5fb717dc445b75f1cd/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/1b32181da5afc525e5857e71b55891d1/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/f7f3a0f5bd79255fd4e62549d5bc98eb/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/bc4cfca8c8e09c91af448d0011106a82/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/e5784d07c4ad4a243e10bc62559afb45/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/c83bed95635284252e516d2c7a1b9d61/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/6dfc4b939a9d42374c14cf762b2b43e4/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/d6108d0a33d4cda276a38c5166a26dad/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/306398bc2747158d3587dcc5ddf49477/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/SwrveSDKPushSupport/build/intermediates/intermediate-jars/release/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/d0230fe767ceb118a398ca2c730bf7dd/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/3e9b0c3d5fe551f4ea2a4867ce977cf9/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/libs/unity-classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/a7f0290a194fcc555ea898d4bde46740/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/ffd42a1f3b628f23f04a3a39ad335022/jetified-core-3.3.3.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/cf254de769b28738e6c63506e5f82686/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/743a6a717d00e80cc4b0f8b3839a1ee9/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/a31d30846755b3ff6b2f03ee7b410983/jars/classes.jar
/Users/turkeybash/.gradle/caches/modules-2/files-2.1/androidx.lifecycle/lifecycle-common/2.0.0/e070ffae07452331bc5684734fce6831d531785c/lifecycle-common-2.0.0.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/c30fdf68f92f178ea72e5d137f390b3e/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/fd760d17092eba5c486ea913af538348/jars/classes.jar
/Users/turkeybash/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value-annotations/1.6.5/c3dad10377f0e2242c9a4b88e9704eaf79103679/auto-value-annotations-1.6.5.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/a09c4480412e6a783b1406029f226252/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/8164bfc19e26dce5a908f4991a94ebee/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/543e95802eac363b870360d81de86f4d/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/b6463a160d15f09ed3851863047a8161/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/f941645e1371e163af9938b0062d97c8/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/260bf0740efdb40a73127a21798d1a14/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/IronSource/libs/unityadsadapter.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/IronSource/libs/facebookadapter.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/b45ee707e73a1949fa6a8da3f0cf70a4/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/4f41f9c5806083b563657fe85a4c830d/jetified-mediationsdk-6.13.0.1.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/37bc5e9ec46b9d6acaaf24702cb20df8/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/libs/org.jetbrains-annotations-16.0.1.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/c890ca71b9051456c04468dd219ff709/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/cb85d5c065c74e76671452c3f22f0cb2/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/6464e0de38ec16d025e0e7fff6e76550/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/99db557dece88ec352cb8ca138e6ed64/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/18c95ca1410323e2c5cd8cde1f196e93/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/faf7d3eaee1a578117434594c3eba86b/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/5081230f297cc37d85e34361bf683f2f/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/f5c020bf9567cf89dca167ff6265b80b/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/462a9cc6c8925e43a1226282105223aa/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/b57b2745c98102f954f6c0cd463f47a1/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/2790e9d931fe828f07667e1b0bc06ce1/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/IronSource/libs/admobadapter.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/IronSource/build/intermediates/intermediate-jars/release/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/SwrveSDKPushSupport/bin/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/34cbd0f86be48cda79884cdf97c39407/jetified-bolts-applinks-1.4.0.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/bdec4ab3f67bc66d2c21e2eb0a1a8c1d/jars/classes.jar
/Users/turkeybash/.gradle/caches/modules-2/files-2.1/com.parse.bolts/bolts-tasks/1.4.0/d85884acf6810a3bbbecb587f239005cbc846dc4/bolts-tasks-1.4.0.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/944e4b57ebabd144a5ce0dcea946bcc9/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/81959a3524ea9f279a63736759266114/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/d6264f0987e8e73b1738aa706b140338/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/32ac2f02a6857aa8c5aebf11d897e8c3/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/53396bac2c9f428b56a001c9490a05ed/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/IronSource/libs/android-bridge.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/39a26e542fff640961e0b813c7d42d1d/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/be965c0032f6bef33b826291dbbfdb5b/jars/classes.jar
/Users/turkeybash/.gradle/caches/modules-2/files-2.1/androidx.collection/collection/1.0.0/42858b26cafdaa69b6149f45dfc2894007bc2c7a/collection-1.0.0.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/f845891ce8b4288176a267ff8a08245d/jars/classes.jar
/Users/turkeybash/.gradle/caches/modules-2/files-2.1/com.google.dagger/dagger/2.27/7bee2792b10523e298a7de508635c6eed0c309be/dagger-2.27.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/607ef9c8e817a0307715c3b318067023/jetified-applovinadapter.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/IronSource/libs/adcolonyadapter.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/ec1f3eb9bae4e8df758580a267cdd167/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/build/intermediates/javac/release/compileReleaseJavaWithJavac/classes
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/libs/com.google.auto.value-auto-value-annotations-1.6.5.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/9d61fe2e60187d2e21fe7ff3edf8bd1d/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/29ef74aa71c0a4fe0ebd973f9c0fb428/jetified-okhttp-3.12.1.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/03a59754e24d260fc305c269c0a69c6d/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/f0e31f146365f5b25fb61cde670b0ed7/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/7b21b4096c47184bdfa164dc8d0ea009/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/bcad103e788ee6b97bf6b052ff358250/jetified-unibillandroid.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/123f8fbe9fbe6ee8cfbc30f9079ac8c0/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/692e68acbb964a6dc022a2eadc9a8f1c/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/c691abf570948f1954570fc7c4dcb272/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/3c2ec62dd8b34d91e1c732f54a7db2a3/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/SwrveSDKUnityBridge/bin/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/b40c7cdba61144dc341675e388f49dc5/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/025ccf4a3abbbcdc5823131f74407c0c/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/6ac1c6e2bc1447551e91d252d85e1baa/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/3e466b7af40b8c861d32d9eb25189058/jetified-okio-1.15.0.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/libs/in-app-purchasing-1.0.3.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/15c5ef78ac2873e0e55b2095be4d970e/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/7108fe7b41612e1952647bc170370efc/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/6081032f7ca98cafd13a9d83e65b8a6c/jars/classes.jar
/Users/turkeybash/Perforce/uname_dependencies/client/Builds/Game/android/Mimetime_gradle2.apk/Star Horse/SwrveSDKUnityBridge/build/intermediates/intermediate-jars/release/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/7aae005f11858b3b3782580528514877/jars/classes.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/e6545e5d81dfd31acd8b7dd2ae0902e7/jetified-AF-Android-SDK.jar
/Users/turkeybash/.gradle/caches/transforms-2/files-2.1/77c4e1f59d89a4925593cf5475857ae2/jars/classes.jar
/Users/turkeybash/.gradle/caches/modules-2/files-2.1/javax.inject/javax.inject/1/6975da39a7040257bd51d21a231b76c915872d38/javax.inject-1.jar

 
01:38:01.778 [INFO] [com.android.build.gradle.internal.transforms.D8MainDexListTransform] Library files: /Users/turkeybash/Library/Android/sdk/platforms/android-28/android.jar
/Users/turkeybash/Library/Android/sdk/platforms/android-28/optional/org.apache.http.legacy.jar
/Users/turkeybash/Library/Android/sdk/platforms/android-28/optional/android.test.mock.jar
/Users/turkeybash/Library/Android/sdk/platforms/android-28/optional/android.test.base.jar
/Users/turkeybash/Library/Android/sdk/platforms/android-28/optional/android.test.runner.jar

Using gradle's --stacktrace option yielded the following.

com.android.builder.multidex.D8MainDexList$MainDexListException: 
com.android.tools.r8.errors.CompilationError: 
Program type already present: com.google.auto.value.extension.memoized.Memoized
        at com.android.builder.multidex.D8MainDexList.generate(D8MainDexList.java:87)

So resolving it seems to be a matter of finding out which dependencies are using this program. I'm not sure what I'd do at that point, since these are all precompiled external libraries. I also have no idea how to find which particular dependencies have it.


Solution

  • I ended up solving this by writing a utility that traverses the contents of the .apk directory, finding .jar and .aar files, and unzipping them into subdirectories. This could be done manually, without a script, as well. You see references to these files in the log lines containing Mimetime_gradle2.apk/Star Horse/ in this project.

    After unzipping all of these, it was simply a matter of searching the parent directory recursively for references to the library called out in the output of the gradle command using --stacktrace. In this case it was com.google.auto.value.extension.memoized.Memoized

    Then it was simply a matter of removing one of the libraries that contained it from the Unity project. In this case it was an outdated copy of another library which was included with ExternalDependencyManager.

    The difficulty is that you need to export a gradle project from the Unity build menu, rather than build the APK directly. Then you need to copy the command for the specific version of gradle that your specific version of Unity uses, and run it with the --stacktrace option in order to get the output you need to identify the duplicate package. THEN you can unzip the resultant build directory (which takes the place of the APK file) and search it for the offending package.