flutterflutter-patrol

patrol does not run the tests when using --release flag


I have patrol running fine when performing the tests on debug:

# this comes from a makefile command I wrote: 
patrol test --target integration_test/$(test_path)  --dart-define="MODE=test" --dart-define="ENV=local" --dart-define="SUPABASE_ADMIN_TEST_PASSWORD=$(SUPABASE_ADMIN_LOCAL_TEST_PASSWORD)" --verbose -d emulator-5554

which ends successfully:

...
        : BUILD SUCCESSFUL in 7s
        : 327 actionable tasks: 6 executed, 321 up-to-date
✓ Completed executing apk with entrypoint test_bundle.dart on emulator-5554 (8.1s)

But when performing it by adding the --release flag, gradle complains:

patrol test --target integration_test/$(test_path)  --dart-define="MODE=test" --dart-define="ENV=local" --dart-define="SUPABASE_ADMIN_TEST_PASSWORD=$(SUPABASE_ADMIN_LOCAL_TEST_PASSWORD)" --verbose -d emulator-5554

...

        FAILURE: Build failed with an exception.
        
        * What went wrong:
        Cannot locate tasks that match ':app:assembleReleaseAndroidTest' as task 'assembleReleaseAndroidTest' not found in project ':app'.
        
        * Try:
        > Run gradlew tasks to get a list of available tasks.
        > Run with --stacktrace option to get the stack trace.
        > Run with --info or --debug option to get more log output.
        > Run with --scan to get full insights.
        
        * Get more help at https://help.gradle.org

Here is my app/build.gradle

plugins {
    id "com.android.application"
    id "kotlin-android"
    id "dev.flutter.flutter-gradle-plugin"
}

def localProperties = new Properties()
def localPropertiesFile = rootProject.file('local.properties')
if (localPropertiesFile.exists()) {
    localPropertiesFile.withReader('UTF-8') { reader ->
        localProperties.load(reader)
    }
}

def flutterVersionCode = localProperties.getProperty('flutter.versionCode')
if (flutterVersionCode == null) {
    flutterVersionCode = '1'
}

def flutterVersionName = localProperties.getProperty('flutter.versionName')
if (flutterVersionName == null) {
    flutterVersionName = '1.0'
}

def keystoreProperties = new Properties()
def keystorePropertiesFile = rootProject.file('key.properties')
if (keystorePropertiesFile.exists()) {
    keystoreProperties.load(new FileInputStream(keystorePropertiesFile))
}

android {
    namespace "net.xxx.xxx.xxx"
    compileSdk flutter.compileSdkVersion
    ndkVersion flutter.ndkVersion

    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }

    kotlinOptions {
        jvmTarget = '1.8'
    }

    sourceSets {
        main.java.srcDirs += 'src/main/kotlin'
    }

    defaultConfig {
        // TODO: Specify your own unique Application ID (https://developer.android.com/studio/build/application-id.html).
        applicationId "net.xxx.xxxx.xx"
        // You can update the following values to match your application needs.
        // For more information, see: https://docs.flutter.dev/deployment/android#reviewing-the-gradle-build-configuration.
        minSdkVersion 21
        multiDexEnabled true
        targetSdkVersion flutter.targetSdkVersion
        versionCode flutterVersionCode.toInteger()
        versionName flutterVersionName
        testInstrumentationRunner "pl.leancode.patrol.PatrolJUnitRunner"
        testInstrumentationRunnerArguments clearPackageData: "true"
    }

    testOptions {
        execution "ANDROIDX_TEST_ORCHESTRATOR"
    }

    signingConfigs {
        release {
            keyAlias keystoreProperties['keyAlias']
            keyPassword keystoreProperties['keyPassword']
            storeFile keystoreProperties['storeFile'] ? file(keystoreProperties['storeFile']) : null
            storePassword keystoreProperties['storePassword']
        }
    }
    buildTypes {
        release {
            signingConfig signingConfigs.release
        }
    }
}

flutter {
    source '../..'
}

dependencies {
    androidTestUtil "androidx.test:orchestrator:1.4.2"
    implementation 'com.android.support:multidex:1.0.3'
    coreLibraryDesugaring 'com.android.tools:desugar_jdk_libs:1.1.5'
}

and my pubspec:

name: xxxx_zzzz
description: xxxx xxxx
# The following line prevents the package from being accidentally published to
# pub.dev using `flutter pub publish`. This is preferred for private packages.
publish_to: 'none' # Remove this line if you wish to publish to pub.dev

# The following defines the version and build number for your application.
# A version number is three numbers separated by dots, like 1.2.43
# followed by an optional build number separated by a +.
# Both the version and the builder number may be overridden in flutter
# build by specifying --build-name and --build-number, respectively.
# In Android, build-name is used as versionName while build-number used as versionCode.
# Read more about Android versioning at https://developer.android.com/studio/publish/versioning
# In iOS, build-name is used as CFBundleShortVersionString while build-number is used as CFBundleVersion.
# Read more about iOS versioning at
# https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/CoreFoundationKeys.html
# In Windows, build-name is used as the major, minor, and patch parts
# of the product and file versions while build-number is used as the build suffix.
version: 1.1.0+01

environment:
  sdk: '>=3.1.0 <4.0.0'

# Dependencies specify other packages that your package needs in order to work.
# To automatically upgrade your package dependencies to the latest versions
# consider running `flutter pub upgrade --major-versions`. Alternatively,
# dependencies can be manually updated by changing the version numbers below to
# the latest version available on pub.dev. To see which dependencies have newer
# versions available, run `flutter pub outdated`.
dependencies:
  flutter:
    sdk: flutter

  # The following adds the Cupertino Icons font to your application.
  # Use with the CupertinoIcons class for iOS style icons.
  cupertino_icons: ^1.0.2
  supabase_flutter: ^1.7.0
  flutter_dotenv: ^5.0.2
  table_calendar: ^3.0.0
  syncfusion_flutter_pdfviewer: ^22.2.12
  flutter_launcher_icons: ^0.13.1
  firebase_core: ^2.10.0
  firebase_messaging: ^14.4.1
  mocktail: ^1.0.0
  week_of_year: ^2.0.0
  supabase_auth_ui: ^0.2.1
  dart_dotenv: ^1.0.1
  firebase_crashlytics: ^3.3.5
  event_bus: ^2.0.0
  url_launcher: ^6.1.14
  package_info: ^2.0.2
  flutter_local_notifications: ^15.1.1
  jwt_decoder: ^2.0.1
  pub_semver: ^2.1.4
  flutter_markdown: ^0.6.18+2
  timezone: ^0.9.2
  infinite_scroll_pagination: ^4.0.0
  intl: ^0.18.1

dev_dependencies:
  patrol: ^3.6.1
  flutter_test:
    sdk: flutter
  integration_test:
    sdk: flutter
  flutter_lints: ^2.0.0

flutter:
  uses-material-design: true

  # To add assets to your application, add an assets section, like this:
  assets:
    - assets/.local.env
    - assets/.test.env
    - assets/.prod.env
    - assets/.staging.env
flutter_launcher_icons:
  android: true
  ios: true
  image_path_android: "assets/images/logo_android.png"
  adaptive_icon_background: "#ffffff"
  adaptive_icon_foreground: "assets/images/logo_android.png"
  image_path_ios: "assets/images/logo_ios.png"

patrol:
  app_name: XXX xxxx
  android:
    package_name: net.xxx.xxxx
  ios:
    bundle_id: net.xxx.xxx

and some doctor analysis:

$ flutter doctor                                                                   130 
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.19.5, on macOS 14.0 23A344 darwin-arm64, locale en-ES)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.3)
[✓] IntelliJ IDEA Ultimate Edition (version 2023.3.6)
[✓] Connected device (4 available)
[✓] Network resources

• No issues found!

$ patrol doctor                                                                    130 
Patrol doctor:
Patrol CLI version: 2.7.0
Flutter command: flutter 
  Flutter 3.19.5 • channel stable
Android: 
• Program adb found in /Users/zzz/Library/Android/sdk/platform-tools/adb
• Env var $ANDROID_HOME set to /Users/zzz/Library/Android/sdk
iOS / macOS: 
• Program xcodebuild found in /usr/bin/xcodebuild
• Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller

Solution

  • You run Patrol UI tests in release mode on Android, you need to modify your app/build.gradle to include testBuildType "release" line, like so (link):

    android {
        // ...
        buildTypes {
          release {
            // ...
          }
        }
    
        testBuildType "release"
    }
    

    Please bear in mind that Patrol has problems with release mode when ProGuard is enabled. Learn more in this issue