flutterdartintegration-testingflutter-integration-testflutter-patrol

Unable to run Patrol test properly

i'm able to run integration_test using this codes:

##########
# integration_test/openapp_test.dart
##########

import 'package:myapp/main.dart' as app;

group('end-to-end test', () {
  testWidgets('tap on the floating action button, verify counter',
          (tester) async {
        app.main(isTesting: true);
        await tester.pumpAndSettle(
          const Duration(seconds: 10),
        );

        // first screen
        expect(find.text('Mulai baru'), findsOneWidget);

        // I go to 'Login from Onboarding' page
        await tester.tap(find.byKey(
          const Key(OnboardingKey.buttonMasukKeAkun),
        ));
  });
});

##########
# test_driver/integration_test.dart
##########

// @dart=2.9
import 'package:integration_test/integration_test_driver.dart';

Future<void> main() => integrationDriver();

my main.dart looks something like this:

##########
# lib/main.dart
##########

Future<void> main({bool isTesting = false}) async {
  WidgetsFlutterBinding.ensureInitialized();
  await SystemChrome.setPreferredOrientations(<DeviceOrientation>[
    DeviceOrientation.portraitUp,
    DeviceOrientation.portraitDown,
  ]);
  await Firebase.initializeApp();

  _initCrashlytics();


  await SharedPreference.wipeSecureStorage();

  runZonedGuarded<Future<void>>(
    () async {
      runApp(App(isTesting: isTesting));
    },
    (Object object, StackTrace stackTrace) {
      ...
      ...
      ,,,
    },
  );
}

class App extends StatelessWidget {
  const App({Key? key, this.isTesting = false}) : super(key: key);
  final bool isTesting;

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    ...
    ...
    ...
  }
}

command i used to run test above (runs on android device):

flutter drive --driver=test_driver/integration_test.dart --target=integration_test/openapp_test_copy.dart --flavor=staging

it works fine when i run above command.. the script installs the apk into my device, run test, and showing passed

when i try to change it using patrolTest to something like this

##########
# integration_test/openapp_test.dart
##########

void main() {
  group('end-to-end test', () {
    patrolTest('tap on the floating action button, verify counter',
        (PatrolTester $) async {
      // app.main(isTesting: true);
      await $.pumpWidgetAndSettle(
        const App(
          isTesting: true,
        ),
        duration: const Duration(seconds: 20),
      );
      await $.pump(const Duration(seconds: 50));
      expect(find.text('Mulai baru'), findsNothing);
    });
  });
}

or this (i tried 2 different ways to start the app because i don't know which one is right

##########
# integration_test/openapp_test.dart
##########

void main() {
  group('end-to-end test', () {
    patrolTest('tap on the floating action button, verify counter',
        (PatrolTester $) async {
        await $.pumpWidget(
          const App(
            isTesting: true,
          ),
        );
        await $.pumpAndSettle(
          duration: const Duration(seconds: 20),
        );

        expect(find.text('Mulai baru'), findsNothing);
    });
  });
}

and try to run it using this command

patrol test --flavor staging

it always showing test passed.. but i don't see the test running.. it doesn't install the app, and directly states that the test passed
it should show failed because this command expect(find.text('Mulai baru'), findsNothing); should be failed

FileSystemException: Cannot open file, path = '/Users/thekucays/.zshrc' (OS Error: Permission denied, errno = 13)
No device specified, using the first one (P0AA003810060900077)
Every test target will be run 1 time(s)
• Building apk with entrypoint openapp_test.dart...
✓ Completed building apk with entrypoint openapp_test.dart (22.3s)
• Executing tests of apk with entrypoint openapp_test.dart on device P0AA003810060900077...
✓ Completed executing apk with entrypoint openapp_test.dart on device P0AA003810060900077 (12.2s)
 PASS  openapp_test.dart on P0AA003810060900077

what went wrong here? do i missed something?

flutter doctor -v output

[!] Flutter (Channel stable, 3.7.1, on macOS 12.6.1 21G217 darwin-arm64, locale en-ID)
    • Flutter version 3.7.1 on channel stable at /Users/thekucays/fvm/versions/3.7.1
    ! Warning: `dart` on your path resolves to /opt/homebrew/Cellar/dart/2.18.4/libexec/bin/dart, which is not inside your current Flutter SDK checkout at
      /Users/thekucays/fvm/versions/3.7.1. Consider adding /Users/thekucays/fvm/versions/3.7.1/bin to the front of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 7048ed95a5 (9 weeks ago), 2023-02-01 09:07:31 -0800
    • Engine revision 800594f1f4
    • Dart version 2.19.1
    • DevTools version 2.20.1
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/thekucays/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0
    • ANDROID_HOME = /Users/thekucays/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13A1030d
    • CocoaPods version 1.12.0

[✗] Chrome - develop for the web (Cannot find Chrome executable at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[✓] Android Studio (version 2021.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)

[✓] VS Code (version 1.73.1)
    • VS Code at /Users/thekucays/Documents/Installers/Visual Studio Code.app/Contents
    • Flutter extension version 3.60.0

[✓] Connected device (2 available)
    • Nokia 5 3 (mobile) • P0AA003810060900077 • android-arm64 • Android 11 (API 30)
    • macOS (desktop)    • macos               • darwin-arm64  • macOS 12.6.1 21G217 darwin-arm64

[✓] HTTP Host Availability
    • All required HTTP hosts are available

patrol doctor output

Patrol CLI version: 1.1.3
Program adb found in /Users/thekucays/Library/Android/sdk/platform-tools/adb
Env var $ANDROID_HOME set to /Users/thekucays/Library/Android/sdk
Program xcodebuild found in /usr/bin/xcodebuild
Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller
Program ios-deploy found in /opt/homebrew/bin/ios-deploy

patrol -V output

patrol_cli v1.1.3
Solution

  • looks like this is my own bad lol
    i finally managed it to run using these configurations

    package my.package.name

import org.junit.Rule;
import org.junit.runner.RunWith;
import pl.leancode.patrol.PatrolTestRule;
import pl.leancode.patrol.PatrolTestRunner;

@RunWith(PatrolTestRunner.class)
public class MainActivityTest {
    @Rule
    public PatrolTestRule<MainActivity> rule = new PatrolTestRule<>(MainActivity.class);
}

    patrol:
  app_name: Awesome App
  android:
    package_name: my.package.name

    testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"

     testImplementation "junit:junit:4.13.2"

    and run it using this command

     patrol test --target integration_test/openapp_test.dart

    however, i cannot use

    $.pumpWidgetAndSettle(foo())

    to start the app.. i use WidgetTester's default command like this inside openapp_test.dart

    patrolTest(
    'counter state is the same after going to home and switching apps',
    nativeAutomation: true,
    (PatrolTester $) async {
      app.main();

      await $.tester.pumpAndSettle(
        const Duration(seconds: 10),
      );
    },
  );

    looks like there is slight difference in behavior between $.pumpWidget() and $.pumpWidgetAndSettle() (?)