Problem with running patrol tests on a remote device from DeviceFarm (STF) about patrol

[jBorkowska]

Hi @EskinaKseniya! First I need some details:

• Could you post whole command you use to run the app (without tests, just flutter run)
• Also, how flutter devices prints the remote device you want to use? If you want to pass device as an argument to a patrol command, you need to use the name that appears in flutter devices output (eg. emulator-5554 for local Android emulator)

[EskinaKseniya]

Hi @jBorkowska! Thanks for the quick response, here's what you asked for:

whole command flutter run:
fvm flutter run -d 192.168.**.***:7405 --flavor stagegoogle --dart-define ....

output of the flutter devices:
Found 4 connected devices:
sdk gphone64 arm64 (mobile) • 192.168..*:7405 • android-arm64 • Android 14 (API
34) (emulator)
sdk gphone64 x86 64 (mobile) • emulator-5554 • android-x64 • Android 14 (API
34) (emulator)
Linux (desktop) • linux • linux-x64 • Ubuntu 22.04.4
LTS 5.15.0-113-generic
Chrome (web) • chrome • web-javascript • Google Chrome
125.0.6422.141

Run "flutter emulators" to list and start any available device emulators.

[jBorkowska]

Ok, then I have one more question - does patrol test work on the remote device?

[EskinaKseniya]

patrol test doesn't work too.
I use patrol test --flavor stagegoogle --dart-define ..... --target integration_test/it10*** -d 192.168.**.***:7401 --verbose
and get the output:

        Jul 16, 2024 4:25:45 PM com.google.testing.platform.RunnerImpl run
        SEVERE: Test execution failed with fatal error!
        com.google.testing.platform.api.plugin.PluginException: Exception thrown during onBeforeAll invocation of plugin com.google.testing.platform.plugin.android.AndroidDevicePlugin.
                at com.google.testing.platform.plugin.PluginLifecycleKt.invokeOrThrow(PluginLifecycle.kt:216)
                at com.google.testing.platform.plugin.PluginLifecycleKt.invokeOrThrow$default(PluginLifecycle.kt:205)
                at com.google.testing.platform.plugin.PluginLifecycle$onBeforeAll$1.invoke(PluginLifecycle.kt:92)
                at com.google.testing.platform.plugin.PluginLifecycle$onBeforeAll$1.invoke(PluginLifecycle.kt:88)
                at com.google.testing.platform.core.telemetry.common.noop.NoopDiagnosticsScope.recordEvent(NoopDiagnosticsScope.kt:35)
                at com.google.testing.platform.core.telemetry.TelemetryKt.recordEvent(Telemetry.kt:105)
                at com.google.testing.platform.core.telemetry.TelemetryKt.recordEvent$default(Telemetry.kt:98)
                at com.google.testing.platform.plugin.PluginLifecycle.onBeforeAll(PluginLifecycle.kt:88)
                at com.google.testing.platform.executor.SingleDeviceExecutor$execute$4.invoke(SingleDeviceExecutor.kt:86)
                at com.google.testing.platform.executor.SingleDeviceExecutor$execute$4.invoke(SingleDeviceExecutor.kt:86)
                at com.google.testing.platform.executor.SingleDeviceExecutor.runUnlessCancelled(SingleDeviceExecutor.kt:105)
                at com.google.testing.platform.executor.SingleDeviceExecutor.execute(SingleDeviceExecutor.kt:86)
                at com.google.testing.platform.RunnerImpl.run(RunnerImpl.kt:108)
                at com.google.testing.platform.server.strategy.NonInteractiveServerStrategy$run$4.invoke(NonInteractiveServerStrategy.kt:80)
                at com.google.testing.platform.server.strategy.NonInteractiveServerStrategy$run$4.invoke(NonInteractiveServerStrategy.kt:79)
                at com.google.testing.platform.core.telemetry.common.noop.NoopDiagnosticsScope.recordEvent(NoopDiagnosticsScope.kt:35)
                at com.google.testing.platform.core.telemetry.TelemetryKt.recordEvent(Telemetry.kt:66)
                at com.google.testing.platform.server.strategy.NonInteractiveServerStrategy.run(NonInteractiveServerStrategy.kt:79)
                at com.google.testing.platform.main.MainKt$main$4.invokeSuspend(Main.kt:67)
                at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
                at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
                at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
                at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
                at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
                at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
                at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
                at com.google.testing.platform.main.MainKt.main(Main.kt:66)
                at com.google.testing.platform.main.MainKt.main$default(Main.kt:34)
                at com.google.testing.platform.main.MainKt.main(Main.kt)
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
                at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.base/java.lang.reflect.Method.invoke(Method.java:568)
                at com.google.testing.platform.launcher.Launcher.main(Launcher.java:149)
        Caused by: kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 120000 ms
                at kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:158)
                at kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:128)
                at kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:497)
                at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
                at kotlinx.coroutines.DefaultExecutor.run(DefaultExecutor.kt:68)
                at java.base/java.lang.Thread.run(Thread.java:840)

        Jul 16, 2024 4:25:45 PM com.android.tools.utp.plugins.host.additionaltestoutput.AndroidAdditionalTestOutputPlugin deviceShellAndCheckSuccess
        WARNING: Shell command failed (1): ls "/sdcard/Android/media/********/additional_test_output"
        ls: /sdcard/Android/media/********/additional_test_output: No such file or directory

        Jul 16, 2024 4:25:45 PM com.android.tools.utp.plugins.host.additionaltestoutput.AndroidAdditionalTestOutputPlugin afterAll
        WARNING: Failed to retrieve additional test outputs from device.
        com.android.ddmlib.SyncException: Remote object doesn't exist!
                at com.android.ddmlib.SyncService.pullFile(SyncService.java:341)
                at com.android.ddmlib.internal.DeviceImpl.pullFile(DeviceImpl.java:1083)
                at com.android.tools.utp.plugins.deviceprovider.ddmlib.DdmlibAndroidDevice.pullFile(DdmlibAndroidDevice.kt)
                at com.android.tools.utp.plugins.deviceprovider.ddmlib.DdmlibAndroidDeviceController.pull(DdmlibAndroidDeviceController.kt:254)
                at com.google.testing.platform.executor.DeviceControllerProxy$pull$2.invoke(DeviceControllerProxy.kt:82)
                at com.google.testing.platform.executor.DeviceControllerProxy$pull$2.invoke(DeviceControllerProxy.kt:82)
                at com.google.testing.platform.core.telemetry.common.noop.NoopDiagnosticsScope.recordEvent(NoopDiagnosticsScope.kt:35)
                at com.google.testing.platform.core.telemetry.TelemetryKt.recordEvent(Telemetry.kt:66)
                at com.google.testing.platform.executor.DeviceControllerProxy.pull(DeviceControllerProxy.kt:79)
                at com.android.tools.utp.plugins.host.additionaltestoutput.AndroidAdditionalTestOutputPlugin.copyAdditionalTestOutputsFromDeviceToHost(AndroidAdditionalTestOutputPlugin.kt:222)
                at com.android.tools.utp.plugins.host.additionaltestoutput.AndroidAdditionalTestOutputPlugin.afterAll(AndroidAdditionalTestOutputPlugin.kt:206)
                at com.google.testing.platform.plugin.PluginLifecycle$onAfterAll$1$2$1.invoke(PluginLifecycle.kt:180)
                at com.google.testing.platform.plugin.PluginLifecycle$onAfterAll$1$2$1.invoke(PluginLifecycle.kt:179)
                at com.google.testing.platform.plugin.PluginLifecycleKt$invokeOrThrow$1.invoke(PluginLifecycle.kt:213)
                at com.google.testing.platform.plugin.PluginLifecycleKt$invokeOrThrow$1.invoke(PluginLifecycle.kt:212)
                at com.google.testing.platform.core.telemetry.common.noop.NoopDiagnosticsScope.recordEvent(NoopDiagnosticsScope.kt:35)
                at com.google.testing.platform.core.telemetry.TelemetryKt.recordEvent(Telemetry.kt:105)
                at com.google.testing.platform.core.telemetry.TelemetryKt.recordEvent$default(Telemetry.kt:98)
                at com.google.testing.platform.plugin.PluginLifecycleKt.invokeOrThrow(PluginLifecycle.kt:212)
                at com.google.testing.platform.plugin.PluginLifecycleKt.invokeOrThrow$default(PluginLifecycle.kt:205)
                at com.google.testing.platform.plugin.PluginLifecycle$onAfterAll$1.invoke(PluginLifecycle.kt:179)
                at com.google.testing.platform.plugin.PluginLifecycle$onAfterAll$1.invoke(PluginLifecycle.kt:173)
                at com.google.testing.platform.core.telemetry.common.noop.NoopDiagnosticsScope.recordEvent(NoopDiagnosticsScope.kt:35)
                at com.google.testing.platform.core.telemetry.TelemetryKt.recordEvent(Telemetry.kt:105)
                at com.google.testing.platform.core.telemetry.TelemetryKt.recordEvent$default(Telemetry.kt:98)
                at com.google.testing.platform.plugin.PluginLifecycle.onAfterAll(PluginLifecycle.kt:173)
                at com.google.testing.platform.executor.SingleDeviceExecutor$execute$3.invoke(SingleDeviceExecutor.kt:83)
                at com.google.testing.platform.executor.SingleDeviceExecutor$execute$3.invoke(SingleDeviceExecutor.kt:82)
                at com.google.testing.platform.result.TestResultListenerManager.afterTestSuite(TestResultListenerManager.kt:140)
                at com.google.testing.platform.RunnerImpl.run(RunnerImpl.kt:118)
                at com.google.testing.platform.server.strategy.NonInteractiveServerStrategy$run$4.invoke(NonInteractiveServerStrategy.kt:80)
                at com.google.testing.platform.server.strategy.NonInteractiveServerStrategy$run$4.invoke(NonInteractiveServerStrategy.kt:79)
                at com.google.testing.platform.core.telemetry.common.noop.NoopDiagnosticsScope.recordEvent(NoopDiagnosticsScope.kt:35)
                at com.google.testing.platform.core.telemetry.TelemetryKt.recordEvent(Telemetry.kt:66)
                at com.google.testing.platform.server.strategy.NonInteractiveServerStrategy.run(NonInteractiveServerStrategy.kt:79)
                at com.google.testing.platform.main.MainKt$main$4.invokeSuspend(Main.kt:67)
                at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
                at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
                at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
                at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
                at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
                at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
                at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
                at com.google.testing.platform.main.MainKt.main(Main.kt:66)
                at com.google.testing.platform.main.MainKt.main$default(Main.kt:34)
                at com.google.testing.platform.main.MainKt.main(Main.kt)
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
                at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.base/java.lang.reflect.Method.invoke(Method.java:568)
                at com.google.testing.platform.launcher.Launcher.main(Launcher.java:149)


        Platform error occurred when running the UTP test suite
        : 
        : Test results saved as file:/home/user/AndroidStudioProjects/********/build/app/outputs/androidTest-results/connected/flavors/stageGoogle/sdk_gphone64_arm64%20-%2014/test-result.pb. Inspect these results in Android Studio by selecting Run > Import Tests From File from the menu bar and importing test-result.pb.
        : 
        : > Task :app:connectedStageGoogleDebugAndroidTest FAILED
        : 
        : Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
        : 
        : You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
        : 
        : See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
        : 720 actionable tasks: 6 executed, 714 up-to-date

        FAILURE: Build failed with an exception.

        * What went wrong:
        Execution failed for task ':app:connectedStageGoogleDebugAndroidTest'.
        See the report at: file:///home/user/AndroidStudioProjects/********/build/app/reports/androidTests/connected/flavors/stageGoogle/index.html

        * Try:
        > Run with --stacktrace option to get the stack trace.
        > Run with --info or --debug option to get more log output.
        > Run with --scan to get full insights.

        * Get more help at https://help.gradle.org

        BUILD FAILED in 2m 19s
✗ Failed to execute tests of apk with entrypoint test_bundle.dart on 192.168.**.***:7401 (Gradle test execution failed with code 1) (140.1s)
Exception: Gradle test execution failed with code 1
#0      AndroidTestBackend.execute.<anonymous closure> (package:patrol_cli/src/android/android_test_backend.dart:140:9)
<asynchronous suspension>
#1      DisposeScope.run (package:dispose_scope/src/dispose_scope.dart:46:7)
<asynchronous suspension>
#2      AndroidTestBackend.execute (package:patrol_cli/src/android/android_test_backend.dart:106:5)
<asynchronous suspension>
#3      TestCommand._execute (package:patrol_cli/src/commands/test.dart:323:7)
<asynchronous suspension>
#4      TestCommand.run (package:patrol_cli/src/commands/test.dart:215:23)
<asynchronous suspension>
#5      CommandRunner.runCommand (package:args/command_runner.dart:212:13)
<asynchronous suspension>
#6      PatrolCommandRunner.runCommand (package:patrol_cli/src/runner/patrol_command_runner.dart:347:18)
<asynchronous suspension>
#7      PatrolCommandRunner.run (package:patrol_cli/src/runner/patrol_command_runner.dart:291:18)
<asynchronous suspension>
#8      patrolCommandRunner (package:patrol_cli/src/runner/patrol_command_runner.dart:70:20)
<asynchronous suspension>
#9      main (file:///home/user/.pub-cache/hosted/pub.dev/patrol_cli-2.7.0/bin/main.dart:6:20)
<asynchronous suspension>

[jBorkowska]

Ok, thank you for this info.
I don't have any idea right now on how to fix it, we need more time to check out this device farm (we haven't used it yet). We'll post updates here if we figure out how to make it work

[piotruela]

Hi @EskinaKseniya. It's hard to say based on the logs what exactly is causing the issue. I found this issue which looks alike the one you have. Is there a maybe this dialog on the screen that blocks the execution?

Please send as much logs and artifacts as possible (screen recording, other logs, execution commands you use). Maybe it will give some more information.

[EskinaKseniya]

Hi, @piotruela. A dialog box does not appear on the emulator during the build. Here is a video recording, but literally nothing happens there for 3 minutes.
device logs:
device_logs.txt

what other logs do I need to provide?

[piotruela]

I looked into the device logs but still I couldn't find anything that causes the issue. I suggest trying those things:

• (if possible) Increase timeout duration. Maybe 2 minutes is not enough to boot the device, connect to it and install the app,
• (if possible) Run patrol tests on the same device locally (without STF).
  To do some further investigation, we would need to have a setup which we could try on our own.

[EskinaKseniya]

@piotruela, thank you for the response
Could you tell me how to increase the duration of the timeout?

[piotruela]

Timeout duration is determined by STF so you should look inside it's docs.