pedrovgs / shot

Screenshot testing library for Android

Shot

🔝 Top sponsors 🔝

Shot is an Android project you can use to write screenshot for your apps in a simple and friendly way.

What is this?

Shot is a Gradle plugin and a core android library thought to run screenshot tests for Android. This project provides a handy interface named ScreenshotTest and a ready to use ShotTestRunner you can use in order to write tests like these:

class GreetingScreenshotTest : ScreenshotTest {

    // If you are using regular Android views

    @Test
    fun theActivityIsShownProperly() {
        val mainActivity = startMainActivity()

        compareScreenshot(activity)
    }

    // If you are using Jetpack Compose

    @Test
    fun rendersGreetingMessageForTheSpecifiedPerson() {
        composeRule.setContent { Greeting(greeting) }

        compareScreenshot(composeRule)
    }
}

Since Shot 5.0.0 we provide screenshot testing support for Jetpack Compose. If you are testing your components using Shot we strongly recommend you to configure your emulator using the gpu mode swiftshader_indirect. This will help you to avoid rendering issues when verifying your screenshots from any CI environment.

Record your screenshots executing ./gradlew executeScreenshotTests -Precord

And verify your tests executing ./gradlew executeScreenshotTests

If Shot finds any error in your tests execution the Gradle plugin will show a report as follows:

You can find the complete Facebook SDK documentation here.

Getting started

Setup the Gradle plugin:

Modify your root build.gradle file:

  buildscript {
    // ...
    dependencies {
      // ...
      classpath 'com.karumi:shot:<LATEST_RELEASE>'
    }
  }

Apply the plugin from any of the modules configured in your project. In a simple Android app your app/build.gradle file:

apply plugin: 'shot'

You will also have to configure the instrumentation test runner in your module build.gradle file as follows:

android {
    // ...
    defaultConfig {
        // ...
        testInstrumentationRunner "com.karumi.shot.ShotTestRunner"
    }
    // ...

We created this test runner for you extending from the Android one.

This plugin sets up a few convenience commands you can list executing ./gradlew tasks and reviewing the Shot associated tasks:

If you are using flavors the available Shot gradle tasks will be configured based on your flavors and build types configuration. You can find all the available shot tasks by executing ./gradlew tasks. For example, if your app has two flavors: green and blue the list of available Shot tasks will be:

executeScreenshotTests - Checks the user interface screenshot tests. If you execute this task using -Precord param the screenshot will be regenerated.
blueDebugDownloadScreenshots - Retrieves the screenshots stored into the Android device where the tests were executed for the build BlueDebug
blueDebugExecuteScreenshotTests - Records the user interface tests screenshots. If you execute this task using -Precord param the screenshot will be regenerated for the build BlueDebug
blueDebugRemoveScreenshotsBefore - Removes the screenshots recorded before the tests execution from the Android device where the tests were executed for the build BlueDebug
blueDebugRemoveScreenshotsAfter - Removes the screenshots recorded after the tests execution from the Android device where the tests were executed for the build BlueDebug
greenDebugDownloadScreenshots - Retrieves the screenshots stored into the Android device where the tests were executed for the build GreenDebug
greenDebugExecuteScreenshotTests - Records the user interface tests screenshots. If you execute this task using -Precord param the screenshot will be regenerated for the build GreenDebug
greenDebugRemoveScreenshotsBefore - Removes the screenshots recorded before the tests execution from the Android device where the tests were executed for the build GreenDebug
greenDebugRemoveScreenshotsAfter - Removes the screenshots recorded after the tests execution from the Android device where the tests were executed for the build GreenDebug

You can specify a directory suffix where screenshot & report will be saved ({FLAVOR}/{BUILD_TYPE}/{DIRECTORY_SUFFIX}) :
./gradlew executeScreenshotTests -PdirectorySuffix=Api26

If for some reason you are running your tests on a different machine and you want to skip the instrumentation tests execution and just compare the sources remember you can use the following shot configuration:

  shot {
    runInstrumentation = false
  }

Create this AndroidManifest.xml file inside your androidTest folder.

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="<YOUR_APP_ID>.test"
    android:sharedUserId="<YOUR_APP_ID>.uid">

</manifest>

You'll have to add the same android:sharedUserId="<YOUR_APP_ID>.uid" configuration to your app/AndroidManfiest.xml file in order to let the testing APK write into the SDCard.. If you don't do this, you can end up facing a weird error with this message while running your tests:

java.lang.RuntimeException: Failed to create the directory /sdcard/screenshots/com.example.snapshottesting.test/screenshots-default for screenshots. Is your sdcard directory read-only?

If you want to use Shot to test your Android libraries code, you will have to configure the testApplicationId parameter as follows

android {
    // ...
    defaultConfig {
        // ...
        testApplicationId "<MY_TEST_APPLICATION_ID>"
    }
    // ...

If you are using AGP 7.X in your Android library you may need to configure the applicationId as part of the Shot extension configuration as follows:

shot {
  applicationId = "com.myapp"
}

AGP introduced a bug in 7.0.1-alpha4 we are not able to fix without this workaround for now. In the future, once AGP fixes the bug, we will remove this extension property.

Be careful and do not use the same id you are using for any other of the installed apps.

Now you are ready to use the ScreenshotTest interface from your tests:

class MyActivityTest: ScreenshotTest {
    @Test
    fun theActivityIsShownProperly() {
            val mainActivity = startMainActivity();
           /*
             * Take the actual screenshot. At the end of this call, the screenshot
             * is stored on the device and the gradle plugin takes care of
             * pulling it and displaying it to you in nice ways.
             */
            compareScreenshot(activity);
    }
}

Since Shot 5.0.0, if you are using Jetpack Compose your tests will look like this:

class GreetingScreenshotTest : ScreenshotTest {

    @Test
    fun rendersGreetingMessageForTheSpecifiedPerson() {
        composeRule.setContent { Greeting(greeting) }

        compareScreenshot(composeRule)
    }

This interface is full of useful methods you can use to take your screenshots with your activities, dialogs fragments, view holders or even custom views

You can find a complete example in this repository under the folder named shot-consumer or review this kata.***

Now you are ready to record and verify your screenshot tests!

Using shot on API 28+

If you want to use Shot on devices running API >= 28, you will have to enable access to non-SDK interfaces.
Execute the following commands:

For Android 9 (API level 28)

adb shell settings put global hidden_api_policy_pre_p_apps 1
adb shell settings put global hidden_api_policy_p_apps 1

For Android 10 (API level 29) or higher

adb shell settings put global hidden_api_policy 1

Recording tests

You can record your screenshot tests executing this command:

./gradlew <Flavor><BuildType>ExecuteScreenshotTests -Precord

or

./gradlew executeScreenshotTests -Precord

This will execute all your integration tests and it will pull all the generated screenshots into your repository so you can easily add them to the version control system.

Executing tests

Once you have a bunch of screenshot tests recorded you can easily verify if the behaviour of your app is the correct one executing this command:

./gradlew <Flavor><BuildType>ExecuteScreenshotTests

or

./gradlew executeScreenshotTests

After executing your screenshot tests using the Gradle task <Flavor><BuildType>ExecuteScreenshotTests a report with all your screenshots will be generated.

ScreenshotTest interface

ScreenshotTest interface has been designed to simplify the usage of the library. These are the features you can use:

• Take a screenshot of any activity by using compareScreenshot(activity). Activity height, width, background color and screenshot name are configurable.
• Take a screenshot of any fragment by using compareScreenshot(fragment). Fragment height, width and screenshot name are configurable.
• Take a screenshot of any dialog by using compareScreenshot(dialog). Dialog height, width and screenshot name are configurable.
• Take a screenshot of any view by using compareScreenshot(view). View height, width and screenshot name are configurable.
• Take a screenshot of any view holder by using compareScreenshot(holder). View holder height, width and screenshot name are configurable.
• Take a screenshot of any Jetpack Component compareScreenshot(composeTestRule). The screenshot name is configurable.
• Take a screenshot of any SemanticInteractionNode compareScreenshot(node). The screenshot name is configurable.

Before taking the screenshot, Shot performs some tasks in order to stabilize the screenshot. You can find the detail about the tasks performed in ScreenshotTest#disableFlakyComponentsAndWaitForIdle:

• Invokes disableFlakyComponentsAndWaitForIdle method you can override if you want to add any custom task before taking the screenshot.
• Hides every EditText cursor.
• Hides every ScrollView and HorizontalScrollView scrollbars.
• Hides all the ignored views. You can specify the views you want to ignore by overriding viewToIgnore.
• Wait for animations to finish and also waits until Espresso considers the UI thread is idle. This is really interesting if you are using Espresso in your tests.

You can find examples of the usage of this interface and every feature mentioned inside the examples named shot-consumer and shot-consumer-flavors.

In case you need to override the inner dependency Shot includes in your project during the plugin configuration you can do it as follows:

dependencies {
  ....
  shotDependencies "com.karumi:shot-android:ANY_VERSION_YOU_WANT_TO_USE"
  ....
}

Keep in mind you'll need to use a compatible version or Shot won't work as expected.

ActivityScenario support

ActivityTestRule has been deprecated and now the usage of ActivityScenario is recommended. However, Shot needs to be executed from the instrumentation thread to be able to extract all the test metadata needed to record and verify screenshots. That's why we've created an ActivityScenario extension method named waitForActivity. This extension is needed get the activity instance from the instrumentation thread instead of running Shot from the app target thread. Using this extension you can write tests like this:

class MainActivityTest: ScreenshotTest {
    @Test
    fun rendersTheDefaultActivityState() {
        val activity = ActivityScenario.launch(MainActivity::class.java)

        compareScreenshot(activity)
    }
}

I hope we can find a better solution for this issue in the future.

Executing tests from Android Studio

Shot is a Gradle plugin and it does not integrate with AS by default. After running your tests, Shot Gradle plugin will fetch the screenshots generated during tests' execution and use them to check if your tests are passing or not. You always can run your tests from command line as explained above. However, if you want to run your tests from AS you can create a configuration like this:

Keep in mind the debugger may not work if use use this option. If you want to debug your tests you can run them from Android Studio as you'd do with any other instrumentation test and you may need to execute this command before running your test:

adb rm -rf /storage/emulated/0/Download/screenshots/*

Executing tests in multiple devices

If after some time writing screenshot tests your build takes too long to run our recommendation is to run your tests in multiple devices. Sharding your tests execution will split your test suite into different devices so your tests execution time will be reduced. This feature is not designed to test the UI across different platforms or screen resolutions, to do that we'd recommend you to configure the size of the screenshot taken by modifing the view height and width. To run your tests in multiple devices you can use Composer and the official Gradle Plugin they provide. Composer will take all your tests and will split the test suite execution between all the connected devices. Remember, if you are going to use more than one device all the devices should use the same Android OS and the same screen resolution and density! Keep also in mind composer needs Gradle 5.4.1 to be able to run your tests using multiple devices.

Once you've configured composer to run your tests you only need to update Shot to use the composer task as the instrumentation test task as follows:

shot {
  useComposer = true
}

If you are using orchestrator remember to enable it in composer configuration:

composer {
  // ...
  withOrchestrator true
  // ...
}

Tolerance

Shot provides a simple mechanism to be able to configure a threshold value when comparing recorded images with the new ones during the verification stage. You may need to use tolerance in your tests when testing compose components because the API Shot uses to record screenshots depending on the device where your tests are executed. There are other scenarios where you may need to configure a tolerance value, but these are not so common. If you want to configure it you can use this config in your build.gradle file.

shot {
    tolerance =  0.1 // 0,1% tolerance
}

If you configure tolerance = 100 it means the tolerance threshold will be 100% and all your tests will pass even if they should fail...so be careful when configuring this param.

Take into account the instrumentationTestTask could be different if you use different flavors or build types. Remember also you should use Shot > 3.0.0 because this feature was introduced in this release!

CI Reporting

Shot generates an HTML report you can review at the end of the recording or verification build. However, if you are running Shot in a CI environment which does not support saving the reporting files generated by the build you can verify your tests using this command ./gradlew debugExecuteScreenshotTests -PprintBase64. This will change how Shot show comparision errors displaying a command you can copy and paste on your local terminal for every screenshot test failed.

If you want to see only failing tests in output, please add showOnlyFailingTestsInReports = true as an option to your build.gradle.

Running only some tests

You can run a single test or test class, just add the android.testInstrumentationRunnerArguments.class parameter within your gradle call. This option works for both modes, verification and recording, just remember to add the -Precord if you want to do the latter.

Running all tests in a class:

./gradlew <Flavor><BuildType>executeScreenshotTests -Pandroid.testInstrumentationRunnerArguments.class=com.your.package.YourClassTest

Running a single test:

./gradlew <Flavor><BuildType>executeScreenshotTests -Pandroid.testInstrumentationRunnerArguments.class=com.your.package.YourClassTest#yourTest

Using shot on API 29+

Since Shot 5.8.0 we provide support for devices running API >= 29. There is no need to configure android:requestLegacyExternalStorage="true" or provide any special storage permission in your test ``AndroidManifest`.

Development documentation

Inside this repository you'll find two types of projects. The first one is all the code related to the Gradle Plugin. The second one is all the code related to the example projects we use as consumers of the Gradle Plugin and also as end to end tests. Review the following folders when developing a new feature for Shot or fixing a bug:

Plugin related folders:

• shot: Main Gradle Plugin module.
• core: Gradle Plugin independent module where all the shot business logic is placed.
• shot-android: Android library included in the project where Shot is installed automatically when the Gradle Plugin is initialized.

Consumers and end to end tests folder:

• shot-consumer: A simple project created to test Shot and show how the library works.
• shot-consumer-flavors: A simple project focused on testing how the flavors feature works.

When developing a feature or fixing a bug, plugin related folders are the place where most of the code will be. When writing end to end tests you should review the consumers' folders. Unit or integration tests should be added in the first group of folders using all the tooling already installed and configured for you.

When talking about the IDE we should use, our recommendation is simple. We should use IntelliJ Idea for the Gradle Plugin related code and Android Studio for the consumers. Keep in mind the consumers' configuration is not linked with the root Gradle file so you won't be able to build the consumers from the root of this project using Android Studio. That's why we recommend the usage of different IDEs using different root Gradle configurations when working with this repository.

Steps to develop a feature or fix a bug:

• Review our CONTRIBUTING.md file. There you'll find some general development rules.
• Open an issue or drop a comment to explain what you'd like to implement or to fix. Communication is really important for us and we recommend dropping a comment or opening an issue before to avoid development issues.
• Fork this repository and create a new branch where you'll develop the new feature or fix.
• Install IntelliJ and Android Studio.
• Import the root Gradle file configuration with IntelliJ. You will have to install the Scala plugin recommended by IntelliJ, but don't worry, as soon as you start IntelliJ you'll see a pop-up with the recommended plugin.
• Add RELEASE_SIGNING_ENABLED=false to your local properties to be able to install artifacts in your local maven repository without signing them.
• Develop the code you'd like to implement. Remember to add the unit/integration test coverage to the code you are working on.
• Execute the Gradle task ./gradlew publishToMavenLocal from the root folder in order to update your plugin local installation.
• Using Android Studio import shot-android or shot-consumer-flavors and write an example of the feature or fix you coded before. The example test you write will work as an end to end test.
• Commit and push the code. Our CI configuration will ensure everything is working as expected!
• Remember to execute ./gradlew publishToMavenLocal whenever you change the Gradle plugin code related in order to update your local repository and be able to use it from the consumers folder.
• Once you are ready, send a PR. We will review it and help you to contribute to the official repository. Once everything is ready, we will merge it and release a new version.

In case you need to start an Android emulator you have scripts inside the consumers' folder you can execute to create the emulators you'll need to run the tests 😃

This is the list of most useful Gradle tasks you might need divided by type of project:

• Plugin related tasks:

  • ./gradlew publishToMavenLocal: Install Shot in your local gradle repository.
  • ./gradlew test: Execute all the tests related to the Gradle plugin.
  • ./gradlew scalafmtAll: Review the Gradle Plugin's checkstyle.

• Consumers related tasks:

  • ./gradlew executeScreenshotTests: Execute all the screenshot tests for the associated consumer in verification mode.
  • ./gradlew executeScreenshotTests -Precord: Execute all the screenshot tests for the associated consumer in record mode.

iOS support

If you want to apply the same testing technique on iOS you can use Swift Snapshot Testing

Developed By

• Pedro Vicente Gómez Sánchez - [email protected]

License

Copyright 2022 Pedro Vicente Gómez Sánchez

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

shot's Issues

No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack @Test annotations).

Hi, Thanks for providing this great library. I need help from you when I am trying to execute screenshot through gradle command and it gives some gradle related exceptions, previously It was working fine but from last week I testing and getting one issue explained below:
No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack @Test annotations).

When I am running manually it is succeed, but using command it is giving error after 97% , when pulling screenshots

Feature Request: Only start a single test for recording

I might be overlooking something, but is it possible to only run the "recording" mode for one single layout?

Example where this feature would be useful:
Let's say you have to change a xml layout foo.xml i.e. change the padding-top value which actually uses a dimen defined under @dimen/padding_top.
If some other layout like bar.xml also uses@dimen/padding_top which you are not aware of, bar.xml layout will also look different suddenly. However, if you re-record all screenshots then you will miss the changes on bar.xml because not only the screenshot for foo.xml has been re-recoreded, but also the screenshot of bar.xml (which means the "visual error" in bar.xml is not detected)

Can we retain the existing report instead of overwriting the previous one.This will be more useful for many.

Expected behaviour

Actual behaviour

Steps to reproduce

Version of the library

Start comparing images and getting the difference between every pair of screenshots. To be able to do this we will have to parse the metadata file we've pulled from the connected device.

Trying to run with runInstrumentation = false

I want to verify the snapshots with files downloaded from a remote device, but I dont know where I should put the files to be compared.

The recorded files are being saved at folder $projectdir/screenshots ( I think that those files are the current value of valid snapshots right?)

I just need to know where I should put the files downloaded from the remote device to verify with runInstrumentation = false

Can you help me?

Multi-test record issues

Hi,

I used : executeScreenshotTests -Precord --stacktrace -Pandroid.testInstrumentationRunnerArguments.class=<com.your.package.ClassTest> to record all tests in this class

in each test at the end of the test :
Screenshot.snapActivity(activity).record()

But only the last test is uploaded to /screenshots
Expected behaviour : all tests should be recorded

Version of the library : 2.2.0

I found : before execute a test, the folder in emulator sdcard/screenshots/appid.test/ is earsed

Multiple module support?

Expected behaviour

Others modules from android solution that apply shot plugin should have access to shot and com.facebook.testing.screenshot.Screenshot and com.facebook.testing.screenshot.ScreenshotRunner

Actual behaviour

Only app module have access to com.facebook.testing.screenshot.Screenshot and com.facebook.testing.screenshot.ScreenshotRunner

Steps to reproduce

1 - apply shot plugin to app and other module.

Version of the library

3.0.2

Additional info

Kotlin version 1.3.41
compileSdk 28
targetSdk 28
minSdk 23
gradle version: 3.4.2

plugins that other is applying:

1. com.android.library
2. kotlin-android
3. kotlin-kapt
4. androidx.navigation.safeargs.kotlin
5. shot

Add a configurable tolerance.

Our current implementation compares the recorded screenshots used as a baseline and the new one taken from the device using a pixel-perfect strategy. We should provide a config value we can use to compare the screenshots letting the user consider two screenshots as the same even if some pixels are not the same. This is the code we should modify. I'd count the number of different pixels as a percentage before considering the comparison as a failure. The tolerance value should be easily configured from the build.gradle file.

NPE recording screenshots

I want to run yours plugin
I do:
./gradlew executeScreenshotTests -Precord

But get some errors:

⬇️  Pulling screenshots from your connected devices!

> Task :presentation:executeScreenshotTests FAILED
💾  Saving screenshots.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':presentation:executeScreenshotTests'.
> java.io.FileNotFoundException: /Users/ivan/AndroidStudioProjects/android-client/presentation/screenshots/screenshots-default/com.kg.project.presentation.RegressSuite_test_dump.json (No such file or directory)

But I found this Json at zip archive screenshot_bundle.zip

What I should do to avoid this error?

Thanks in advance

Version of the library

com.karumi:shot:3.0.2

Test fail due to cursor on EditText

Hi,
Im trying to Screenshot my Login screen. My EditTex has focus when the activity start, and shows the cursor over it. When I execute the Screenshot Tests this fail because some times the cursor is there some time not (native animation of cursor). What can i do?. (Remove the focus in not an option due to UX)
Thanks.

Verification report with diff between two different screenshots

Improve our verification report showing a diff between the orginal screenshot and the new one.

Snapshot record/verify to support large views

After upgrading from Karumi 2.0 to Karumi 3.0.2, I get the following error when tried to record/verify the snapshot of screens with heightPixels >= 9000 (Which is used to take the snapshot of entire screen without having to scroll)

java.lang.RuntimeException: java.lang.IllegalStateException: View too large: (1440, 9000)

Function called - ViewHelpers.setupView(view)
.setExactHeightPx(screenHeight)
.setExactWidthPx(displayMetrics.widthPixels)
.layout()
where screenHeight = 9000 and displayMetrics.widthPixels returns 1440

Failure executing task executeScreenshotTests due to SAXParseException

Expected behaviour

executeScreenshotTests or executeScreenshotTests -Precord should work as expected

Actual behaviour

Exception happens executing those tasks:

* What went wrong:
Execution failed for task ':app:executeScreenshotTests'.
> org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; El contenido no est? permitido en el pr?logo.

Version of the library

Tested on 1.0.0, 0.3.0, 0.2.0

Extended Stacktrace

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':app:executeScreenshotTests'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:100)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:70)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:63)
        at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:88)
        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.run(DefaultTaskGraphExecuter.java:248)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:336)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:328)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:197)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:107)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:241)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:230)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.processTask(DefaultTaskPlanExecutor.java:124)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.access$200(DefaultTaskPlanExecutor.java:80)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:105)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:99)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.execute(DefaultTaskExecutionPlan.java:625)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.executeWithTask(DefaultTaskExecutionPlan.java:580)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:99)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: org.gradle.internal.UncheckedException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; El contenido no est? permitido en el pr?logo.
        at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:63)
        at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:40)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:76)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.doExecute(DefaultTaskClassInfoStore.java:141)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:134)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:121)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:731)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:705)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:122)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:336)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:328)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:197)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:107)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:111)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:92)
        ... 27 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; El contenido no est? permitido en el pr?logo.
        at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
        at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
        at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)
        at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
        at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1472)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:994)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:327)
        at scala.xml.factory.XMLLoader.loadXML(XMLLoader.scala:41)
        at scala.xml.factory.XMLLoader.loadXML$(XMLLoader.scala:37)
        at scala.xml.XML$.loadXML(XML.scala:60)
        at scala.xml.factory.XMLLoader.loadString(XMLLoader.scala:60)
        at scala.xml.factory.XMLLoader.loadString$(XMLLoader.scala:60)
        at scala.xml.XML$.loadString(XML.scala:60)
        at com.karumi.shot.xml.ScreenshotsSuiteXmlParser$.parseScreenshotSize(ScreenshotsSuiteXmlParser.scala:57)
        at com.karumi.shot.Shot.$anonfun$readScreenshotsMetadata$1(Shot.scala:121)
        at scala.collection.parallel.AugmentedIterableIterator.map2combiner(RemainsIterator.scala:112)
        at scala.collection.parallel.AugmentedIterableIterator.map2combiner$(RemainsIterator.scala:109)
        at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2combiner(ParVector.scala:62)
        at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1052)
        at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:49)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
        at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:63)
        at scala.collection.parallel.Task.tryLeaf(Tasks.scala:52)
        at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:46)
        at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1049)
        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal(Tasks.scala:156)
        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal$(Tasks.scala:153)
        at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:440)
        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:146)
        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:145)
        at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
        at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync(Tasks.scala:375)
        at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync$(Tasks.scala:375)
        at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:440)
        at scala.collection.parallel.ForkJoinTasks.executeAndWaitResult(Tasks.scala:423)
        at scala.collection.parallel.ForkJoinTasks.executeAndWaitResult$(Tasks.scala:414)
        at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(TaskSupport.scala:56)
        at scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult(Tasks.scala:555)
        at scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult$(Tasks.scala:555)
        at scala.collection.parallel.ExecutionContextTaskSupport.executeAndWaitResult(TaskSupport.scala:80)
        at scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParIterableLike.scala:956)
        at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:49)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
        at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:63)
        at scala.collection.parallel.Task.tryLeaf(Tasks.scala:52)
        at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:46)
        at scala.collection.parallel.ParIterableLike$ResultMapping.tryLeaf(ParIterableLike.scala:951)
        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:149)
        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:145)
        at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
        Suppressed: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; El contenido no est? permitido en el pr?logo.
                at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
                at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
                at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)
                at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
                at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1472)
                at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:994)
                at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
                at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
                at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
                at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
                at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
                at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
                at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
                at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:327)
                at scala.xml.factory.XMLLoader.loadXML(XMLLoader.scala:41)
                at scala.xml.factory.XMLLoader.loadXML$(XMLLoader.scala:37)
                at scala.xml.XML$.loadXML(XML.scala:60)
                at scala.xml.factory.XMLLoader.loadString(XMLLoader.scala:60)
                at scala.xml.factory.XMLLoader.loadString$(XMLLoader.scala:60)
                at scala.xml.XML$.loadString(XML.scala:60)
                at com.karumi.shot.xml.ScreenshotsSuiteXmlParser$.parseScreenshotSize(ScreenshotsSuiteXmlParser.scala:57)
                at com.karumi.shot.Shot.$anonfun$readScreenshotsMetadata$1(Shot.scala:121)
                at scala.collection.parallel.AugmentedIterableIterator.map2combiner(RemainsIterator.scala:112)
                at scala.collection.parallel.AugmentedIterableIterator.map2combiner$(RemainsIterator.scala:109)
                at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2combiner(ParVector.scala:62)
                at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1052)
                at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:49)
                at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
                at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:63)
                at scala.collection.parallel.Task.tryLeaf(Tasks.scala:52)
                at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:46)
                at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1049)
                at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal(Tasks.scala:156)
                at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal$(Tasks.scala:153)
                at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:440)
                at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:146)
                at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:145)
                at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
                at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
                at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
                at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
                at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
                at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
                Suppressed: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; El contenido no est? permitido en el pr?logo.
                        at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
                        at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
                        at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)
                        at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
                        at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1472)
                        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:994)
                        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
                        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
                        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
                        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
                        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
                        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
                        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
                        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:327)
                        at scala.xml.factory.XMLLoader.loadXML(XMLLoader.scala:41)
                        at scala.xml.factory.XMLLoader.loadXML$(XMLLoader.scala:37)
                        at scala.xml.XML$.loadXML(XML.scala:60)
                        at scala.xml.factory.XMLLoader.loadString(XMLLoader.scala:60)
                        at scala.xml.factory.XMLLoader.loadString$(XMLLoader.scala:60)
                        at scala.xml.XML$.loadString(XML.scala:60)
                        at com.karumi.shot.xml.ScreenshotsSuiteXmlParser$.parseScreenshotSize(ScreenshotsSuiteXmlParser.scala:57)
                        at com.karumi.shot.Shot.$anonfun$readScreenshotsMetadata$1(Shot.scala:121)
                        at scala.collection.parallel.AugmentedIterableIterator.map2combiner(RemainsIterator.scala:112)
                        at scala.collection.parallel.AugmentedIterableIterator.map2combiner$(RemainsIterator.scala:109)
                        at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2combiner(ParVector.scala:62)
                        at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1052)
                        at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:49)
                        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
                        at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:63)
                        at scala.collection.parallel.Task.tryLeaf(Tasks.scala:52)
                        at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:46)
                        at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1049)
                        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:149)
                        ... 7 more

Improve support for running on cloud devices

Something I faced when trying to configure Shot for a real project.

The example setup looks something along the lines: the project is built on Circle CI (or any other provider), connected tests are run on Firebase Test Lab. The Test Lab takes an app apk + a test apk and runs them on a real device or an emulator, as an output producing contents of the device's sdcard. The further flow then is to download the sdcard contents, find the screenshots folder, and move it to a location Shot expects it in.

Then we can compare the screenshots against the baseline with the executeScreenshotTests task. The problem is the task is hardcoded to execute the connected tests on a device + do all the downloading / cleaning up stuff. So in the environment without any access to a device, the user of the library has to execute something like ./gradlew executeScreenshotTests -x downloadScreenshots -x removeScreenshots -x connectedAndroidTest, which is obviously a hack.

While I'm not sure what the best solution would be, maybe it makes sense to extract out a screenshot comparing task, so that it can be run separately if needed?

Improve coverage

Cover the image comparison implementation using real images and improve unit tests written for the main class named Shot. Integration tests will be hard to implement, but we should at list write unit tests.

Implement record mode as described in the project readme

Right now we can just compare screenshots, we should also be able to remove the already recorded screenshots and pull the new one storing the result in our screenshots folder.

Ensure Shot is working with java 12

Failure to generate test results

Expected behaviour

After running the command ./gradlew executeScreenshotTests -Precord using the following repository https://github.com/AOrobator/ScreenshotTest, I expect the build to pass and screenshots to be recorded.

Actual behaviour

Crashes when I run the recording command.

Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
        at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
        at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
        at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)
        at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
        at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1472)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:994)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:327)
        at scala.xml.factory.XMLLoader.loadXML(XMLLoader.scala:41)
        at scala.xml.factory.XMLLoader.loadXML$(XMLLoader.scala:37)
        at scala.xml.XML$.loadXML(XML.scala:60)
        at scala.xml.factory.XMLLoader.loadString(XMLLoader.scala:60)
        at scala.xml.factory.XMLLoader.loadString$(XMLLoader.scala:60)
        at scala.xml.XML$.loadString(XML.scala:60)
        at com.karumi.shot.xml.ScreenshotsSuiteXmlParser$.parseScreenshotSize(ScreenshotsSuiteXmlParser.scala:57)
        at com.karumi.shot.Shot.$anonfun$readScreenshotsMetadata$1(Shot.scala:131)
        at scala.collection.parallel.AugmentedIterableIterator.map2combiner(RemainsIterator.scala:112)
        at scala.collection.parallel.AugmentedIterableIterator.map2combiner$(RemainsIterator.scala:109)
        at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2combiner(ParVector.scala:62)
        at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1052)
        at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:49)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
        at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:63)
        at scala.collection.parallel.Task.tryLeaf(Tasks.scala:52)
        at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:46)
        at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1049)
        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:149)
        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:145)
        at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
        at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync(Tasks.scala:375)
        at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync$(Tasks.scala:375)
        at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:440)
        at scala.collection.parallel.ForkJoinTasks.executeAndWaitResult(Tasks.scala:423)
        at scala.collection.parallel.ForkJoinTasks.executeAndWaitResult$(Tasks.scala:414)
        at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(TaskSupport.scala:56)
        at scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult(Tasks.scala:555)
        at scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult$(Tasks.scala:555)
        at scala.collection.parallel.ExecutionContextTaskSupport.executeAndWaitResult(TaskSupport.scala:80)
        at scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParIterableLike.scala:956)
        at scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:49)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
        at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:63)
        at scala.collection.parallel.Task.tryLeaf(Tasks.scala:52)
        at scala.collection.parallel.Task.tryLeaf$(Tasks.scala:46)
        at scala.collection.parallel.ParIterableLike$ResultMapping.tryLeaf(ParIterableLike.scala:951)
        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:149)
        at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute$(Tasks.scala:145)
        at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)

Steps to reproduce

Run ./gradlew executeScreenshotTests -Precord using the following repository https://github.com/AOrobator/ScreenshotTest

Version of the library

'com.facebook.testing.screenshot:plugin:0.5.0'
'com.karumi:shot:1.0.0'

Improve travis configuration

Improve Travis CI configuration to use the plugin in the shot-consumer after the build.

Build failure after upgrade to Gradle 3.5.0

Expected behaviour

Gradle sync completes

Actual behaviour

Gradle sync fails with Gradle plugin 3.5.0

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* Where:
Build file '/Users/phil.bayfield/***/***/screenshot/build.gradle' line: 13

* What went wrong:
A problem occurred evaluating project ':screenshot'.
> Failed to apply plugin [id 'shot']
   > Extension not initialized yet, couldn't access compileSdkVersion.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
A problem occurred configuring project ':screenshot'.
> compileSdkVersion is not specified.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

CONFIGURE FAILED in 3s
Extension not initialized yet, couldn't access compileSdkVersion.

Line 13 is: apply plugin: 'shot'

Steps to reproduce

Upgraded Gradle plugin from 3.4.2 to 3.5.0

Version of the library

2.2.0 (tried 3.0.0 but had issue #62)

Error:Configuration with name 'androidTestImplementation' not found.

Expected behaviour

When using the Android gradle plugin <3.0 the plugin should use androidTestCompile

Actual behaviour

it's adding androidTestImplementation so it doesn't compile

The issue seems to be in that function
https://github.com/Karumi/Shot/blob/adeef269ab719ffa776b96d50d7928c0c2b96fa8/shot/src/main/scala/com/karumi/shot/ShotPlugin.scala#L82

Steps to reproduce

Apply the plugin to a project using Gradle 4.2.1 but Android plugin 2.3.3

Version of the library

0.3.0

FileNotFoundException for metdata.xml file while running a particular case

Expection Got :
java.io.FileNotFoundException: /Users/...../app/screenshots/screenshots-default/metadata.xml (No such file or directory)

Command Run : ./gradlew executeScreenshotTests -Pandroid.testInstrumentationRunnerArguments.class=testcases.Test0AppLaunch -Precord

Am I did any mistake ?
while running for the whole test suite its working fine.

NoSuchFileException during record task

Expected behaviour

Screenshots are recorded without any error

Actual behaviour

the gradle task ./gradlew executeScreenshotTests -Precord is crashing with a NoSuchFileException

Execution failed for task ':libs:lego:executeScreenshotTests'.
java.nio.file.NoSuchFileException: C:\R\project\root\libs\lego\screenshots\screenshots-default\com.company.android.ui.components.screenshot.JustAtest_theActivityIsShownProperly.png

Steps to reproduce

execute gradle command

Version of the library

0.2.0

Notes

After checking the folder, the file is not there, but I figured out that there is another folder called "screenshots-default" inside the "screenshots-default" which actually includes the file. So the correct to the file would be C:\R\project\root\libs\lego\screenshots\screenshots-default\screenshots-default\com.company.android.ui.components.screenshot.JustAtest_theActivityIsShownProperly.png

Generation of screenshots throws NPE with `runInstrumentation false`

Looks like it's because https://github.com/Karumi/Shot/blob/64d160291ed1564307507a5830bbb56e5e932d51/core/src/main/scala/com/karumi/shot/Shot.scala#L136 returns null if the directory doesn't exist.

Worked around by creating /screenshots/screenshot-default/.

Caused by: java.lang.NullPointerException
	at scala.collection.mutable.ArrayOps$ofRef$.newBuilder$extension(ArrayOps.scala:195)
	at scala.collection.mutable.ArrayOps$ofRef.newBuilder(ArrayOps.scala:191)
	at scala.collection.TraversableLike.filterImpl(TraversableLike.scala:246)
	at scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:245)
	at scala.collection.mutable.ArrayOps$ofRef.filterImpl(ArrayOps.scala:191)
	at scala.collection.TraversableLike.filter(TraversableLike.scala:259)
	at scala.collection.TraversableLike.filter$(TraversableLike.scala:259)
	at scala.collection.mutable.ArrayOps$ofRef.filter(ArrayOps.scala:191)
	at com.karumi.shot.Shot.readScreenshotsMetadata(Shot.scala:137)
	at com.karumi.shot.Shot.recordScreenshots(Shot.scala:48)
	at com.karumi.shot.tasks.ExecuteScreenshotTests.executeScreenshotTests(Tasks.scala:56)
	at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:103)

Running multiple tests using Espresso.

I am combing the espresso with Karumi shot. It generates the report successfully for 2 tests if they fail the comparison. However, if the espresso cannot find some element for 1 test and fail at some point. The report won't be generated at all.
Thus I am not able to know any information, any ideas how to fix this?
Thanks

Problems with gradle version 3.0.0-beta or upper.

Expected behaviour

execute the gradle task

Actual behaviour

when running the gradle task throw this error:

Configure project :uripatch
Configuration 'androidTestCompile' in project ':uripatch' is deprecated. Use 'androidTestImplementation' instead.
Configuration 'compile' in project ':uripatch' is deprecated. Use 'implementation' instead.

and after that this cause a Multidex exception:

Dex: Error converting bytecode to dex:
Cause: com.android.dex.DexException: Multiple dex files define Lcom/android/dx/dex/file/IdItem;
UNEXPECTED TOP-LEVEL EXCEPTION:
com.android.dex.DexException: Multiple dex files define Lcom/android/dx/dex/file/IdItem;

Steps to reproduce

1. create a kotlin proyect with gradle build version 3.0.0-beta-1 or upper.
2. add shot plugin
3. try to record the tests with ./gradlew executeScreenshotTests -Precord
4. Throw the exception

Version of the library

version 0.1.2

Not getting the screenshots even when granting permissions in api23.

I posted the question in stackoverflow, all the details are there: https://stackoverflow.com/questions/46126100/facebooks-screenshot-tests-failing-with-api-bigger-then-23

Exception when Gradle Sync: Could not find com.karumi:core:2.2.1-SNAPSHOT

Expected behaviour

Gradle sync works correctly

Actual behaviour

An exception is thrown when gradle sync is running

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring root project 'tipsterchat-android'.
> Could not resolve all artifacts for configuration ':classpath'.
   > Could not find com.karumi:core:2.2.1-SNAPSHOT.
     Searched in the following locations:
       - https://dl.google.com/dl/android/maven2/com/karumi/core/2.2.1-SNAPSHOT/maven-metadata.xml
       - https://dl.google.com/dl/android/maven2/com/karumi/core/2.2.1-SNAPSHOT/core-2.2.1-SNAPSHOT.pom
       - https://dl.google.com/dl/android/maven2/com/karumi/core/2.2.1-SNAPSHOT/core-2.2.1-SNAPSHOT.jar
       - https://jcenter.bintray.com/com/karumi/core/2.2.1-SNAPSHOT/maven-metadata.xml
       - https://jcenter.bintray.com/com/karumi/core/2.2.1-SNAPSHOT/core-2.2.1-SNAPSHOT.pom
       - https://jcenter.bintray.com/com/karumi/core/2.2.1-SNAPSHOT/core-2.2.1-SNAPSHOT.jar
       - https://repo.maven.apache.org/maven2/com/karumi/core/2.2.1-SNAPSHOT/maven-metadata.xml
       - https://repo.maven.apache.org/maven2/com/karumi/core/2.2.1-SNAPSHOT/core-2.2.1-SNAPSHOT.pom
       - https://repo.maven.apache.org/maven2/com/karumi/core/2.2.1-SNAPSHOT/core-2.2.1-SNAPSHOT.jar
       - http://dl.bintray.com/kotlin/kotlin-eap/com/karumi/core/2.2.1-SNAPSHOT/maven-metadata.xml
       - http://dl.bintray.com/kotlin/kotlin-eap/com/karumi/core/2.2.1-SNAPSHOT/core-2.2.1-SNAPSHOT.pom
       - http://dl.bintray.com/kotlin/kotlin-eap/com/karumi/core/2.2.1-SNAPSHOT/core-2.2.1-SNAPSHOT.jar
       - https://maven.fabric.io/public/com/karumi/core/2.2.1-SNAPSHOT/maven-metadata.xml
       - https://maven.fabric.io/public/com/karumi/core/2.2.1-SNAPSHOT/core-2.2.1-SNAPSHOT.pom
       - https://maven.fabric.io/public/com/karumi/core/2.2.1-SNAPSHOT/core-2.2.1-SNAPSHOT.jar
     Required by:
         project : > com.karumi:shot:3.0.0

Steps to reproduce

Update the version to 3.0.0

Version of the library

3.0.0

Tested on Android Studio 3.4.2 and 3.5

Auto import not working in our gradle plugin for the newest Gradle version.

We should review the code adding the facebook library under the hood because it is not working as @Serchinastico discovered yesterday.

downloadScreenshots FAILED - Nonzero exit value: 1

Hi,
I'm trying to use Shot in my app, but I don't what im doing wrong. I tried to follow all steps, somebody can help me? 😄

Actual error

Task :presentation:downloadScreenshots FAILED
⬇️ Pulling screenshots from your connected devices!

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':presentation:downloadScreenshots'.

Nonzero exit value: 1

Actual exception

* Exception is: org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':presentation:downloadScreenshots'. at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:187) at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:263) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:185) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:166) at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:109) at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46) at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:62) at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57) at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56) at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52) at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416) at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406) at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158) at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102) at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52) at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:41) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:374) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:361) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:354) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:340) at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127) at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191) at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182) at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64) at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48) at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56) Caused by: java.lang.RuntimeException: Nonzero exit value: 1 at scala.sys.package$.error(package.scala:27) at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.slurp(ProcessBuilderImpl.scala:134) at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.$bang$bang(ProcessBuilderImpl.scala:104) at com.karumi.shot.android.Adb.executeAdbCommandWithResult(Adb.scala:33) at com.karumi.shot.android.Adb.pullScreenshots(Adb.scala:23) at com.karumi.shot.Shot.$anonfun$pullScreenshots$1(Shot.scala:121) at com.karumi.shot.Shot.$anonfun$pullScreenshots$1$adapted(Shot.scala:118) at scala.collection.immutable.List.foreach(List.scala:389) at com.karumi.shot.Shot.pullScreenshots(Shot.scala:118) at com.karumi.shot.Shot.$anonfun$downloadScreenshots$1(Shot.scala:40) at com.karumi.shot.Shot.$anonfun$downloadScreenshots$1$adapted(Shot.scala:38) at com.karumi.shot.Shot.executeIfAppIdIsValid(Shot.scala:103) at com.karumi.shot.Shot.downloadScreenshots(Shot.scala:38) at com.karumi.shot.tasks.DownloadScreenshotsTask.downloadScreenshots(Tasks.scala:84) at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:104) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:49) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:42) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:28) at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:721) at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:688) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$3.run(ExecuteActionsTaskExecuter.java:539) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:402) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:394) at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:92) at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:524) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:507) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.access$300(ExecuteActionsTaskExecuter.java:109) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.executeWithPreviousOutputFiles(ExecuteActionsTaskExecuter.java:258) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.execute(ExecuteActionsTaskExecuter.java:247) at org.gradle.internal.execution.steps.ExecuteStep.lambda$execute$1(ExecuteStep.java:33) at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:33) at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:26) at org.gradle.internal.execution.steps.CleanupOutputsStep.execute(CleanupOutputsStep.java:63) at org.gradle.internal.execution.steps.CleanupOutputsStep.execute(CleanupOutputsStep.java:35) at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:49) at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:34) at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:43) at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:73) at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:54) at org.gradle.internal.execution.steps.CatchExceptionStep.execute(CatchExceptionStep.java:34) at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:44) at org.gradle.internal.execution.steps.SnapshotOutputsStep.execute(SnapshotOutputsStep.java:54) at org.gradle.internal.execution.steps.SnapshotOutputsStep.execute(SnapshotOutputsStep.java:38) at org.gradle.internal.execution.steps.CacheStep.executeWithoutCache(CacheStep.java:153) at org.gradle.internal.execution.steps.CacheStep.execute(CacheStep.java:67) at org.gradle.internal.execution.steps.CacheStep.execute(CacheStep.java:41) at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:49) at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:44) at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:33) at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:38) at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:24) at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:92) at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$0(SkipUpToDateStep.java:85) at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:55) at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:39) at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:76) at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:37) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:36) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:26) at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:94) at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:49) at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:79) at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:53) at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:74) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.lambda$execute$2(SkipEmptyWorkStep.java:78) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:78) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:34) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:39) at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:40) at org.gradle.internal.execution.steps.LoadExecutionStateStep.execute(LoadExecutionStateStep.java:28) at org.gradle.internal.execution.impl.DefaultWorkExecutor.execute(DefaultWorkExecutor.java:33) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:174) ...

Version of the library

classpath "com.android.tools.build:gradle:3.5.0"
classpath "com.karumi:shot:3.0.2"

Improve error or success feedback with a reporting summary

Most of the testing frameworks show an execution summary as part of the execution result using html, xml or json formats. We should do the same.

Update Faceebook library

As you can see here, Facebook published new releases of their library so it is time to update the library on our side.

Delete temporal data generated while pulling screenshots

We need to delete temporal data pulled from the connected device while pulling the screenshots taken. Every file inside screenshots/screenshots-default

Show a list of reference images that are not being used

I'd like Shot to tell me if it finds some reference images that are not being used while running it. When you decide to rename your tests it often leaves some orphan screenshots so I think it's great if Shot displayed a warning in that situation.

ExecuteScreenshots fails with exception: Unable to delete file metadata.xml on Windows10 with ADB over WIFI

Expected behaviour

Recording screenshots on Windows 10 with Nexus 4 (Android 5.1.1) connected via ADB over WIFI

Actual behaviour

The task gradlew executeScreenshotTests -Precord takes the screenshots, pulls and saves them and the report, but the process finishes with:

Execution failed for task ':app:executeScreenshotTests'. java.io.IOException: Unable to delete file: C: ... \app\screenshots\screenshots-default\metadata.xml

• The same task works fine on Linux Ubuntu 18.04.1 (Nexus 4, ADB over WIFI)
• The same task works fine using USB connection (Nexus 4, Win 10)
• The same task works fine with Samsung Galaxy Note 8 (ADB over Wifi, Win 10)

Steps to reproduce

execute command on Windows 10 in AndroidStudio Terminal or PowerShell
tested on: Nexus 4 with android 5.1.1
targetSdkVersion 28
minSdkVersion 15
custom testInstrumentationRunner:

public class ScreenshotAndroidJunitRunner extends AndroidJUnitRunner {
    @Override
    public void onCreate(Bundle arguments) {
        ScreenshotRunner.onCreate(this, arguments);
        super.onCreate(arguments);
    }

    @Override
    public void finish(int resultCode, Bundle results) {
        ScreenshotRunner.onDestroy();
        super.finish(resultCode, results);
    }
}

shot gradle settings:

shot {
    appId = 'myAppId'
    instrumentationTestTask = 'connectedDebugAndroidTest'
    packageTestApkTask = 'packageDebugAndroidTest'
}

Test class:

@LargeTest
@RunWith(AndroidJUnit4.class)
public class FirstEspressoTest {

    @Rule
    public ActivityTestRule<MainActivity> mActivityRule =
            new ActivityTestRule<>(MainActivity.class);

    @Test
    public void settingsArePresent() throws InterruptedException {
        Thread.sleep(5000);

        //ui interaction
        onView(withId(R.id.action_settings))
                .perform(click());

        //take screenshot
        TakeScreenshotUtil.takeScreenshot("settingsArePresent", mActivityRule.getActivity());

        //check result
        Context targetContext = InstrumentationRegistry.getTargetContext();
        onView(withText(targetContext.getResources().getString(R.string.action_settings)))
                .check(matches(ViewMatchers.isDisplayed()));
    }
}

TakeScreenshotUtil class:

public class TakeScreenshotUtil {
    public static void takeScreenshot(String name, Activity activity) throws InterruptedException {
        Thread.sleep(2500);
        Log.i("Taking screenshot of ", activity.getLocalClassName() + "(named: " + name + ")");
        InstrumentationRegistry.getInstrumentation().waitForIdleSync();
        Screenshot.snapActivity(activity).record();
    }
}

I can not delete the file metadata.xml manually when AndroidStudio is open. I tried to run the command from PowerShell with AndroidStudio closed, but I get the same error.

Version of the library

shot 2.1.1
with facebooks screenshot testing plugin 0.8.0
with gradle 3.0.0

Add support for Spoon

I am trying to use Shot with Spoon by configuring

shot {
appId = ""
instrumentationTestTask = "spoonNormalDevDebugAndroidTest"
packageTestApkTask = "packageNormalDevDebugAndroidTest"
}

but when I run the task ./gradlew executeScreenshotTests -Precord it is throwing

Task :app-gomoney:executeScreenshotTests FAILED
💾 Saving screenshots.

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':app-gomoney:executeScreenshotTests'.

java.io.FileNotFoundException: path of directory/screenshots/screenshots-default/metadata.xml (No such file or directory)

However it works fine with

shot {
appId = ""
instrumentationTestTask = "connectedNormalDevDebugAndroidTest"
packageTestApkTask = "packageNormalDevDebugAndroidTest"
}

Could you please let me know what bit I am missing.

Many thanks in advance.

Release Shot 2.3.0

I want to use multiple device but it needs version 2.2.1-SNAPSHOT

Expected behaviour

We can use version 2.2.1-SNAPSHOT

Actual behaviour

The latest version still 2.2.0

Print feedback when a screenshot doesn't match

There are situations where we aren't able to access CI environments where tests are executed. That means that if a test fails we don't have any way to know what are the differences with the reference screenshot. The easiest fix for this sort of problems is to watch for failing tests and print a base64 of the new screenshot so that we can easily decode it in our machines and effectively see what changed. This feature should only run on CI environments and as such, Shot should provide an opt-in preference to enable it.

Start working on the error handling to be able to break a task execution when needed by returning an error modeled using Either and sealed traits.

Remember to check the configuration param and show errors when needed. We will also have to break the build if something goes wrong.

We will also have to break the build if adb is not installed or returns -1.

Running Across Multiple Devices

Expected behavior

Facebook Screenshot test supports running on multiple devices https://facebook.github.io/screenshot-tests-for-android/ with this flag
screenshots { // ... multipleDevices true }
But I don't see the same option when using Shot.

Actual behaviour

Shot has support for multipleDevices with gradle flags

Steps to reproduce

Version of the library

com.karumi:shot:2.0.0

Flavors support

Add support for applications using different flavors.

Configure HTML report to just show SS comparison for failed tests

Expected behaviour

Would be nice to provide a configurable property to allow users fetching the lib to just show screenshot comparisons under the HTML report for failed tests (since comparing the ones for green tests can be a bit irrelevant).

Actual behaviour

It's showing screenshot comparison for both, passing and non-passing tests.

Steps to reproduce

Run screenshot tests by command line and check the HTML report.

Version of the library

0.2.0

PD: The new HTML report is helpful! That's exactly the thing you want to have on CI for failing tests. Thanks for adding the feature.

Upgrade to newest Gradle Version

Expected behaviour

executeScreenshotTests should not return error with new gradle plugin.

Actual behaviour

I've updated my project to new android gradle plugin 3.2.1 and have updated gradle to version gradle-4.10.2. But, when I run executeScreenshotTests, the task exits with an error status.
I also get an error -

Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.

Steps to reproduce

I am running the task in a shell script and enclosed in a trap to catch any non zero exit status.

Version of the library

V2.1.1

Show fallback image when the reference screenshot is missing

Expected behaviour

Show a fallback image when there is no reference image to compare with in the HTML report.

Actual behaviour

When the reference image is missing we are showing a broken link in the HTML report.

Steps to reproduce

Create a new screenshot test in your application and run screenshot tests with shot by executing gradle executeScreenshotTests. The HTML report will show a broken link like the image below (ignore the gray rectangles 😅 ):

Version of the library

1.0.0

Refactor part of the system initialisation to use cake pattern.

No such file or directory error

⬇️ Pulling screenshots from your connected device!
rm: /sdcard/screenshots/nz.co.myappid.test/screenshots-default/: No such file or directory
Any Idea how to fix this?

Thanks

Shot doesn't work with API 23+ devices

Expected behaviour

Screenshot tests run on API 23+

Actual behaviour

        java.lang.RuntimeException: This does not currently work on API 23+, see https://github.com/facebook/screenshot-tests-for-android/issues/16 for details.
        at com.facebook.testing.screenshot.internal.ScreenshotDirectories.checkPermissions(ScreenshotDirectories.java:38)

Steps to reproduce

run ./gradlew executeScreenshotTests -Precord

Version of the library

1.0.0

Just a note, the facebook screenshot library now supports API 23+

facebook/screenshot-tests-for-android#89

*_dump.json (No such file or directory)

Expected behaviour

Complete executeScreenshotTests task tests without FileNotFoundException

Actual behaviour

Throws FileNotFoundException

Steps to reproduce

Add test to project and run executeScreenshotTests

Version of the library

3.0.2

Replace the HTML template engine.

The current implementation of the reporting system is using Freemarker to render the HTML template shown to the user as a result. As the support for Scala is not so good we need to migrate the templates to Scalate.

Trying to run test in a project with flavors fails. Main page documentation refers to non-existent version '3.0.0'

Expected behaviour

With a project set-up with multiple build flavors, when I correctly specify the instrumentationTestTask as such:

shot {
    appId = 'com.my.app'
    instrumentationTestTask = 'connectedDevDebugAndroidTest'
  }

The test should be able to pass.

Additionally, the documentation on the main page says:

Remember also you should use Shot 3.0.0 because this feature was introduced in this release!

However the latest released version is 2.2.0 (and as far as I can tell, the next planned is 2.3.0)

Actual behaviour

When running command
./gradlew executeScreenshotTests -Precord --stacktrace
I get the following error:

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':app:downloadScreenshots'.
> Task with path 'packageDebugAndroidTest' not found in project ':app'.

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.internal.tasks.TaskDependencyResolveException: Could not determine the dependencies of task ':app:downloadScreenshots'.
...

(I can provide full stack trace if necessary, but it doesn't seem helpful)

Steps to reproduce

As far as I can tell, just need a project with multiple build flavors to get this issue.

Version of the library

2.2.0