GithubHelp home page GithubHelp logo

cucumber-android's Introduction

Build

Cucumber-Android

This project implements Android support for Cucumber-JVM. It allows running cucumber tests with Android Test Orchestrator and using sharding.

NOTE: Although minSdkVersion for cucumber-android is 14 it requires Java 8 language features and minimum Android API level 26. This is done purposely to allow using cucumber in apps with lower minSdk (to avoid compile errors) but tests should be run on devices with API >= 26. However with desugaring enabled it may work in some configurations on lower API levels assuming that desugaring covers all the Java 8 api. Not all features from cucumber-jvm are supported in cucumber-android due to differences in Android vs JDK (especially junit and html plugins which requires xml factory classes not available in Android)

Developers

Prerequisites

This is ordinary multimodule Android project

  • cucumber-android - main library
  • cucumber-android-hilt - Hilt object factory
  • cucumber-junit-rules-support - internal module for Junit rules support
  • cukeulator - sample application with instrumented tests

Building

./gradlew assemble

Setting up the dependency

The first step is to include cucumber-android into your project, for example, as a Gradle androidTestImplementation dependency:

androidTestImplementation "io.cucumber:cucumber-android:$cucumberVersion"

Using Cucumber-Android

  1. Create a class in your testApplicationId package (usually it's a namespace from build.gradle with .test suffix) and add @CucumberOptions annotation to that class. You can also put such class in different package or have many such classes in different packages but then you have to provide path to it in instrumentation argument optionsAnnotationPackage.

Gradle example:

android {
    defaultConfig {
        testInstrumentationRunner "io.cucumber.android.runner.CucumberAndroidJUnitRunner"
        testInstrumentationRunnerArguments(optionsAnnotationPackage: "some.other.package")
    }
}

Commandline example:

adb shell am instrument -w -e optionsAnnotationPackage some.other.package com.mycompany.app.test/com.mycompany.app.test.MyTests

This class doesn't need to have anything in it, but you can also put some codes in it if you want. The purpose of doing this is to provide cucumber options. A simple example can be found in cukeulator. Or a more complicated example here:

package com.mycompany.app.test;

@CucumberOptions(glue = { "com.mytest.steps" }, tags = "~@wip" , features = { "features" })
public class MyTests 
{
}

glue is the list of packages which contain step definitions classes and also classes annotated with @WithJunitRule, tags is the tags placed above scenarios titles you want cucumber-android to run or not run, features is the path to the feature files in android test assets directory.

  1. Write your .feature files under your project's android test assets/<features-folder> folder. If you specify features = "features" in @CucumberOptions like the example above then it's androidTest/assets/features (might be also androidTest<Flavor/BuildType>/assets/features).

  2. Write your step definitions under the package name specified in glue. For example, if you specified glue = ["com.mytest.steps"], then create a new package under your androidTest/java (or androidTest/kotlin) named com.mytest.steps and put your step definitions under it. Note that all subpackages will also be included, so you can also put in com.mytest.steps.mycomponent.

  3. Set instrumentation runner to io.cucumber.android.runner.CucumberAndroidJUnitRunner or class that extends it

android.defaultConfig.testInstrumentationRunner "io.cucumber.android.runner.CucumberAndroidJUnitRunner"

If needed you can specify some cucumber options using instrumentation arguments. Check available options in io.cucumber.android.CucumberAndroidJUnitArguments.PublicArgs class

For example to specify glue package use:

android.defaultConfig.testInstrumentationRunnerArguments(glue: "com.mytest.steps")

Debugging

Please read the Android documentation on debugging.

Examples

Currently there is one example in subproject cukeulator

To create a virtual device and start an Android emulator:

$ANDROID_HOME/tools/android avd

Junit rules support

Experimental support for Junit rules was added in version 4.9.0. Cucumber works differently than junit - you cannot just add rule to some steps class because during scenario execution many such steps classes can be instantiated. Cucumber has its own Before/After mechanism. If you have just 1 steps class then this could work If you have many steps classes then it is better to separate rule and @Before/@After hooks
from steps classes

To let Cucumber discover that particular class has rules add

@WithJunitRule
class ClassWithRules {
    ...
}

and put this class in glue package. Glue packages are specified in @CucumberOptions annotation, see Using Cucumber-Android

You can specify tag expression like @WithJunitRule("@MyTag") to control for which scenarios this rule should be executed. See compose.feature and ComposeRuleHolder for example

Sharding and running with Android Test Orchestrator

CucumberAndroidJUnitRunner works with Android Test Orchestrator and sharding because it reports tests and classes as feature names and scenario names like My feature#My scenario and is able to parse -e class argument from instrumentation. It also supports multiple names in -e class argument separated by comma. This means that feature and scenario name cannot have comma in it's name because it is reserved for separating multiple names (only if you want to use Orchestrator or in general class argument, for other use cases comma is allowed).

Jetpack Compose rule

@WithJunitRule
class ComposeRuleHolder {

    @get:Rule
    val composeRule = createEmptyComposeRule()
}

then inject this object in steps, e.g. (can be also inject as lateinit var field (depending on injection framework used)

class KotlinSteps(val composeRuleHolder: ComposeRuleHolder, val scenarioHolder: ActivityScenarioHolder):SemanticsNodeInteractionsProvider by composeRuleHolder.composeRule {

    ...
    
    @Then("^\"([^\"]*)\" text is presented$")
    fun textIsPresented(arg0: String) {
        onNodeWithText(arg0).assertIsDisplayed()
    }
}

Check Junit rules support for more information of adding classes with JUnit rules

Hilt

There are 2 solutions for using Hilt with Cucumber:

1. HiltObjectFactory

Add dependency:

androidTestImplementation "io.cucumber:cucumber-android-hilt:$cucumberVersion"

Don't use any other dependency with ObjectFactory like cucumber-picocontainer

HiltObjectFactory will be automatically used as ObjectFactory.

To inject object managed by Hilt into steps or hook or any other class managed by Cucumber:

@HiltAndroidTest
class KotlinSteps(
    val composeRuleHolder: ComposeRuleHolder,
    val scenarioHolder: ActivityScenarioHolder
):SemanticsNodeInteractionsProvider by composeRuleHolder.composeRule {

    @Inject
    lateinit var greetingService:GreetingService

    @Then("I should see {string} on the display")
    fun I_should_see_s_on_the_display(s: String?) {
       Espresso.onView(withId(R.id.txt_calc_display)).check(ViewAssertions.matches(ViewMatchers.withText(s)))
    }

}

Such class:

  • must have @HiltAndroidTest annotation to let Hilt generate injecting code
  • can have Cucumber managed objects like hooks injected in constructor
  • can have Cucumber managed objects injected in fields but such objects have to be annotated with @Singleton annotation and constructor has to be annotated with @Inject annotation
  • can have Hilt managed objects injected using field injection or constructor
  • can have objects injected in base class

Also: after each scenario Hilt will clear all objects and create new ones (even these marked as @Singleton) (like it does for each test class in Junit)

2. @WithJunitRule

Hilt requires to have rule in actual test class (which for cucumber is impossible because there is no such class). To workaround that:

See https://developer.android.com/training/dependency-injection/hilt-testing#multiple-testrules how to use hilt with other rules (like compose rule)

@WithJunitRule(useAsTestClassInDescription = true)
@HiltAndroidTest
class HiltRuleHolder {

    @Rule(order = 0) 
    @JvmField
    val hiltRule = HiltAndroidRule(this)

   //if you need it to be injected   
    @Inject
    lateinit var greetingService: GreetingService

    @Before
    fun init() {
        //if you have anything to inject here and/or used elsewhere in tests    
        hiltRule.inject()
    }

}

then you can inject such class to steps class using Cucumber dependency injector (like picocontainer)

Running scenarios from IDE

There is third-party plugin (not related with Cucumber organisation and this repository) which allows running scenarios directly from Android Studio or Intellij

Cucumber for Kotlin and Android

Troubleshooting

  1. Compose tests fails

java.lang.IllegalStateException: Test not setup properly. Use a ComposeTestRule in your test to be able to interact with composables

Solution

Check Jetpack Compose rule section. Make sure that your class with @WithJunitRule annotation is placed in glue package as described in Using Cucumber-Android

cucumber-android's People

Contributors

andreasmarkussen avatar aslakhellesoy avatar aygalinc avatar brasmusson avatar cody1024d avatar fluxtah avatar friederbluemle avatar gnuechtel avatar hormesiel avatar k76154 avatar kirill-vlasov avatar lifedemons avatar lsuski avatar mfellner avatar mkochenough avatar mpkorstanje avatar nhajratw avatar rhavran avatar sierragolf avatar vacxe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cucumber-android's Issues

Jetifier Issues with cucumber-java dependency

for some circumstances .. in some cases we can't figure out what is different in setup we get

 > Could not resolve all artifacts for configuration ':app:debugAndroidTestCompileClasspath'.
    > Failed to transform artifact 'cucumber-java.jar (io.cucumber:cucumber-java:4.4.0)' to match attributes {artifactType=android-classes, org.gradle.libraryelements=jar, org.gradle.usage=java-runtime}.
       > Execution failed for JetifyTransform: /project/.gradle/caches/modules-2/files-2.1/io.cucumber/cucumber-java/4.4.0/f2a407a1159ca63271da836ca9c2b8db592bec9e/cucumber-java-4.4.0.jar.
          > Failed to transform '/project/.gradle/caches/modules-2/files-2.1/io.cucumber/cucumber-java/4.4.0/f2a407a1159ca63271da836ca9c2b8db592bec9e/cucumber-java-4.4.0.jar' using Jetifier. Reason: Malformed input or input contains unmappable characters: cucumber/api/java/kn/?????.class. (Run with --stacktrace for more details.)

for

android {
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    defaultConfig {
        applicationId "com.deutschebahn.bahnbonus"
        versionCode generateVersionCode(false)
        versionName generateVersionName()

        logger.info("using buildToolsVersion $buildToolsVersion defined by build plugin")

        minSdkVersion 21
        targetSdkVersion 28
        compileSdkVersion 28
    }
}
/// โ€ฆ
dependencies {
    androidTestImplementation "io.cucumber:cucumber-android:4.4.0"
}
android.useAndroidX=true
android.enableJetifier=true

any glue what is going wrong here? and yes, we need jetifier ;/

Is there a way to continue to next scenario when there is a crash

Is there a way to continue to next scenario when there is a crash. An example, I see the following error and the next few tests do not run after the process has crashed. Is there a way to work around the crash

INSTRUMENTATION_RESULT: shortMsg=Process crashed.
INSTRUMENTATION_CODE: 0

Unable to create instrumentation test configuration

Hello!

As far as I can see it is no longer possible to create an instrumentation test configuration. I am not able to create one in my application, so I tried to do the same in cucumber-android. The provided configuration works, but there seems to be no way to create the same configuration from scratch: Only 'android.test.InstrumentationTestRunner' is available and no selection of another Runner is possible (button is disabled).
Might be an AS problem, but I put it here first, maybe I miss something.

cucumber-android does not integrate very well with Android Orchestrator

From @mariusbob on February 15, 2018 9:54

I have just added android orchestrator in my project and it seems that the tests are running, the reports are generated, but I get this error:

com.android.builder.testing.ConnectedDevice > No tests found.[Moto G (4) - 7.0] FAILED 
No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack @Test annotations).

As I know, orchestrator takes every test method(annotated with @test, obviously) and executes it in a separate process. Is it really a problem for cucumber-android?

Copied from original issue: cucumber/cucumber-jvm#1324

Retry mechanism

I'd love to have a mechanism that retries failed tests but not with the rerun file mechanism.
I'm interested to have a report that is OK if at least one of the X attempts is OK (retry a test only when it fails indeed), and generate a unique report files. (Better again if the report can flag the flaky tests.)

Is something already available for this use case?

If no, I think about forking this repository to be able to implement my own AndroidFeatureRunner + RunNotifier, to be able to call child.run(myNotifier) as much as I need. Is it a good way to approach the problem?

Thanks for your time and this great tool!

cucumber.runtime.CucumberException: No CucumberOptions annotation

Hello, I'm trying to configure an automation project with this tool but I'm getting this error :C

cucumber.runtime.CucumberException: No CucumberOptions annotation
at io.cucumber.junit.AndroidJunitRuntimeOptionsFactory.createRuntimeOptions(AndroidJunitRuntimeOptionsFactory.java:55)
at io.cucumber.junit.CucumberJUnitRunner.<init>(CucumberJUnitRunner.java:83)
at io.cucumber.junit.CucumberJUnitRunnerBuilder.runnerForClass(CucumberJUnitRunnerBuilder.java:11)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at androidx.test.internal.runner.AndroidRunnerBuilder.runnerForClass(AndroidRunnerBuilder.java:147)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at androidx.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:73)
at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:104)
at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:793)
at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:547)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:390)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2189)

I have tried several things but i haven't been able to fix it. Maybe I'm doing something wrong but I don't know what it is. ยฟCould someone help me?

This is my app/build.gradle configuration:

android {

    defaultConfig {
        multiDexEnabled true
        applicationId "com.exampleโ€
        testApplicationId "com.example.test"
        testInstrumentationRunner "io.cucumber.android.runner.CucumberAndroidJUnitRunner"
    }
}

dependencies {

    //Cucumber
    androidTestImplementation "io.cucumber:cucumber-android:4.8.3"

    //UIAutomator
    androidTestImplementation 'androidx.test:rules:1.2.0'
    androidTestImplementation 'androidx.test.uiautomator:uiautomator:2.2.0'

    //OKHttp3
    implementation 'com.squareup.okhttp3:okhttp:3.14.1'

}

src/androidTest/assets/features/SearchUsers.feature:

Feature: Search users

    Background: Open App
      Given the user open the app

    Scenario Outline: Check error message
      When the user enters the ID number "<number>" and tap on search
      Then the message "<message>" is visible
      Examples:
        | number   | message                |
        | ABCDE    | The user doesn't exist.|
        | 792486201| The user doesn't exist |
        |          | Internal server error  |

Cucumber Options file:

package com.example.test;

import io.cucumber.junit.CucumberOptions;

@CucumberOptions(
        glue = "com.example.test.steps",
        plugin = {"pretty"},
        features = "assets/features")

public class Cucumber { }

And my step definitions:

package com.example.test.steps;

import androidx.test.uiautomator.UiObjectNotFoundException;
import com.example.test.pageObjects.IndexPage;
import org.json.JSONException;
import java.io.IOException;
import io.cucumber.java.en.*;

public class SearchSteps {

    private IndexPage indexPage = new IndexPage();

    @Given("the user open the app")
    public void openApp() {
        indexPage.openApp();
    }

    @When("the user enters the ID number {string} and tap on search")
    public void introduceCredentials(String number) throws UiObjectNotFoundException {
        indexPage.search(number);
    }

    @Then("the message {string} is visible")
    public void checkMessage(String message) {
        indexPage.checkMessage(message);
    }

}

I added steps in another file in the example project, but it failed to run.

Firstly, I successfully run the example test project, which passed all the test cases. so I began to make some modification to understand it.
I write a new file ChansonSteps.java based on the CalculatorActivitySteps.java, modified steps from "I have a CalculatorActivity" to "Chanson have a CalculatorActivity" and etc.
but it fail to run the test cases after the ChansonSteps.java added.
Here is the ChansonSteps.java I added

package cucumber.cukeulator.test;

import android.app.Activity;

import androidx.test.rule.ActivityTestRule;

import cucumber.api.java.After;
import cucumber.api.java.Before;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;
import cucumber.cukeulator.CalculatorActivity;
import cucumber.cukeulator.R;

import static androidx.test.espresso.Espresso.onView;
import static androidx.test.espresso.action.ViewActions.click;
import static androidx.test.espresso.assertion.ViewAssertions.matches;
import static androidx.test.espresso.matcher.ViewMatchers.withId;
import static androidx.test.espresso.matcher.ViewMatchers.withText;
import static org.junit.Assert.assertNotNull;

/**
 * We use {@link ActivityTestRule} in order to have access to methods like getActivity
 * and getInstrumentation.
 * </p>
 * The CucumberOptions annotation is mandatory for exactly one of the classes in the test project.
 * Only the first annotated class that is found will be used, others are ignored. If no class is
 * annotated, an exception is thrown.
 * <p/>
 * The options need to at least specify features = "features". Features must be placed inside
 * assets/features/ of the test project (or a subdirectory thereof).
 */

public class ChansonSteps {

    /**
     * Since {@link cucumber.runtime.android.CucumberJUnitRunner} and {@link cucumber.api.android.CucumberInstrumentationCore} have the control over the
     * test lifecycle, activity test rules must not be launched automatically. Automatic launching of test rules is only
     * feasible for JUnit tests. Fortunately, we are able to launch the activity in Cucumber's {@link Before} method.
     */
    ActivityTestRule rule = new ActivityTestRule<>(CalculatorActivity.class, false, false);

    public ChansonSteps(SomeDependency dependency) {
        assertNotNull(dependency);
    }

    /**
     * We launch the activity in Cucumber's {@link Before} hook.
     * See the notes above for the reasons why we are doing this.
     *
     * @throws Exception any possible Exception
     */
    @Before
    public void launchActivity() throws Exception {
        rule.launchActivity(null);
    }

    /**
     * All the clean up of application's data and state after each scenario must happen here
     */
    @After
    public void finishActivity() throws Exception {
        getActivity().finish();
    }

    /**
     * Gets the activity from our test rule.
     *
     * @return the activity
     */
    private Activity getActivity() {
        return rule.getActivity();
    }

    @Given("Chanson have a CalculatorActivity")
    public void Chanson_have_a_CalculatorActivity() {
        assertNotNull(getActivity());
    }

    @When("Chanson press {digit}")
    public void Chanson_press_d(final int d) {
        switch (d) {
            case 0:
                onView(withId(R.id.btn_d_0)).perform(click());
                break;
            case 1:
                onView(withId(R.id.btn_d_1)).perform(click());
                break;
            case 2:
                onView(withId(R.id.btn_d_2)).perform(click());
                break;
            case 3:
                onView(withId(R.id.btn_d_3)).perform(click());
                break;
            case 4:
                onView(withId(R.id.btn_d_4)).perform(click());
                break;
            case 5:
                onView(withId(R.id.btn_d_5)).perform(click());
                break;
            case 6:
                onView(withId(R.id.btn_d_6)).perform(click());
                break;
            case 7:
                onView(withId(R.id.btn_d_7)).perform(click());
                break;
            case 8:
                onView(withId(R.id.btn_d_8)).perform(click());
                break;
            case 9:
                onView(withId(R.id.btn_d_9)).perform(click());
                break;
        }
    }

    @When("Chanson press {operator}")
    public void Chanson_press_op(final char op) {
        switch (op) {
            case '+':
                onView(withId(R.id.btn_op_add)).perform(click());
                break;
            case 'โ€“':
                onView(withId(R.id.btn_op_subtract)).perform(click());
                break;
            case 'x':
                onView(withId(R.id.btn_op_multiply)).perform(click());
                break;
            case '/':
                onView(withId(R.id.btn_op_divide)).perform(click());
                break;
            case '=':
                onView(withId(R.id.btn_op_equals)).perform(click());
                break;
        }
    }

    @Then("Chanson should see {string} on the display")
    public void Chanson_should_see_s_on_the_display(final String s) {
        onView(withId(R.id.txt_calc_display)).check(matches(withText(s)));
    }
}

here is the error.

java.lang.NullPointerException: Attempt to invoke virtual method 'void android.app.Activity.finish()' on a null object reference
at cucumber.cukeulator.test.ChansonSteps.finishActivity(ChansonSteps.java:64)
at java.lang.reflect.Method.invoke(Native Method)
at cucumber.runtime.Utils$1.call(Utils.java:26)
at cucumber.runtime.Timeout.timeout(Timeout.java:16)
at cucumber.runtime.Utils.invoke(Utils.java:20)
at cucumber.runtime.java.JavaHookDefinition.execute(JavaHookDefinition.java:60)
at cucumber.runner.HookDefinitionMatch.runStep(HookDefinitionMatch.java:16)
at cucumber.runner.TestStep.executeStep(TestStep.java:65)
at cucumber.runner.TestStep.run(TestStep.java:50)
at cucumber.runner.TestCase.run(TestCase.java:50)
at cucumber.runner.Runner.runPickle(Runner.java:50)
at cucumber.runtime.junit.AndroidPickleRunner.run(AndroidPickleRunner.java:45)
at cucumber.runtime.junit.AndroidFeatureRunner.runChild(AndroidFeatureRunner.java:41)
at cucumber.runtime.junit.AndroidFeatureRunner.runChild(AndroidFeatureRunner.java:12)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at cucumber.runtime.android.CucumberJUnitRunner.runChild(CucumberJUnitRunner.java:266)
at cucumber.runtime.android.CucumberJUnitRunner.runChild(CucumberJUnitRunner.java:71)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at cucumber.runtime.android.CucumberJUnitRunner$2.evaluate(CucumberJUnitRunner.java:324)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:392)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2119)

so, I want to figure out

  1. what's wrong with my code ? and
    2 where is the code to set the steps file CalculatorActivitySteps.java in the CukeulatorAndroidJUnitRunner.java

Thanks for anyone who might help.

No CucumberOptions Annotation

I ecountered into this error when I wanted to run my test in several build flavors, while in the other build flavors I didn't have this error.

cucumber.runtime.CucumberException: No CucumberOptions annotation at cucumber.runtime.android.CucumberJUnitRunner.createRuntimeOptions(CucumberJUnitRunner.java:300)
at cucumber.runtime.android.CucumberJUnitRunner.(CucumberJUnitRunner.java:102)
at cucumber.runtime.android.CucumberJUnitRunnerBuilder.runnerForClass(CucumberJUnitRunnerBuilder.java:12)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at androidx.test.internal.runner.AndroidRunnerBuilder.runnerForClass(AndroidRunnerBuilder.java:147)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at androidx.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:73)
at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:104)
at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:793)
at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:547)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:390)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2145)

Can you guys please help me

Cucumber tests parallel execution with Testlab

Problem:
I am running Cucumber UI tests with Firebase Testlab. All are working fine. The sequential run is successful and getting all reports.
Currently, I am trying to distribute the tests to run it on Testlab parallelly But seems like it is failing to find the tests.
I am getting some log which says 0 test cases found

0 test / 1 shard
1 matrix ids created in 0m 10s

It is actually run all the cases sequentially on the 1 shard created. But no parallel runs.
My Gcloud configuration looks like this

gcloud:
  # test and app are the only required args
  app: .apkpath
  test: .testapkPath
  auto-google-login: false
  use-orchestrator: true
  instrumentation: true
  environment-variables:
    coverage: true
    coverageFilePath: /sdcard/
    clearPackageData: true
  directories-to-pull:
    - /sdcard/
  performance-metrics: false
  num-flaky-test-attempts: 0
  device:
    - model: "NexusLowRes"
      version: 29

  results-bucket: bucket_name
  results-dir: results
  locale: en
  test-runner-class: CucumberInstrumentationRunner
  record-video: true
  timeout: 30m
  num-uniform-shards: 8 //Not working

Cucumber configuration:

@CucumberOptions(
        features = "features",
        glue = "com.test",
        tags = {"@mySevenTests"},
        monochrome = true)
class Hooks {
    @Rule
    var activityTestRule = ActivityTestRule(
            InitialActivity::class.java,
            shouldLaunchActivity = true,
            isInitialTouchMode = true)
            .with(MockWebServerExecutorTestTask())

    private var activity: Activity? = null

    @Before("@mySevenTests")
    fun setup(scenario: Scenario) {
            activityTestRule.launchActivity(null)
            activity = activityTestRule.activity
        }

    @After("@mySevenTests")
    fun tearDown(){
        activity?.finish()
    }
}

Cucumber Android version 4.8.4

Can someone please help on this.

CucumberAndroidJUnitRunner isn't a test class

I am unable to get cucumber running through android studio now with the androidx upgrade. I am using the same Runner as in cukelator. Mine is written in kotlin:

import android.os.Bundle
import cucumber.api.CucumberOptions
import cucumber.api.android.CucumberAndroidJUnitRunner
import java.io.File

@CucumberOptions(features = ["features"], tags=["~@wip"]])
class CucumberRunner : CucumberAndroidJUnitRunner() {

override fun onCreate(bundle: Bundle) {
bundle.putString("plugin", getPluginConfigurationString()) // we programmatically create the plugin configuration
super.onCreate(bundle)
}

/**

  • Since we want to checkout the external storage directory programmatically, we create the plugin configuration
  • here, instead of the [CucumberOptions] annotation.
  • @return the plugin string for the configuration, which contains XML, HTML and JSON paths
    */
    private fun getPluginConfigurationString(): String {
    val cucumber = "cucumber"
    val separator = "--"
    return "junit:${getAbsoluteFilesPath()}/$cucumber.xml${separator}html:${getAbsoluteFilesPath()}/" +
    "$cucumber.html${separator}json:${getAbsoluteFilesPath()}/$cucumber.json"
    }

/**

  • The path which is used for the report files.
  • @return the absolute path for the report files
    */
    private fun getAbsoluteFilesPath(): String {
    // Since stupidly, connected-check tasks uninstall the applications,
    // we have to find a directory outside the application directory.
    val directory = targetContext.getExternalFilesDir(null)
    return File(directory, "reports").absolutePath
    }
    }

Version 4.24 Reports and Rerun require TestRunFinished Event

TL;DR - In 4.2.4 TestRunFinished Event is not posted on the EventBus which breaks reporting and rerun features.

Longer Explanation: Previous to 4.2.2 CucumberExecutor posted TestRunFinished during the execute method. In 4.2.4 CucumberJunitRunner is not posting that event on the EventBus anymore.

I propose the following change to fix this Bug. In CucumberJunitRunner similar to Cucumber class from core I would override childrenInvoker like this

   @Override
    protected Statement childrenInvoker(RunNotifier notifier) {
        final Statement features = super.childrenInvoker(notifier);
        return new Statement() {
            @Override
            public void evaluate() throws Throwable {
                features.evaluate();
                bus.send(new TestRunFinished(bus.getTime()));
            }
        };
    }

This change also requires to make EventBus bus a class wide variable

private EventBus bus;

That can as previously be initialized during constructor

ResourceLoader resourceLoader = new AndroidResourceLoader(context);

bus = new TimeServiceEventBus(TimeService.SYSTEM);
Plugins plugins = new Plugins(classLoader, new PluginFactory(), bus, runtimeOptions);

Please let me know what you think about my suggestion or if I can help in creating a Pull Request for this issue.

Running features not found in resources

I'd like to be able to source my feature files externally (the use case is not your typical Android application - it's a custom device and test automation is running as part of some end-user scenarios).

Having .feature files on the local filesystem would work (but a mechanism that would allow me to supply gherkin scripts from a source of my choosing would be better). I've tried running "am instrument -e features [path.to.file]" but it looks like cucumber-android is still looking for a path within resources, not the filesystem.

Curious if this is possible with the current implementation?

test runner must be placed under `test` dir if it's the suffix of application id

If I had the application ids as following

  defaultConfig {
    applicationId "io.abdelraouf.learn.cucumber"
    testApplicationId "io.abdelraouf.learn.cucumber.test"
}

then the library expects the runner to be under the package io.abdelraouf.learn.cucumber.test and I need to set the

testInstrumentationRunner "io.abdelraouf.learn.cucumber.test.runner.AndroidCucumberTestRunner"

Is that normal ?

issue canot run test example

Testing started at 5:19 PM ...

10/10 17:19:43: Launching Cucumber
$ adb push /Users/apple/Downloads/cucumber-android-master/cukeulator/build/outputs/apk/debug/cukeulator-debug.apk /data/local/tmp/cucumber.cukeulator
$ adb shell pm install -t -r "/data/local/tmp/cucumber.cukeulator"
Success
APK installed in 7 s 871 ms
$ adb push /Users/apple/Downloads/cucumber-android-master/cukeulator/build/outputs/apk/androidTest/debug/cukeulator-debug-androidTest.apk /data/local/tmp/cucumber.cukeulator.test
$ adb shell pm install -t -r "/data/local/tmp/cucumber.cukeulator.test"
Success
APK installed in 9 s 451 ms
Running tests

$ adb shell am instrument -w -r -e debug false cucumber.cukeulator.test/cucumber.cukeulator.test.CucumberRunner
Client not ready yet..
Started running tests
Test running failed: Instrumentation run failed due to 'Process crashed.'
Empty test suite.

Android Example : Failed to download cucumber report

From @aygalinc on June 14, 2018 16:1

I simply download and run the android studio example and the gradle task downloadCucumberReports failed because the specified files not exist. I check with the device file explorer.

I think that you have to uncomment in the CucumberRunner in the onCreate method the plugin configuration. It will allowed the creation of the several files and so on their downloading.

Moreover, i have a side question : You provide a debug Manifest.xml with WRITE_EXTERNAL_STORAGE and READ_EXTERNAL_STORAGE so the gradle task allowedgrantedpermession is redundant with this or not ? (because it raises some warning during compile time)

Copied from original issue: cucumber/cucumber-jvm#1396

Running tests on robolectric and actual device

Hello, Thanks for the great library. Yet I have a question.

Now we can run android tests using robolectric on JVM, Can I run cucumber tests using robolectric?

I have the following structure: build.gradle.kts

sourceSets.all {
    java.srcDirs("src/$name/kotlin")
    when (name) {
        "test", "androidTest" -> {
            java.srcDirs("src/sharedTest/kotlin")
            assets.srcDirs("src/sharedTest/assets")
        }
    }
}

So I've sharedTest folder available to jvm and androidTest. Evantually, I'll put cucumber feature files in src/sharedTest/assets/features and step defs inside src/sharedTest/kotlin. As those will be shared between both jvm and androidTest.

My question how to run tests on both robolectric and actual device?

Run different suites

Hello. I want to have a few test's suites (Regression, smoke, etc) Also our application has several flavours and buildTypes.
Is it possible to setup cucumber for this purpose with this type of application?

Support for java8 API?

Hi, is it possible to write java8-style step definitions with cucumber-android? It's helpful especially with Kotlin, it's almost like DSL. I tried to add java8 dependencies, but the build failed on not finding cucumber on classpath.

java.lang.SecurityException: Calling from not trusted UID!

I am getting crash with message "java.lang.SecurityException: Calling from not trusted UID!" while running with following command at the end of test execution.
"gradle connectedAndroidTest -Ptags="@TagName""

I am working on BDD automation with the following project structure.

androidTest -> assets
java

In "assets" folder we have feature file and Golden Images.
In "java" folder we have Stepdefintion files.
And One Instrumentation Class with below structure

@CucumberOptions(features = "features", glue = "com.android.automation")
public class Instrumentation extends MonitoringInstrumentation {

}

cucumber.runtime.android.MissingStepDefinitionError

cucumber.runtime.android.MissingStepDefinitionError:

@then("^We see an activity Screen$")
public void we_see_an_activity_Screen() throws Throwable {
// Write code here that turns the phrase above into concrete actions
throw new PendingException();
}

at cucumber.runtime.android.AndroidInstrumentationReporter.endOfScenarioLifeCycle(AndroidInstrumentationReporter.java:153)
at java.lang.reflect.Method.invoke(Native Method)
at cucumber.runtime.Utils$1.call(Utils.java:40)
at cucumber.runtime.Timeout.timeout(Timeout.java:16)
at cucumber.runtime.Utils.invoke(Utils.java:34)
at cucumber.runtime.RuntimeOptions$1.invoke(RuntimeOptions.java:294)
at java.lang.reflect.Proxy.invoke(Proxy.java:1006)
at $Proxy3.endOfScenarioLifeCycle(Unknown Source)
at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:47)
at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
at cucumber.runtime.android.CucumberExecutor.execute(CucumberExecutor.java:112)
at cucumber.api.android.CucumberInstrumentationCore.start(CucumberInstrumentationCore.java:88)
at spaceinvaders.test.Instrumentation.onStart(Instrumentation.java:35)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2337)

Feature file:
Feature: UH from SpaceInvaders application

Background:
Given We open the app

@UserStories-feature
Scenario: Open a game
When We start the game
Then We see an activity Screen

steps definition:
public class UserStoriesDef {

private MainActivity mainActivity;
private GameView view;
private int fleetSize;
private int points;


@Rule
public ActivityTestRule<MainActivity> activityTestRule = new ActivityTestRule<>(MainActivity.class);

private Activity activity;

@Before("@UserStories-feature")
public void setup() {
    activityTestRule.launchActivity(new Intent());
    activity = activityTestRule.getActivity();
}

@After("@UserStories-feature")
public void tearDown() {
    activityTestRule.finishActivity();
}

@Given("^We open the app$")
public void weOpenTheApp() {
    mainActivity = new MainActivity();
    view = mainActivity.getGameView();
}

@When("^We start the game$")
public void weStartTheGame() {
    onView(isDisplayed()).perform(click());
    fleetSize = view.getInvaderSpaceFleet().getInvaderSpaceShips().size();
    points = view.getScoreValue();
}

@Then("^We see an activity Screen$")
public void weSeeAnActivityScreen() {
    int state = view.getGameState();
    assertEquals(2, state);
}

run cucumber with:
@CucumberOptions(
features = "features",
glue = "spaceinvaders.test")
public class Instrumentation extends MonitoringInstrumentation {

private final CucumberInstrumentationCore instrumentationCore = new CucumberInstrumentationCore(this);

@Override
public void onCreate(Bundle arguments) {
    super.onCreate(arguments);

    String tags = BuildConfig.TEST_TAGS;
    if (!tags.isEmpty()) {
        arguments.putString("tags", tags.replaceAll(",", "--").replaceAll("\\s",""));
    }


    instrumentationCore.create(arguments);
    start();
}

@Override
public void onStart() {
    super.onStart();
    waitForIdleSync();
    instrumentationCore.start();
}

}

Why does it just mark that step as not done?

Cucumber Android not running anymore on R / 30 / Android 11

while testing migration to targetLevel 30 we faced issues running cucumber androidTests on emulator with 30 / android 11.

While our main project just tells us "no tests found"
=>
on console :connectedVariantAndroidTest with -e debug true

com.android.build.gradle.internal.testing.ConnectedDevice > No tests found.[Pixel_4_API_30(AVD) - 11] FAILED 
No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack @Test annotations).

we have a CucumberAndroidJUnitRunner as testInstrumentationRunner
with an

@CucumberOptions(
    monochrome = true,
    features = ["features"],
    glue = ["com.example.uitests"],
    tags = ["@fast"]
)
class RunCucumberTest {}

cuculator fails with


cucumber.runtime.CucumberException: java.io.IOException: Failed to create directory /reports
	at cucumber.runtime.formatter.PluginFactory.newInstance(PluginFactory.java:110)
	at cucumber.runtime.formatter.PluginFactory.instantiate(PluginFactory.java:91)
	at cucumber.runtime.formatter.PluginFactory.create(PluginFactory.java:76)
	at cucumber.runtime.formatter.Plugins.createPlugins(Plugins.java:63)
	at cucumber.runtime.formatter.Plugins.<init>(Plugins.java:35)
	at io.cucumber.junit.CucumberJUnitRunner.<init>(CucumberJUnitRunner.java:96)
	at io.cucumber.junit.CucumberJUnitRunnerBuilder.runnerForClass(CucumberJUnitRunnerBuilder.java:11)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at androidx.test.internal.runner.AndroidRunnerBuilder.runnerForClass(AndroidRunnerBuilder.java:147)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at androidx.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:73)
	at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:104)
	at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:793)
	at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:547)
	at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:390)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2205)
Caused by: java.io.IOException: Failed to create directory /reports
	at cucumber.runtime.formatter.URLOutputStream.ensureParentDirExists(URLOutputStream.java:53)
	at cucumber.runtime.formatter.URLOutputStream.<init>(URLOutputStream.java:33)
	at cucumber.runtime.formatter.URLOutputStream.<init>(URLOutputStream.java:24)
	at cucumber.runtime.formatter.JUnitFormatter.<init>(JUnitFormatter.java:87)
	at java.lang.reflect.Constructor.newInstance0(Native Method)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
	at cucumber.runtime.formatter.PluginFactory.newInstance(PluginFactory.java:104)
	... 15 more

both projects working like a charm on Q/29/android10

  • macOs
  • SystemImage 30 r6
  • Emulator 30.0.14
  • SDK PlatformTools 30.0.3
  • ABI x86 or x86_64
  • targetVariant "Google Play"

No tests found. when running gradle app:connectedCheck

I'm trying to setup cucumber-android but when running gradle app:connectedCheck the cucumber tests are not recognized. Any ideas on what's wrong/what I'm missing are welcome

build.gradle

testInstrumentationRunner "cucumber.api.android.CucumberAndroidJUnitRunner"
//dependencies
androidTestImplementation "io.cucumber:cucumber-android:4.3.1"
androidTestImplementation "io.cucumber:cucumber-picocontainer:4.3.1"

TestRunner

import cucumber.api.CucumberOptions

@CucumberOptions(
        glue = ["com.leodeleon.bootstrap.test.steps"],
        plugin = [ "junit:/data/data/com.leodeleon.bootstrap/JUnitReport.xml", "json:/data/data/com.leodeleon.bootstrap/JSONReport.json" ],
        features = ["features"])
class TestRunner{

}

Test

import cucumber.api.java.en.Given
import cucumber.api.java.en.Then

class TestSteps {
    private val robot = TestRobot()

    @Given("I open app")
    fun openApp(){
    }

    @Then("I should see the button")
    fun checkButton() {
        with(robot){
            verifyThat { button1IsShowing() }
        }
    }
}

Error

01:50:59 V/ddms: execute: running am instrument -w -r   com.leodeleon.bootstrap.test/cucumber.api.android.CucumberAndroidJUnitRunner
01:51:01 V/InstrumentationResultParser: INSTRUMENTATION_RESULT: stream=
01:51:01 V/InstrumentationResultParser: 
01:51:01 V/InstrumentationResultParser: Time: 0.001
01:51:01 V/InstrumentationResultParser: 
01:51:01 V/InstrumentationResultParser: OK (0 tests)
01:51:01 V/InstrumentationResultParser: 
01:51:01 V/InstrumentationResultParser: 
01:51:01 V/InstrumentationResultParser: INSTRUMENTATION_CODE: -1
01:51:01 V/InstrumentationResultParser: 
Starting 0 tests on Pixel XL - 10
01:51:01 I/XmlResultReporter: XML test result file generated at /Users/leo/Projects/Bootstrap/app/build/outputs/androidTest-results/connected/TEST-Pixel XL - 10-app-.xml. Total tests 0, 
01:51:01 V/ddms: execute 'am instrument -w -r   com.leodeleon.bootstrap.test/cucumber.api.android.CucumberAndroidJUnitRunner' on 'HT69N0202751' : EOF hit. Read: -1
01:51:01 V/ddms: execute: returning

com.android.builder.testing.ConnectedDevice > No tests found.[Pixel XL - 10] FAILED 
No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack @Test annotations).
01:51:01 I/XmlResultReporter: XML test result file generated at /Users/leo/Projects/Bootstrap/app/build/outputs/androidTest-results/connected/TEST-Pixel XL - 10-app-.xml. Total tests 1, failure 1, 

Empty test suite: can't get it to working

Firstly, I want to thank you all for your great effort, But I come with sad news, at least for me.

I can't get the lib to be working ever on android.

Here is my project: repository link

Note: The sample project is very tightly coupled with the deps of the library, please can you make it depend only on the library as if you are an end user? like me ?

Testing started at 11:17 PM ...

03/10 23:17:36: Launching Test
No apk changes detected since last installation, skipping installation of /home/abdelraouf/AndroidStudioProjects/cucumber/app/build/outputs/apk/debug/app-debug.apk
$ adb push /home/abdelraouf/AndroidStudioProjects/cucumber/app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk /data/local/tmp/io.abdelraouf.learn.cucumber.test
$ adb shell pm install -t -r "/data/local/tmp/io.abdelraouf.learn.cucumber.test"
Success
APK installed in 414 ms
No apk changes detected since last installation, skipping installation of /mnt/Data/Development/.gradle/caches/modules-2/files-2.1/androidx.test/orchestrator/1.1.1/7553b6b0848f4ac80a15520d82df2e261f156091/orchestrator-1.1.1.apk
No apk changes detected since last installation, skipping installation of /mnt/Data/Development/.gradle/caches/modules-2/files-2.1/androidx.test.services/test-services/1.1.1/9dd8143d514bccd46ba59db1011d8a8eedf47a3b/test-services-1.1.1.apk
Running tests

$ adb shell am instrument -w -r   -e debug false -e class 'io.abdelraouf.learn.cucumber.test.AndroidCucumberTestRunner' io.abdelraouf.learn.cucumber.test/cucumber.api.android.CucumberAndroidJUnitRunner
Client not ready yet..
Started running tests
Tests ran to completion.


Empty test suite.

Report not generated

as mentioned in readme Used
"format = {"junit:/data/data/com.mytest/JUnitReport.xml", "json:/data/data/com.mytest/JSONReport.json"}" also tried
plugin = {"junit:/data/data/com.mytest/JUnitReport.xml", "json:/data/data/com.mytest/JSONReport.json"}

none of these generated reports, can you please elaborate more on generating reports?

Running both Cucumber tests and standard Android tests

My problem is that when I use CucumberAndroidJUnitRunner I'm not able to run the Android instrumented tests and the cucumber ones, only the cucumber tests are running. Even with CukeulatorAndroidJUnitRunner. I check PR #39 and issue #43 but still dont working.

Reports are getting deleted before the gradle tasks can pull the reports.

i am running cucumber tests in android and trying to store the report in the externalDirectory which seems to be "/storage/emulated/0/Android/data//files". During the test execution i can see the reports getting generated. But the reports are deleted when the app gets un-installed.
Can anyone please help me copy the reports to the desktop before they get deleted.

i have tried to store the reports in a couple of other locations such as "/sdcard", "/mnt/sdcard" , "/storage/emulated/0" . I haven't had luck with any of these. i get IOException. seems i don't have permissions to write to these locations.

Reported duration time of scenario is about 0ms on Android

From @lsuski on July 5, 2018 18:7

Hi,
I've recently build locally cucumber 4.0.0-SNAPSHOT and tried it on Android and encounter issue with scenario duration time. For most of scenarios it is about 0ms or 25ms or 75ms so generally it is very short but test takes naturally longer. With Cucumber 2.4.0 execution time is normal (average 2s per scenario). I was digging whole day in code base but could not find any reason for that.
Was there any change between 2.4.0 and 4.0.0-SNAPSHOT which could introduce such issue on Android?

Copied from original issue: cucumber/cucumber-jvm#1413

cucumber.runtime.CucumberException: No CucumberOptions annotation thrown when @CucumberOptions provided

I am getting this error when trying to run cucumber tests using cucumber-android version 4.3.1:
cucumber.runtime.CucumberException: No CucumberOptions annotation at cucumber.runtime.android.CucumberJUnitRunner.createRuntimeOptions(CucumberJUnitRunner.java:297) at cucumber.runtime.android.CucumberJUnitRunner.<init>(CucumberJUnitRunner.java:102) at cucumber.runtime.android.CucumberJUnitRunnerBuilder.runnerForClass(CucumberJUnitRunnerBuilder.java:12) at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59) at androidx.test.internal.runner.AndroidRunnerBuilder.runnerForClass(AndroidRunnerBuilder.java:147) at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59) at androidx.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:73) at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:104) at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:793) at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:547) at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:390) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2189)

In my app/gradle.build file I have these set:

testApplicationId "com.example.acceptancetestingexample.test" testInstrumentationRunner "cucumber.api.android.CucumberAndroidJUnitRunner"

And I have this class defined for the @CucumberOptions:

@CucumberOptions( glue = ["com.example.acceptancetestingexample.test.LoginDetailsSteps"], features = ["/AcceptanceTestingExample/app/src/androidTest/resources"], plugin = ["pretty"]) class RunAcceptanceTests

Retry Mechanism For Failed Tests For 'N' times

Do we have a mechanism to rerun the failed tests 'n' times. As I can see the re-run by file mechanism cannot be implemented since I can see a comment from the cuckulater example

/**
 * The CucumberOptions annotation is mandatory for exactly one of the classes in the test project.
 * Only the first annotated class that is found will be used, others are ignored. If no class is
 * annotated, an exception is thrown. This annotation does not have to placed in runner class
 */

How can we implement a retry mechanism? Any advice/ docs?
Thanks

Failed to read resource:file + filePath

I'm trying to integrate Cucumber to my existing project with existing tests.

I keep following some tutorials from medium and your tutorial as well. I'm able to run the sample projects successfully but when I try to apply to my project, it keeps saying some error.

I tried to make some tricks but still doesn't work, if you guys can tell me what did I do wrong, so much appreciated.

My setup:

app.gradle

defaultConfig {
        testInstrumentationRunner "cucumber.api.android.CucumberAndroidJUnitRunner"
    }
androidTest {
            java {
                srcDirs = ['tests/instrumentationTest']
                resources.srcDirs = ['tests/instrumentationTest/assets']
            }
            manifest.srcFile file('tests/instrumentationTest/AndroidManifest.xml')
        }

My Feature file

image

Test Folder

image

Test Class
image

The error log I have:

image

Then I changed the @CucumberOptions to

image

And I got this log:
image

I have tried so many ways for finding a workaround solution but no result at all.
Any suggestion would be much appreciated.
Thank you.

issue of running sh cmd with pipe

it only execute the first cmd when to run sh cmd with pipe.

example:
device.executeShellCommand("ls | grep xxxx"), cmd "ls" can run successfully, but "grep xxx" not

Add dependency description in README

Maybe a section in README file where dependency management of cucumber-android
(just a line:
// Use the stable Cucumber android version in androidTest
androidTestImplementation 'io.cucumber:cucumber-android:3.0.2') is described
would be nice in order to avoid confusion with older info.cukes:cucumber-android dependency.

Calling from Untrusted UID Error

I am using ```
androidTestImplementation 'com.squareup.spoon:spoon-client:1.7.1'

androidTestImplementation 'androidx.test.uiautomator:uiautomator:2.2.0'

androidTestImplementation 'io.cucumber:cucumber-android:4.4.1'
androidTestImplementation 'io.cucumber:cucumber-picocontainer:4.4.0'

Even with those packages, I am getting

06-04 22:28:41.023   915  2699 W ActivityManager: Crash of app <Package> running instrumentation ComponentInfo{<Package>.test/<Package>.test.cucumber.runner.CucumberTestRunner}
06-04 22:28:41.026  5097  7077 W Binder  : Binder call failed.
06-04 22:28:41.026  5097  7077 W Binder  : java.lang.SecurityException: Calling from not trusted UID!
06-04 22:28:41.026  5097  7077 W Binder  : 	at android.app.UiAutomationConnection.throwIfCalledByNotTrustedUidLocked(UiAutomationConnection.java:468)
06-04 22:28:41.026  5097  7077 W Binder  : 	at android.app.UiAutomationConnection.shutdown(UiAutomationConnection.java:372)
06-04 22:28:41.026  5097  7077 W Binder  : 	at android.app.IUiAutomationConnection$Stub.onTransact(IUiAutomationConnection.java:222)
06-04 22:28:41.026  5097  7077 W Binder  : 	at android.os.Binder.execTransact(Binder.java:731)

IllegalArgumentException: "Unknown pattern character 'X'" on API Level < 24

When starting instrumented tests on devices with API < 24 an IllegalArgumentException gets thrown. As described in https://stackoverflow.com/questions/44588429/unknown-pattern-character-x-when-using-simpledateformat/44588864, the format specifier 'X' is available only from API Level >= 24

build.gradle dependencies:

  • cucumber-android 4.4.0
  • cucumber-java8

Stacktrace:

E/TestRunner: java.lang.IllegalArgumentException: Unknown pattern character 'X' at java.text.SimpleDateFormat.validatePatternCharacter(SimpleDateFormat.java:323) at java.text.SimpleDateFormat.validatePattern(SimpleDateFormat.java:312) at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:365) at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:258) at cucumber.runtime.formatter.JSONFormatter.getDateTimeFromTimeStamp(JSONFormatter.java:392) at cucumber.runtime.formatter.JSONFormatter.createTestCase(JSONFormatter.java:197) at cucumber.runtime.formatter.JSONFormatter.handleTestCaseStarted(JSONFormatter.java:125) at cucumber.runtime.formatter.JSONFormatter.access$100(JSONFormatter.java:42) at cucumber.runtime.formatter.JSONFormatter$2.receive(JSONFormatter.java:64) at cucumber.runtime.formatter.JSONFormatter$2.receive(JSONFormatter.java:61) at cucumber.runner.AbstractEventPublisher.send(AbstractEventPublisher.java:45) at cucumber.runner.AbstractEventBus.send(AbstractEventBus.java:9) at cucumber.runner.TimeServiceEventBus.send(TimeServiceEventBus.java:3) at cucumber.runner.ThreadLocalRunnerSupplier$SynchronizedEventBus.send(ThreadLocalRunnerSupplier.java:93) at cucumber.runner.ThreadLocalRunnerSupplier$LocalEventBus.send(ThreadLocalRunnerSupplier.java:61) at cucumber.runner.TestCase.run(TestCase.java:38) at cucumber.runner.Runner.runPickle(Runner.java:50) at cucumber.runtime.junit.AndroidPickleRunner.run(AndroidPickleRunner.java:45) at cucumber.runtime.junit.AndroidFeatureRunner.runChild(AndroidFeatureRunner.java:41) at cucumber.runtime.junit.AndroidFeatureRunner.runChild(AndroidFeatureRunner.java:12) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at cucumber.runtime.android.CucumberJUnitRunner.runChild(CucumberJUnitRunner.java:266) at cucumber.runtime.android.CucumberJUnitRunner.runChild(CucumberJUnitRunner.java:71) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at cucumber.runtime.android.CucumberJUnitRunner$2.evaluate(CucumberJUnitRunner.java:324) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56) at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:392) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1879) ----- end exception -----

Cucumber sample failure on Android 4

The sample works fine on newer devices.

But on Android 4, it fails with this error:

java.lang.NoClassDefFoundError: java.util.Objects at io.cucumber.core.model.GluePath.parse(GluePath.java:32) at cucumber.runtime.android.CucumberJUnitRunner.createRuntimeOptions(CucumberJUnitRunner.java:286) at cucumber.runtime.android.CucumberJUnitRunner.<init>(CucumberJUnitRunner.java:100) at cucumber.runtime.android.CucumberJUnitRunnerBuilder.runnerForClass(CucumberJUnitRunnerBuilder.java:12) at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59) at androidx.test.internal.runner.AndroidRunnerBuilder.runnerForClass(AndroidRunnerBuilder.java:147) at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59) at androidx.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:73) at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:104) at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:789) at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:543) at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:386) at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1584)

initializationError with running standart tests

When i try to run standart instrumental test via ./gradlew connectedMainDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.cucumberUseAndroidJUnitRunner=true i see fail

        java.lang.IllegalArgumentException: Not a file or directory: /features
        at cucumber.runtime.io.FileResourceIterator$FileIterator.<init>(FileResourceIterator.java:63)

> Task :app:connectedMainDebugAndroidTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:connectedMainDebugAndroidTest'.
> There were failing tests. See the report at: file:///home/x1opya/AndroidProjects/yard/app/build/reports/androidTests/connected/flavors/mainDebugAndroidTest/index.html

image

initializationError
java.lang.IllegalArgumentException: Not a file or directory: /features
at cucumber.runtime.io.FileResourceIterator$FileIterator.<init>(FileResourceIterator.java:63)
at cucumber.runtime.io.FileResourceIterator.<init>(FileResourceIterator.java:28)
at cucumber.runtime.io.FileResourceIterator.createFileResourceIterator(FileResourceIterator.java:14)
at cucumber.runtime.io.FileResourceIterable.iterator(FileResourceIterable.java:19)
at cucumber.runtime.model.FeatureLoader.loadFromFeaturePath(FeatureLoader.java:31)
at cucumber.runtime.model.FeatureLoader.load(FeatureLoader.java:23)
at cucumber.runtime.FeaturePathFeatureSupplier.get(FeaturePathFeatureSupplier.java:33)
at io.cucumber.junit.Cucumber.<init>(Cucumber.java:115)
at java.lang.reflect.Constructor.newInstance0(Native Method)
at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:104)
at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
at androidx.test.internal.runner.junit4.AndroidAnnotatedBuilder.runnerForClass(AndroidAnnotatedBuilder.java:63)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:37)
at androidx.test.internal.runner.AndroidRunnerBuilder.runnerForClass(AndroidRunnerBuilder.java:153)
at androidx.test.internal.runner.TestLoader$ScanningRunnerBuilder.runnerForClass(TestLoader.java:144)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at androidx.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:73)
at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:105)
at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:804)
at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:575)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:393)
at `android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2189)```

Running both Cucumber tests and standard Junit tests

I'm adding Cucumber to a module that already has instrumented tests (I have many modules and each of them has several instrumented tests).
My problem is that when I use CucumberAndroidJUnitRunner I'm not able to run the existing instrumented tests (they are ignored) unless I switch back to the previous runner (in which case the cucumber tests are ignored).

Is this the expected behaviour or I'm doing something wrong?
If this is the expected behaviour which workaround can I use?

cucumber.runtime.formatter.MissingStepDefinitionError when multiDexEnabled is set to true

Summary

I am using cucumber 4.0.0 with android instrumented tests and it worked fine till now but when I set
multiDexEnabled to true in my build.gradle I start getting below run time error. I have only enable multidex to true and haven't added other dependencies.

cucumber.runtime.formatter.MissingStepDefinitionError:

@then("AC :I take another screen shot")
public void ac_I_take_another_screen_shot() {
// Write code here that turns the phrase above into concrete actions
throw new PendingException();
}

at cucumber.runtime.formatter.AndroidInstrumentationReporter.finishTestCase(AndroidInstrumentationReporter.java:213)
at cucumber.runtime.formatter.AndroidInstrumentationReporter$4.receive(AndroidInstrumentationReporter.java:138)
at cucumber.runtime.formatter.AndroidInstrumentationReporter$4.receive(AndroidInstrumentationReporter.java:135)
at cucumber.runner.EventBus.send(EventBus.java:28)
at cucumber.runner.TestCase.run(TestCase.java:58)
at cucumber.runner.Runner.runPickle(Runner.java:44)
at cucumber.runtime.android.CucumberExecutor.execute(CucumberExecutor.java:128)
at cucumber.api.android.CucumberInstrumentationCore.start(CucumberInstrumentationCore.java:88)
at com.steps.Implement.RunCucumberTest.onStart(RunCucumberTest.java:43)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)

My project set up is as below

|-src
  |-androidTest
    |-assets
       |-features
    |-java
      |-com
        |-steps
          |-Implement
              -RunCucumberTest
              -Stepdefs

here's what is in RunCucumberTest.java:

@CucumberOptions(
                features = "features",
                )
public class RunCucumberTest  extends MonitoringInstrumentation {
}

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

Context & Motivation

Your Environment

  • Version used:cucumber-java:4.0.0 , cucumber-junit:4.0.0 and cucumber-core:4.0.0
  • Operating System and version:ubuntu 14.04 LTS
  • Link to your project:

Cucumber + Android x + Custom Instrumentation = failure to run the tests.

I am facing the exact situation mentioned in here:
https://stackoverflow.com/questions/54832425/android-cucumber-test-running-failed

Looks like AndroidJUnitRunner is somehow interfering with CustomInstrmentationRunner.
Here is the error:
No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack @test annotations).

My project looks exactly like the one in the sample mentioned in the StackOverflow link above

State not reset between tests when using Test Orchestrator

When clearPackageData: true is set and the Test Orchestrator is used, state should not persist between Scenarios.
State is not reset between tests when using test orchestrator.
Note: Originally I thought clearPackageData: true must be set for state to be reset with Test Orchestrator. Apparently that is the default behavior, and clearPackageData deals with file system (shared prefs, DB, etc)

How to reproduce:
A branch with failing tests are here: #57
Run tests with ./gradlew runInstrumentationTests and observe failures.

Changes were made to the cukelator example where a static field is set on first run of MainActivity.onCreate(). If the flag is set in subsequent tests, it finishes the activity and fails the test.
Commented settings in build.gradle were uncommented to enable Orchestrator and clearPackageData.

As a sanity check I completed the same test on the testing-samples calculator tests. This PR shows code that pass the tests.
patjackson52/testing-samples#1

Android Test Orchestrator - cucumber test report is deleted after each test

Steps:

  1. Setup Android project with cucumber android - create several tests
  2. Enable Android Test Orchestrator + set clear package data after each test to true (I followed setup from cukeulator)
  3. Run tests using adb and android test orchestrator (hint: https://developer.android.com/training/testing/junit-runner#ato-command-line)
  4. Pull test reports from device

Result:
4. Report contains only the last test

Expected:
4. Report should contain all tests

The reason seems to be quite straightforward - package data is deleted after each test by test orchestrator. I tried to setup another locations for the reports, but I can only write to external storage belonging to the package. Any idea how to workaround that?

Possible bug specifying multiple individual scenarios?

How can we specify individual test scenarios?

With the AndroidJUnitRunner you can specify multiple individual test cases using a comma with the "class" args, eg:

"class=ClassName1#methodName1,ClassName2#methodName2"

But the CucumberRunner only seems to support specifying only one scenario this way. Is this a bug?

I have a list of features and the scenarios I want to run, how do I specify that to the runner?

Using feature file line numbers won't work because the runner doesn't report the line numbers to TestRunListener. So we will have to somehow map the reported className (featureName) and methodName (scenarioName) back to the feature file number in other for third party tools we use to continue working.

Are there any plans to update to the newer versions of cucumber?

Hi, I was wondering if you were planning to update the dependencies to the current version. Cucumber-jvm is on version 6.x now, while cucumber-android is dependent on 4.x
I tried to use the current cucumber-jvm with an android emulator, but was unsuccessful.
Thanks.

Cucumber does 't work in project when it is defined applicationIdSuffix

Hi, Good morning.
I am not cucumber expert.
When my project use cucumber along with build variants + applicationIdSuffix It run into troubles like:

io.cucumber.junit.CucumberJUnitRunnerBuilder > initializationError[Mi A2 - 10] FAILED
cucumber.runtime.CucumberException: No CucumberOptions annotation
at io.cucumber.junit.AndroidJunitRuntimeOptionsFactory.createRuntimeOptions(AndroidJunitRuntimeOptionsFactory.java:55)

Does somebody knows how can I configure a cucumber's project along with build variants + applicationIdSuffix ?

Thanks in advance.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.