retry is not working for BDD examples about frameworkium-bdd

[ham1]

Where did you add this? The feature file or src/test/resources/frameworkium-cucumber-runner.vm? It will only work if you edit the latter file to include it as it's a Java annotation not a cucumber tag.

[AriKatze]

I added it on the Step file, as the documentation does not specify that it has to be added on the runner. https://frameworkium.github.io/#_pages/Retrying-tests.md
Now, I edited the tag to the custom runner:

Ran the script again.. but still the test is only executed once

May I be missing something?

[ham1]

Sorry, the docs have been largely written from a non-BDD perspective.

Could you try adding it to the test method inside the class, in the vm file?

[AriKatze]

By adding it on the vm file works!
Thanks for the help

[AriKatze]

the only odd thing is that even when both executions failed the first one is marked as PASS

[ham1]

I can't see why that's happening at the moment.

Could you try a mvn clean and then re-run the tests to see if the same issues occurs.
If so, try removing the retryAnalyzer= ... from the @Test on the class (where you first put it) inside the .vm file and only have it on the test method.

Finally, if it's still broken, please post the output of mvn dependency:tree (it might be a bug in the version of TestNG we are using but just want to confirm which version you are using).

Thanks.

[AriKatze]

The issue still persists, here is the mvn tree:

[INFO] com.frameworkium:Frameworkium-BDD:jar:3.0.0-BDD-SNAPSHOT
[INFO] +- com.github.frameworkium:frameworkium-core:jar:3.0.0-BETA1:compile
[INFO] |  +- org.seleniumhq.selenium:selenium-java:jar:3.14.0:compile
[INFO] |  |  +- org.seleniumhq.selenium:selenium-api:jar:3.14.0:compile
[INFO] |  |  +- org.seleniumhq.selenium:selenium-chrome-driver:jar:3.14.0:compile
[INFO] |  |  +- org.seleniumhq.selenium:selenium-edge-driver:jar:3.14.0:compile
[INFO] |  |  +- org.seleniumhq.selenium:selenium-firefox-driver:jar:3.14.0:compile
[INFO] |  |  +- org.seleniumhq.selenium:selenium-ie-driver:jar:3.14.0:compile
[INFO] |  |  +- org.seleniumhq.selenium:selenium-opera-driver:jar:3.14.0:compile
[INFO] |  |  +- org.seleniumhq.selenium:selenium-remote-driver:jar:3.14.0:compile
[INFO] |  |  +- org.seleniumhq.selenium:selenium-safari-driver:jar:3.14.0:compile
[INFO] |  |  +- org.seleniumhq.selenium:selenium-support:jar:3.14.0:compile
[INFO] |  |  +- net.bytebuddy:byte-buddy:jar:1.8.15:compile
[INFO] |  |  +- org.apache.commons:commons-exec:jar:1.3:compile
[INFO] |  |  +- commons-codec:commons-codec:jar:1.10:compile
[INFO] |  |  +- commons-logging:commons-logging:jar:1.2:compile
[INFO] |  |  +- org.apache.httpcomponents:httpclient:jar:4.5.5:compile
[INFO] |  |  +- org.apache.httpcomponents:httpcore:jar:4.4.9:compile
[INFO] |  |  +- com.squareup.okhttp3:okhttp:jar:3.10.0:compile
[INFO] |  |  \- com.squareup.okio:okio:jar:1.14.1:compile
[INFO] |  +- ru.yandex.qatools.htmlelements:htmlelements-java:jar:1.19:compile
[INFO] |  +- com.paulhammant:ngwebdriver:jar:1.1.4:compile
[INFO] |  +- net.sf.uadetector:uadetector-resources:jar:2014.10:compile
[INFO] |  |  +- net.sf.uadetector:uadetector-core:jar:0.9.22:compile
[INFO] |  |  |  +- net.sf.qualitycheck:quality-check:jar:1.3:compile
[INFO] |  |  |  \- javax.annotation:jsr250-api:jar:1.0:compile
[INFO] |  |  \- org.slf4j:slf4j-api:jar:1.7.7:compile
[INFO] |  +- com.saucelabs:sauce_testng:jar:2.1.24:compile
[INFO] |  |  +- com.saucelabs:sauce_java_common:jar:2.1.24:compile
[INFO] |  |  +- com.googlecode.json-simple:json-simple:jar:1.1.1:compile
[INFO] |  |  \- commons-lang:commons-lang:jar:2.6:compile
[INFO] |  +- com.saucelabs:saucerest:jar:1.0.39:compile
[INFO] |  +- org.testng:testng:jar:6.14.3:compile
[INFO] |  |  +- com.beust:jcommander:jar:1.72:compile
[INFO] |  |  \- org.apache-extras.beanshell:bsh:jar:2.0b6:compile
[INFO] |  +- io.qameta.allure:allure-testng:jar:2.7.0:compile
[INFO] |  |  \- io.qameta.allure:allure-descriptions-javadoc:jar:2.7.0:compile
[INFO] |  +- com.google.truth:truth:jar:0.42:compile
[INFO] |  |  +- org.checkerframework:checker-compat-qual:jar:2.5.3:compile
[INFO] |  |  +- org.checkerframework:checker-qual:jar:2.5.3:compile
[INFO] |  |  +- junit:junit:jar:4.12:compile
[INFO] |  |  +- com.googlecode.java-diff-utils:diffutils:jar:1.3.0:compile
[INFO] |  |  +- com.google.auto.value:auto-value-annotations:jar:1.6.2:compile
[INFO] |  |  \- com.google.errorprone:error_prone_annotations:jar:2.3.1:compile
[INFO] |  +- com.google.truth.extensions:truth-java8-extension:jar:0.42:compile
[INFO] |  +- com.google.guava:guava:jar:26.0-jre:compile
[INFO] |  |  +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO] |  |  +- com.google.j2objc:j2objc-annotations:jar:1.1:compile
[INFO] |  |  \- org.codehaus.mojo:animal-sniffer-annotations:jar:1.14:compile
[INFO] |  +- org.apache.logging.log4j:log4j-api:jar:2.11.1:compile
[INFO] |  +- org.apache.logging.log4j:log4j-core:jar:2.11.1:compile
[INFO] |  +- org.slf4j:slf4j-simple:jar:1.7.25:compile
[INFO] |  +- io.rest-assured:rest-assured:jar:3.1.1:compile
[INFO] |  |  +- org.apache.httpcomponents:httpmime:jar:4.5.1:compile
[INFO] |  |  +- org.hamcrest:hamcrest-core:jar:1.3:compile
[INFO] |  |  +- org.hamcrest:hamcrest-library:jar:1.3:compile
[INFO] |  |  +- org.ccil.cowan.tagsoup:tagsoup:jar:1.2.1:compile
[INFO] |  |  +- io.rest-assured:json-path:jar:3.1.1:compile
[INFO] |  |  |  +- org.codehaus.groovy:groovy-json:jar:2.4.12:compile
[INFO] |  |  |  \- io.rest-assured:rest-assured-common:jar:3.1.1:compile
[INFO] |  |  \- io.rest-assured:xml-path:jar:3.1.1:compile
[INFO] |  |     \- javax.xml.bind:jaxb-api:jar:2.2.12:compile
[INFO] |  +- org.apache.commons:commons-lang3:jar:3.8:compile
[INFO] |  +- commons-io:commons-io:jar:2.6:compile
[INFO] |  +- org.json:json:jar:20180813:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.9.6:compile
[INFO] |  \- org.codehaus.groovy:groovy-all:jar:2.4.15:compile
[INFO] +- info.cukes:cucumber-java:jar:1.2.5:compile
[INFO] |  \- info.cukes:cucumber-core:jar:1.2.5:compile
[INFO] |     +- info.cukes:cucumber-html:jar:0.2.3:compile
[INFO] |     \- info.cukes:cucumber-jvm-deps:jar:1.0.5:compile
[INFO] +- info.cukes:cucumber-testng:jar:1.2.5:compile
[INFO] +- io.qameta.allure:allure-cucumber-jvm:jar:2.7.0:compile
[INFO] |  +- info.cukes:cucumber-junit:jar:1.2.5:compile
[INFO] |  +- info.cukes:gherkin:jar:2.12.2:compile
[INFO] |  \- io.qameta.allure:allure-java-commons:jar:2.7.0:compile
[INFO] |     +- io.qameta.allure:allure2-model-api:jar:1.0.0:compile
[INFO] |     |  +- io.qameta.allure:allure2-model-pojo:jar:1.0.0:compile
[INFO] |     |  \- io.qameta.allure:allure2-model-jackson:jar:1.0.0:compile
[INFO] |     |     \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.7.0:compile
[INFO] |     +- org.aspectj:aspectjrt:jar:1.9.1:compile
[INFO] |     \- org.jooq:joor-java-8:jar:0.9.9:compile
[INFO] \- com.epam.reportportal:agent-java-cucumber:jar:4.0.2:compile
[INFO]    +- com.epam.reportportal:client-java:jar:4.0.8:compile
[INFO]    |  +- com.fasterxml.jackson.core:jackson-databind:jar:2.8.10:compile
[INFO]    |  |  \- com.fasterxml.jackson.core:jackson-core:jar:2.8.10:compile
[INFO]    |  +- io.reactivex.rxjava2:rxjava:jar:2.1.6:compile
[INFO]    |  |  \- org.reactivestreams:reactive-streams:jar:1.0.1:compile
[INFO]    |  \- org.apache.tika:tika-core:jar:1.4:compile
[INFO]    \- com.epam.reportportal:commons-model:jar:4.2.0:compile
[INFO]       +- io.swagger:swagger-annotations:jar:1.5.13:compile
[INFO]       \- javax.validation:validation-api:jar:1.1.0.Final:compile

[ham1]

Try the latest version we've just released. It's quite a jump though to io.cucumber v4.5.2.