retest / recheck-web Goto Github PK

It would be helpful to be able to access the wrapped WebElement from WebElementWrapper.

This will allow me to work around issues like this when trying to execute a script:

Argument is of an illegal type: de.retest.web.selenium.WebElementWrapper

This seems straight forward so I'll have a PR shortly.

Are you interested in contributions to make retest compatible with gradle projects?
I have a working version locally with only minor modifications and would be happy to help.

Just tried the change to only screen shot the WebElement and works perfectly! Just had to add a NameStrategy like suggested.

But got a report loading exception:

Exception loading a Report

java.lang.ClassCastException: class java.lang.Float cannot be cast to class de.retest.recheck.persistence.Persistable (java.lang.Float is in module java.base of loader 'bootstrap'; de.retest.recheck.persistence.Persistable is in unnamed module of loader 'app')
	at de.retest.recheck.persistence.bin.KryoPersistence.load(SourceFile:52)
	at de.retest.recheck.persistence.PersistenceFactory$DynamicLoadPersistenceProxy.load(SourceFile:85)
	at de.retest.report.LoadTestReportFromInputFlow.a(SourceFile:83)
	at de.retest.report.LoadTestReportFromInputFlow.a(SourceFile:135)
	at de.retest.report.LoadTestReportFromInputFlow.a(SourceFile:75)
	at de.retest.report.LoadTestReportFromInputFlow.a(SourceFile:53)
	at de.retest.gui.review.workers.LoadTestReportWorker.a(SourceFile:69)
	at de.retest.gui.review.workers.LoadTestReportWorker.doInBackground(SourceFile:39)
	at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:304)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:343)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Created from #326

org.openqa.selenium.JavascriptException: javascript error: Cannot read property 'align-self' of null (Session info: chrome=74.0.3729.108) (Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}),platform=Windows NT 10.0.10586 x86_64) (WARNING: The server did not provide any stacktrace information) Command duration or timeout: 0 milliseconds Build info: version: '3.8.1', revision: '6e95a6684b', time: '2017-12-01T18:33:54.468Z' System info: host: 'TXRICMAC0699747.local', ip: 'fe80:0:0:0:18ed:1134:bc20:90bf%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.14.6', java.version: '1.8.0_144' Driver info: org.openqa.selenium.remote.RemoteWebDriver Capabilities {acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: chrome, chrome: {chromedriverVersion: 74.0.3729.6 (255758eccf3d24..., userDataDir: C:\Users\ADMINI~1\AppData\L...}, cssSelectorsEnabled: true, databaseEnabled: false, goog:chromeOptions: {debuggerAddress: localhost:49754}, handlesAlerts: true, hasMetadata: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: XP, platformName: XP, proxy: Proxy(pac: http://127.0.0.1..., rotatable: false, setWindowRect: true, strictFileInteractability: false, takesHeapSnapshot: true, takesScreenshot: true, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unexpectedAlertBehaviour: ignore, unhandledPromptBehavior: ignore, version: 74.0.3729.108, webStorageEnabled: true, webdriver.remote.sessionid: bd66936d5e0440a1b452f03d278...} Session ID: bd66936d5e0440a1b452f03d27846a89 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214) at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166) at org.openqa.selenium.remote.http.JsonHttpResponseCodec.reconstructValue(JsonHttpResponseCodec.java:40) at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:80) at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:44) at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:164) at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:601) at org.openqa.selenium.remote.RemoteWebDriver.executeScript(RemoteWebDriver.java:537) at de.retest.web.RecheckSeleniumAdapter.convert(RecheckSeleniumAdapter.java:109) at de.retest.web.RecheckSeleniumAdapter.convertWebDriver(RecheckSeleniumAdapter.java:96) at de.retest.web.RecheckSeleniumAdapter.convert(RecheckSeleniumAdapter.java:89) at de.retest.web.RecheckSeleniumAdapter.convert(RecheckSeleniumAdapter.java:86) at de.retest.recheck.persistence.RecheckSutState.convert(RecheckSutState.java:25) at de.retest.recheck.RecheckImpl.createActionReplayResult(RecheckImpl.java:138) at de.retest.recheck.RecheckImpl.check(RecheckImpl.java:126) at de.retest.recheck.RecheckImpl.check(RecheckImpl.java:116)

The PR #155 will fix issue #143. The element <pre> still looses its formatting while beeing processed.

When changing the target of a link, for instance

<li><span><a href="https://www.retest.de/en/for-testers/index.html">For Testers</a></span>

to

<li><span><a href="SOME BROKEN URL">For Testers</a></span>

then the change is not detected. Consequently, we should also check href.

No accept/ignore/apply for differences either.

Originally posted by @javydreamercsw in #326 (comment)

There are defaults that are specific to a browser, see e.g. 195b709 which added defaults of a recent Firefox release to the GM. Therefore, it would probably be helpful to not just have defaults.yml, but additionally defaults-chrome.yml, defaults-firefox.yml, …

I'm loving the tool but sometimes feels like is trying to do too much leading to too much noise.

It would be awesome if we could annotate WebElements within a PageObject to be monitored (meaning screenshots taken as it's being interacted with) ignoring the to non-monitored ones.

At work there are multiple regions working on different parts of the complex application that we don't care about in some focused tests.

Looks like this goes against #29. In our case the current behavior generates way too many files already.

Started giving it a try and noticed that the AutocheckingRecheckDriver constructor changed. There seems to no longer be a way to set a naming strategy. In my case I used to pass a custom AutocheckingCheckNamingStrategy and that's no longer possible it seems.

Created from #326

Getting the below exception when calling .check(). I even tried using just the example from the github page and still get exact same error. I've tried 1.5.0-beta.2 and 1.5.0-beta.1. I'm on a mac if it makes a difference. The exception is:
java.lang.NoClassDefFoundError: org/openqa/selenium/WrapsElement at de.retest.web.RecheckSeleniumAdapter.canCheck(RecheckSeleniumAdapter.java:56) at de.retest.recheck.execution.RecheckAdapters.lambda$findAdapterFor$0(RecheckAdapters.java:16) at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174) at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812) at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126) at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.findAny(ReferencePipeline.java:469) at de.retest.recheck.execution.RecheckAdapters.findAdapterFor(RecheckAdapters.java:17) at de.retest.recheck.RecheckImpl.check(RecheckImpl.java:117) at com.tmobile.he.tests.customer.CustomerSanity.itShouldLoginWithValidCredentials(CustomerSanity.java:29) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124) at org.testng.internal.Invoker.invokeMethod(Invoker.java:580) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:716) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:988) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.privateRun(TestRunner.java:648) at org.testng.TestRunner.run(TestRunner.java:505) at org.testng.SuiteRunner.runTest(SuiteRunner.java:455) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415) at org.testng.SuiteRunner.run(SuiteRunner.java:364) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208) at org.testng.TestNG.runSuitesLocally(TestNG.java:1137) at org.testng.TestNG.runSuites(TestNG.java:1049) at org.testng.TestNG.run(TestNG.java:1017) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:135) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:146) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143) Caused by: java.lang.ClassNotFoundException: org.openqa.selenium.WrapsElement at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 43 more

I'm trying to compile the provided example, but I get the following error at the line driver.findElement( By.id( "intro-slider" ) );

Wed Jul 10 14:04:49 CEST 2019:ERROR:java.lang.NoSuchMethodError: de.retest.recheck.TestCaseFinder.findTestCaseMethodInStack()Ljava/lang/StackTraceElement;
   java.lang.NoSuchMethodError: de.retest.recheck.TestCaseFinder.findTestCaseMethodInStack()Ljava/lang/StackTraceElement;
   	at de.retest.web.selenium.TestHealer.writeWarnLogForChangedIdentifier(TestHealer.java:211)
   	at de.retest.web.selenium.TestHealer.findElementById(TestHealer.java:84)
   	at de.retest.web.selenium.TestHealer.findElement(TestHealer.java:54)
   	at de.retest.web.selenium.TestHealer.findElement(TestHealer.java:49)
   	at de.retest.web.selenium.UnbreakableDriver.findElement(UnbreakableDriver.java:88)
   	at org.openqa.selenium.WebDriver$findElement$1.call(Unknown Source)
   	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
   	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
   	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
   	at de.retest.web.it.Script16.run(Script16.groovy:127)
   	at com.eviware.soapui.support.scripting.groovy.SoapUIGroovyScriptEngine.run(SoapUIGroovyScriptEngine.java:90)
   	at com.eviware.soapui.impl.wsdl.teststeps.WsdlGroovyScriptTestStep.run(WsdlGroovyScriptTestStep.java:141)
   	at com.eviware.soapui.impl.wsdl.panels.teststeps.GroovyScriptStepDesktopPanel$RunAction$1.run(GroovyScriptStepDesktopPanel.java:250)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
   	at java.lang.Thread.run(Unknown Source)

hi, is there support for cucumber(bdd)?

I'm getting issues while trying to use Recheck on my project. My project is a TestNG and has a rule to name tests that I suspect its related. See attached simple project reproducing the issue.

# Created at 2019-07-05T09:14:16.485 org.testng.TestNGException: Cannot instantiate class com.xxxx.basetest.DirectLoginTestNGTest at org.testng.internal.ObjectFactoryImpl.newInstance(ObjectFactoryImpl.java:30) at org.testng.internal.ClassHelper.createInstance1(ClassHelper.java:423) at org.testng.internal.ClassHelper.createInstance(ClassHelper.java:336) at org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:125) at org.testng.internal.ClassImpl.getInstances(ClassImpl.java:190) at org.testng.TestClass.getInstances(TestClass.java:95) at org.testng.TestClass.initTestClassesAndInstances(TestClass.java:81) at org.testng.TestClass.init(TestClass.java:73) at org.testng.TestClass.<init>(TestClass.java:38) at org.testng.TestRunner.initMethods(TestRunner.java:389) at org.testng.TestRunner.init(TestRunner.java:271) at org.testng.TestRunner.init(TestRunner.java:241) at org.testng.TestRunner.<init>(TestRunner.java:167) at org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:663) at org.testng.SuiteRunner.init(SuiteRunner.java:260) at org.testng.SuiteRunner.<init>(SuiteRunner.java:198) at org.testng.TestNG.createSuiteRunner(TestNG.java:1295) at org.testng.TestNG.createSuiteRunners(TestNG.java:1273) at org.testng.TestNG.runSuitesLocally(TestNG.java:1128) at org.testng.TestNG.runSuites(TestNG.java:1049) at org.testng.TestNG.run(TestNG.java:1017) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:135) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:146) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.testng.internal.ObjectFactoryImpl.newInstance(ObjectFactoryImpl.java:24) ... 28 more Caused by: java.lang.IllegalStateException: Couldn't identify test class in call stack. This is needed to dynamically name the Golden Master files. Please instantiate RecheckImpl with RecheckOptions, and provide a different FileNamerStrategy. at de.retest.recheck.FileNamerStrategy.lambda$getTestClassName$0(FileNamerStrategy.java:29) at java.util.Optional.orElseThrow(Optional.java:290) at de.retest.recheck.FileNamerStrategy.getTestClassName(FileNamerStrategy.java:29) at de.retest.recheck.RecheckOptions.$default$suiteName(RecheckOptions.java:7) at de.retest.recheck.RecheckOptions.<init>(RecheckOptions.java:7) at de.retest.recheck.RecheckOptions$RecheckOptionsBuilderImpl.build(RecheckOptions.java:7) at com.xxxx.basetest.BaseTest.<init>(BaseTest.java:133) at com.xxxx.basetest.ConsoleLoginTest.<init>(ConsoleLoginTest.java:23) at com.xxxx.basetest.DirectLoginTestNGTest.<init>(DirectLoginTestNGTest.java:11) ... 33 more

RecheckError.zip

When testing locally with the Firefox browser, the screenshots are truncated horizontally. Looks like there's something wrong with the ScrollStrategy.

Steps to reproduce:

• Tested website: https://stackoverflow.com/

Tested under:

• macOS Catalina version 10.15.1
• OpenJDK 11
• recheck 1.6.0
• geckodriver v0.26.0
• Firefox 69.0

Multiple inserted elements should be summarized. E.g. instead of
TR: [expected=null, actual=TR], TD [sunscreen]: [expected=null, actual=TD [sunscreen]], TD [2.02]: [expected=null, actual=TD [2.02]]

We want a TR (with 2 child elements): [expected=null, actual=sunscreen 2.02]

Appears to be a regression for which we don't have tests (i.e. verify insertion, change, and deletion yield a test failure).

Steps to reproduce:

1. Execute e.g. CenterTestIT and verify it passes locally
2. Delete the <div> (ID "center") in lines 35 – 37 from src/test/resources/pages/centered.html
3. Run the test again, it passes (although it should fail)

In contrast, if you change the data of the <div> (e.g. adapt the text content), then the test fails as expected.

EDIT: Inserted elements also seem to be affected. If you simply add another <div> to the <body>, the test passes too.

Many default values come in the form of "em". They need to be converted to px...

This is related to #288 . Trying to run UnbreakableDriver.executeScript lets to the error below. I assume it's not unwrapping the proxy before passing it down.

Here's the relevant stack trace:
java.lang.IllegalArgumentException: Argument is of an illegal type: de.retest.web.selenium.WebElementWrapper at org.openqa.selenium.remote.internal.WebElementToJsonConverter.apply(WebElementToJsonConverter.java:83) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) at org.openqa.selenium.remote.RemoteWebDriver.executeScript(RemoteWebDriver.java:484) at de.retest.web.selenium.UnbreakableDriver.executeScript(UnbreakableDriver.java:231)

I'm using recheck-web 1.3.0.

Instead of creating a screenshot per web element (which results in lots of screenshots), only create an screenshot for the overall site.

This overall screenshot should also be used within retest-review to display the elements, by cutting it out...

Paths currently have the following form:

HTML[1]/BODY[1]/A[1]

The path represents the absolute XPath to the corresponding element. According to the XPath specification:

An absolute location path consists of / optionally followed by a relative location path.

Moreover, although the HTML specification says that "tag names are case-insensitive", they typically only consist of lower-case letters.

Therefore, we should start with / and use lower-case letters:

/html[1]/body[1]/a[1]

When e.g. removing parts of a website, this will often affect many different HTML elements. These show up as individual changes per removed element, which can be cumbersome and irritating.

Instead, have such changes show up as a single change to the topmost removed element and all child elements (like we do for insertions).

Similar to #295, this appears to be a regression for which we don't have tests.

Steps to reproduce:

1. Execute e.g. CenterTestIT and verify it passes locally
2. Change the ID (e.g. from id="center" to id="foo") of the <div> in lines 35 – 37 from src/test/resources/pages/centered.html
3. Run the test again, it passes (although it should fail)

The generated recheck.ignore file has the following link:

https://retest.github.io/docs/recheck/how-ignore-works-in-recheck/

Which leads to a 404.

the Package de.retest.web.testutils are not found in your Maven Central Release.

Problem

When we encounter missing elements with the Unbreakable Selenium, we try to find the old element inside the golden master. Therefore, we query the By used and look what type was used (i.e. By.id, By.class).

However, when we try to find By.cssSelector, we currently only support simple css selectors. The supported selectors look for some known attributes like id or class. For example:

• By.cssSelector( "#id" );
• By.cssSelector( ".class" );

This should be the same for @FindBy.

Solution

We want to support more complex selectors. Note, that this issue should only address selectors that query the current element. It should not look for children or parents. A complete specification of CSS selectors can be found in the CSS specification.

The current element would mean to selects any element (this also includes any * or no selector) and queries a predicate [] or state :. Note that the state may only look at the elements' attributes. Please refer to the examples below.

Examples this issue should address:

• .class1.class2
• p.intro
• div, p
• a[target]
• a[target=_blank
• input:checked

Examples not considered for this issue (see #390):

• div p
• div > p
• p::before
• p:nth-child(4)

Community

I kindly ask the community to provide examples below that should be reflected with this issue, so that we can use these to test the implementation.

We should always unwrap UnbreakableDriver instances before we pass them to 3rd party libs. #277 does this for screenshots because the problem here was that Shutterbug internally uses instanceof checks to adapt the screenshot behavior e.g. in the case of ChromeDriver. However, since UnbreakableDriver is never true for instanceof checks on default WebDriver types, we caused false behavior. This lead to broken tests as Shutterbug didn't scroll back to the top of the page, causing WebElements not being clickable anymore.

Problem

When we encounter missing elements with the Unbreakable Selenium, we try to find the old element inside the golden master. Therefore, we query the By used and look what type was used (i.e. By.id, By.class).

However, when we try to find By.cssSelector, we currently only support simple css selectors. The supported selectors look for some known attributes like id or class. For example:

• By.cssSelector( "#id" );
• By.cssSelector( ".class" );

This should be the same for @FindBy.

Solution

We want to support hierarchical selectors. Note, that this issue should only address selectors that query the parent or its' children of the current element. It should not look for attributes on the current element. A complete specification of CSS selectors can be found in the CSS specification.

The current element would mean to selects any element (this also includes any * or no selector) and queries parent, children, state : or pseudo-element ::. Note that the state may only look at the elements' parent or children. Please refer to the examples below. Note that those examples are not complete, refer to the specification for more.

Examples this issue should address

• div p
• div > p
• p~ul
• p::before
• p:nth-child(4)

Examples not considered for this issue (see #389)

• a[target]
• a[target=_blank
• input:checked

Community

I kindly ask the community to provide examples below that should be reflected with this issue, so that we can use these to test the implementation.

Adding an id and a name to a div like here

https://github.com/retest/recheck-web/blob/master/src/test/resources/pages/showcase/retest.html#L192

Leads to no differences, although we are supposed to track these changes?

When I run SimpleShowcaseIT (current master on 8b9fd72) locally on my Mac, it is green. If I change just a little thing, e.g., the <h1> content in line 138 in the retest.html page, then I get 100 differences (of which are 109 unique) out of nowhere. This is extremely confusing since I only changed a single element. See attachements for details.

Which then leads to a UncheckIOException since the GM can't be read anymore on subsequent test runs.

How can we determine the correct persistence with respect to the project configuration?

Problem

When we encounter missing elements with the Unbreakable Selenium, we try to find the old element inside the golden master. Therefore, we query the By used and look what type was used (i.e. By.id, By.class).

However, when we try to find By.xpath, we currently only support simple XPath. Furthermore, we do not follow common format as described in #132. For example, we require:

• By.xpath( "//input[1]" );
• By.xpath( "/html[1]/body[1]/div[1]" );

This should be the same for @FindBy.

Solution

We want to support the xpath specification. Note that browsers might only support XPath 1.0.

The current element would mean to selects any element (this also includes any * or no selector) and queries parent, children, state : or pseudo-element ::. Note that the state may only look at the elements' parent or children. Please refer to the examples below. Note that those examples are not complete, refer to the specification for more.

Examples this issue should address

• /html/body/div[1]
• /html//div[2]
• //input[@type=file]
• //*[@id="foo"]/summary

Community

I kindly ask the community to provide examples below that should be reflected with this issue, so that we can use these to test the implementation.

Since recheck.ignore is additive, we can also have one per state. This solves some immediate problems, e.g. where a XPath should only be ignored within a certain state.

Describe the Bug

Running test with golden files from 1.5.0 leads to the following error in most cases:

de.retest.web.selenium.NoElementWithHighEnoughMatchFoundException: The element input from the Golden Master could not be matched with high enough confidence.

How to Reproduce?

Steps to reproduce the behavior:

1. Run test with 1.5.0 to create golden files.
2. Re-run with 1.6.0.

Setup

• recheck-web: 1.6.0
• Selenium version: 3.141.59
• Testing framework: TestNG 7.0.0
• OS: MacOS 14
• Java version: OpenJDK 11

Additional Context

N/A

The interface should be more general, i.e. filter only if something applies... this also allows for a general 'filter' functionality, e.g. in the GUI.

Applying, e.g, the IntelliJ HTML formatter to

https://github.com/retest/recheck-web/blob/master/src/test/resources/pages/showcase/retest.html

leads to multiple differences. The formatter introduces line breaks, for instance

<li><a href="https://www.retest.de/en/for-testers/test-automation-without-programming.html">Automation without Programming</a></li>

becomes

<li><a href="https://www.retest.de/en/for-testers/test-automation-without-programming.html">Automation
    without Programming</a></li>

The rendered result within the browser isn't different. (We should probably look into the HTML specification.)

Create a yaml file (or incorporate into existing) the default values for all CSS attributes, as specified here: https://www.w3schools.com/cssref/css_default_values.asp

Conversion sometimes takes up to 5 or more seconds. It would be helpful (for surili as well) if we could somehow improve the conversion performance.

Both absolute outline and outline are pixel based, so whenever small rendering differences occur, they already differ. Therefore, I think they are poor indicators for matching elements and we should make them normal attributes.

Our JavaScript getAllElementsByPath.js is growing and it implements important functionality. We should add unit tests for it.

Open questions:

• How to integrate it in our CI pipeline?
• Which testing framework(s) to use for running tests, mocking, etc. (e.g. Jest)?

For different steps during my frontend test I need to wait for fade-ins, but the .click() method seems to trigger a re.check() call (or similar), causing the tests to fail most of the time due to slightly different opacity (while fading in).

How can I disable the implicit checks during driver action calls?

Test class:

public class FrontendTest {
    private static final String URL = "http://frontend-apache/path/to/page.html";
    private RecheckDriver driver;
    private Recheck re;

    @BeforeEach
    public void setUp() throws MalformedURLException {
        driver = new RecheckDriver(new RemoteWebDriver(new URL("http://127.0.0.1:4445/wd/hub"), DesiredCapabilities.chrome()));
        re = new RecheckWebImpl();
    }

    @Test
    public void test() throws Exception {
        re.startTest("frontend");

        driver.get(URL);
        waitForPageFadeIn(driver);
        re.check( driver, "open" );

        driver.findElement(By.xpath("//*[@id=\"category_1\"]")).click(); // check "01", causing the issue
        waitForSectionFadeIn(driver);
        re.check(driver, "select-category-1"); // check "select-category-1"

        driver.findElement(By.xpath("//*[@id=\"category_2\"]")).click();
        waitForChange();
        re.check(driver, "select-category-2");

        // ...

        re.capTest();
    }

    @AfterEach
    public void tearDown() {
        driver.quit();
        re.cap();
    }

The log of the test execution, showing first step '01' and then step 'select-category-1':

[main] INFO de.retest.web.RecheckSeleniumAdapter - Retrieving attributes for each element.
[main] INFO de.retest.web.RecheckSeleniumAdapter - Checking website http://frontend-apache/path/to/page.html with 545 elements.
[main] DEBUG de.retest.web.RecheckSeleniumAdapter - Found 0 frames, getting data per frame.
[main] DEBUG de.retest.recheck.ui.diff.Alignment - Creating assignment of old to new components, trying to find differences. We are comparing 161 with 161 components.
[main] DEBUG de.retest.recheck.ui.diff.RootElementDifferenceFinder - Finding differences for window took 472ms.
[main] DEBUG de.retest.recheck.execution.RecheckDifferenceFinder - Found 1 differences for step '01'.
[main] INFO de.retest.web.RecheckSeleniumAdapter - Retrieving attributes for each element.
[main] INFO de.retest.web.RecheckSeleniumAdapter - Checking website http://frontend-apache/path/to/page.html with 545 elements.
[main] DEBUG de.retest.web.RecheckSeleniumAdapter - Found 0 frames, getting data per frame.
[main] DEBUG de.retest.recheck.ui.diff.Alignment - Creating assignment of old to new components, trying to find differences. We are comparing 161 with 161 components.
[main] DEBUG de.retest.recheck.ui.diff.RootElementDifferenceFinder - Finding differences for window took 477ms.
[main] DEBUG de.retest.recheck.execution.RecheckDifferenceFinder - Found 1 differences for step 'select-category-1'.

Test result:

01 resulted in:
	div at 'html[1]/body[1]/div[1]/div[1]/div[2]/div[2]':
		opacity: expected="0.00665703", actual="0.0071777"
		style: expected="display: block; opacity: 0.00665703;", actual="display: block; opacity: 0.0071777;"

Java version: 1.8.0_212
recheck-web version: 1.1.0
junit version: 5.5.0-M1
selenium version: 3.141.59

The outline property is overwritten instead of appended.

For Surili we need to get an empty retest ID and filter for custom attributes to keep the number of states smaller. If we use system properties for both, it will affect tests which do use recheck-web and not Surili, too. We cannot guarantee that a customer does not mix both in the same project.

A cleaner approach would be something like this:

final RetestIdProvider suriliRetestIdProvider = new SuriliRetestIdProvider();
final AttributesProvider suriliAttributesProvider = new SuriliAttributesProvider();
final RecheckAdapter recheckAdapter = new RecheckSeleniumAdapter(suriliRetestIdProvider, suriliAttributesProvider);
final Set<RootElement> rootElements = recheckAdapter.convert( driver );
final SutState sutState = new SutState( rootElements );

We need a constructor for RecheckSeleniumAdapter which allows specifying both. The default constructor could still use the singleton instances.
Maybe we should also define interfaces for RetestIdProvider and AttributesProvider.

We want classes of attribute equivalence.

E.g. one system has font "Times", another "Times New Roman", and they should be equivalent. Same for rounding errors of sub-pixel size, margin, padding, etc. Or cursor "pointer" and "null"—just to name a few.

and {clip = [expected=rect(0px, 0px, 0px, 0px), actual=rect(0px 0px 0px 0px)]}

At the moment we get only one test inside a result file but we want to get all tests from a test class inside one result file.

actual:

expected

Looks like this needs to be updated to use By.tag() instead.

at de.retest.web.selenium.TestHealer.retrieveUsableCssSelector(TestHealer.java:161)
	at de.retest.web.selenium.TestHealer.findElementByCssSelector(TestHealer.java:139)
	at de.retest.web.selenium.TestHealer.findElement(TestHealer.java:66)
	at de.retest.web.selenium.TestHealer.findElement(TestHealer.java:49)
	at de.retest.web.selenium.UnbreakableDriver.findElement(UnbreakableDriver.java:81)
	at de.retest.web.selenium.AutocheckingRecheckDriver.findElement(AutocheckingRecheckDriver.java:107)
	at jdk.internal.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.openqa.selenium.support.events.EventFiringWebDriver.lambda$new$1(EventFiringWebDriver.java:105)
	at com.sun.proxy.$Proxy17.findElement(Unknown Source)
	at org.openqa.selenium.support.events.EventFiringWebDriver.findElement(EventFiringWebDriver.java:194)

Describe the Bug

On version 1.6.0 the golden files are named differently. They use a superclass instead of the actual test class rendering old golden files invalid.

How to Reproduce?

Steps to reproduce the behavior:

1. Run test with 1.5.1 to create golden files.
2. Repeat same test with 1.6.0.

Setup

• recheck-web: 1.6.0
• Selenium version: 3.141.59
• Testing framework: TestNG 7.0.0
• OS: [MacOS 14
• Java version: OpenJDK 11

Additional Context

Changing the height and width of an img, e.g., here

https://github.com/retest/recheck-web/blob/master/src/test/resources/pages/showcase/retest.html#L105

to height=13 and width=19, leads to the following console output

The following differences have been found in 'de.retest.web.it.SimpleRecheckShowcaseIT'(with 1 check(s)):
Test 'simple-showcase' has 2 differences (2 unique):
index resulted in:
    A at 'HTML[1]/BODY[1]/HEADER[1]/DIV[1]/DIV[1]/NAV[1]/UL[1]/LI[5]/UL[1]/LI[1]/A[1]':
        outline: expected="java.awt.Rectangle[x=0,y=0,width=-42,height=42]", actual="java.awt.Rectangle[x=0,y=0,width=-41,height=42]"
    IMG at 'HTML[1]/BODY[1]/HEADER[1]/DIV[1]/DIV[1]/NAV[1]/UL[1]/LI[5]/UL[1]/LI[1]/A[1]/IMG[1]':
        outline: expected="java.awt.Rectangle[x=20,y=14,width=-98,height=-30]", actual="java.awt.Rectangle[x=20,y=13,width=-98,height=-29]"
    IMG at 'HTML[1]/BODY[1]/HEADER[1]/DIV[1]/DIV[1]/NAV[1]/UL[1]/LI[5]/UL[1]/LI[1]/A[1]/IMG[1]':
        outline: expected="java.awt.Rectangle[x=20,y=14,width=-98,height=-30]", actual="java.awt.Rectangle[x=20,y=13,width=-98,height=-29]"

Looks like a change in y and height?

Looks like we get stuff like this:

15:20:07.239 [Thread-4] WARN de.retest.recheck.RecheckImpl - You should call 'cap()' after your test 'de.retest.web.ShowcaseIT' has finished, to persist test results

On each integration test:

https://travis-ci.com/retest/recheck-web/builds/90577489#L1833

Although cap() is being invoked:

https://github.com/retest/recheck-web/blob/master/src/test/java/de/retest/web/ShowcaseIT.java

The branch feature/unbreakable-selenium (here) includes a POC for dealing with broken locators, based on recheck-web.

I have already talked to @roesslerj regarding the integration—since we need it for surili—but we disagree whether this should be part of recheck-web (Jeremy) or a separate project (myself). I think this boils down to monoliths vs. microservices, where both have their pros and cons.

However, I would like to use this issue to hear opinions from the team.