donutreport / donut Goto Github PK
View Code? Open in Web Editor NEWSweet reports for the icing over your automated tests
Home Page: http://donutreport.github.io/donut
License: MIT License
Sweet reports for the icing over your automated tests
Home Page: http://donutreport.github.io/donut
License: MIT License
To make Donut ubiquitous, we need to add a jUNit adaptor to read jUnit XML
With the latest version of cucumber-parallel plugin (the one written by Tim), we have the ability to split the execution of cucumber feature files by scenarios as well as feature.
When we split execution by scenarios, we get 1 json report file per scenario, and in this case donut displays that many features in the donut html report.
We need to modify donut to group scenarios under the same feature name, but in different json files, by feature name.
It would be nice to see whether my set of tests executed for a single project & run conformed (or not) to the test pyramid:
https://www.google.com.au/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwi8oJWbnsbTAhXCEpQKHXpbAXEQjRwIBw&url=https%3A%2F%2Fmartinfowler.com%2Fbliki%2FTestPyramid.html&psig=AFQjCNE3berHhPEtexTKPYnwEhKmBVE77w&ust=1493437457556413
Below message is reported in the maven build, in case of test failures, even though donut report is generated successfully:
[ERROR] Failed to execute goal io.magentys:donut-maven-plugin:0.0.3:generate (execution) on project eas-module: Error Found: BUILD FAILED - Check Report For Details -> [Help 1]
Print scenario type ex: Scenarios above all the scenarios, instead of printing 'Scenario' for every scenario.
Why does the donut report generate enormous logs? is there a way to disable the logs to just "Report Generated Successfully"?
Hi,
In the project Wiki I found that we can create HTML report with Generator. Can you guys pls give an example of where this piece of code will @go?
ReportConsole report =
Generator.apply(sourceDirectory, outputDirectory, filePrefix, timestamp,
template, countSkippedAsFailure, countPendingAsFailure,
countUndefinedAsFailure, countMissingAsFailure, projectName, projectVersion);
When running Donut directly from terminal using java -jar donut-0.0.3-one-jar.jar -s .
I sometimes receive an error that prevents the report from being generated. This has been reproduced with many different feature files of varying formats and complexities and I can't spot any obvious pattern.
I have now managed to pin it down to two scenarios, when I run scenario 1, it passes and I can successfully generate a Donut report with the JSON output (test1.json). When I run scenario 2, it also passes and I can successfully generate a Donut report with that JSON output (test2.json). However, if I run scenario 1 and 2 as part of the same test run, attempting to use Donut with that JSON output (test1andtest2.json) throws the following error:
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.IndexOutOfBoundsException: 0
at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
at scala.collection.immutable.List.apply(List.scala:84)
at io.magentys.donut.transformers.cucumber.CucumberTransformer$$anonfun$mapToDonutScenarios$1.apply(CucumberTransformer.scala:55)
at io.magentys.donut.transformers.cucumber.CucumberTransformer$$anonfun$mapToDonutScenarios$1.apply(CucumberTransformer.scala:53)
at scala.collection.immutable.List.flatMap(List.scala:327)
at io.magentys.donut.transformers.cucumber.CucumberTransformer$.mapToDonutScenarios(CucumberTransformer.scala:53)
at io.magentys.donut.transformers.cucumber.CucumberTransformer$.mapToDonutFeature(CucumberTransformer.scala:30)
at io.magentys.donut.transformers.cucumber.CucumberTransformer$$anonfun$mapToDonutFeatures$1.apply(CucumberTransformer.scala:24)
at io.magentys.donut.transformers.cucumber.CucumberTransformer$$anonfun$mapToDonutFeatures$1.apply(CucumberTransformer.scala:21)
at scala.collection.immutable.List.map(List.scala:273)
at io.magentys.donut.transformers.cucumber.CucumberTransformer$.mapToDonutFeatures(CucumberTransformer.scala:21)
at io.magentys.donut.transformers.cucumber.CucumberTransformer$.transform(CucumberTransformer.scala:12)
at io.magentys.donut.gherkin.Generator$$anonfun$3.apply(Generator.scala:61)
at io.magentys.donut.gherkin.Generator$$anonfun$3.apply(Generator.scala:59)
at io.magentys.donut.performance.PerformanceSupport$class.timed(PerformanceSupport.scala:13)
at io.magentys.donut.gherkin.Generator$.timed(Generator.scala:15)
at io.magentys.donut.gherkin.Generator$.createReport(Generator.scala:59)
at io.magentys.donut.gherkin.Generator$.apply(Generator.scala:32)
at io.magentys.donut.Boot$.delayedEndpoint$io$magentys$donut$Boot$1(Boot.scala:82)
at io.magentys.donut.Boot$delayedInit$body.apply(Boot.scala:11)
at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at io.magentys.donut.Boot$.main(Boot.scala:11)
at io.magentys.donut.Boot.main(Boot.scala)
... 6 more
I have attached the respective JSON files for reference. Possibly I'm missing something obvious or doing something wrong when running but this seems to be quite common across our projects.
Currently, the build metadata provided with the report is limited to project version and project name. Users should be able to provide their own additional information.
[ERROR] Failed to execute goalreport.donut:donut-maven-plugin:0.0.6:generate (execution) on project taf-chips: Error Found:: Input length = 1 -> [Help 1]
The error is not very meaningful but we discovered the build fails when there are special characters in the cucumber feature file. For example:
"%!@£$%^&*( “ ” ' ' « » … ° © ® ™ • ½ ¼ ¾ ⅓ ⅔ № † ‡ µ ¢ £ € ♤ ♧
♥️ ♢ ✓)"
Removing those characters from cucumber.json fixes the problem, however it's painful having to manually go through json and remove all special characters especially if the feature file is massive.
Interesting fact is that it only happens on windows machines. Works fine one Mac no matter if the special characters are there or not.
We need to implement a standard model that isn't based on gherkin
Ideally Test Suite, Test Case, Test Step
Test Suite -> Feature
Test Case -> Scenario
Test Step -> Step
This will allow us to more easily support non-gherkin based test tools
The pattern in the ResultLoader does not consider Windows paths contain a drive followed by a colon. If a format is excluded from the path then donut fails with an exception relating to an unsupported format. The pattern regex needs to be tightened.
It would be nice
Examples tables are not being rendered in the report for non-expanded Scenario Outlines. I've created an examples-tables branch on my fork to add them but the <>
delimited parameters are lost in the report.
The branch contains an example 9a.json JSON file used to test this feature. The following is a snippet of a step containing an outline parameter in that file..
"keyword": "Given ",
"name": "string 1 is \"<string 1>\""
I'd expect this to render as Given string 1 is "<string1>"
but instead get Given string 1 is ""
I've tried escaping the value in the HTMLProcessor class but I still get the same problem.
Here is a screenshot showing the examples tables rendering but the parameters not..
If Unit tests are included in the result sources the statistics currently being reported are incorrect.
Details to be added soon...
We need to read one set of non-BDD json files and publish a single merged report with both BDD and non-BDD reports.
Problem:
Generator
is currently producing the html file as well as returning an object ReportConsole
. This breaks the single responsibility, is not easily testable as the file generator also contains the side effect, and also forces the file generation which could be optional in case only the console output is required.
Solution:
Break this in 2 different steps so each client may use only what is necessary or both if that's the case.
java -jar donut-1.2.2-one-jar.jar -s cucumber:C:\Users\VDI023\Desktop\ECS\seleniumDemo\target\cucumber-reports\Cucumber.json -n seleniumDemo
JarClassLoader: Warning: org/slf4j/impl/StaticLoggerBinder.class in lib/logback-classic-1.1.2.jar is hidden by lib/slf4j-simple-1.6.4.jar (with different bytecode)
JarClassLoader: Warning: org/slf4j/impl/StaticMarkerBinder.class in lib/logback-classic-1.1.2.jar is hidden by lib/slf4j-simple-1.6.4.jar (with different bytecode)
JarClassLoader: Warning: org/slf4j/impl/StaticMDCBinder.class in lib/logback-classic-1.1.2.jar is hidden by lib/slf4j-simple-1.6.4.jar (with different bytecode)
Exception in thread "main" java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.NullPointerException
at scala.collection.mutable.ArrayOps$ofRef$.length$extension(ArrayOps.scala:192)
at scala.collection.mutable.ArrayOps$ofRef.length(ArrayOps.scala:192)
at scala.collection.SeqLike$class.size(SeqLike.scala:106)
at scala.collection.mutable.ArrayOps$ofRef.size(ArrayOps.scala:186)
at report.donut.gherkin.processors.JSONProcessor$.getValidFiles(JSONProcessor.scala:49)
at report.donut.gherkin.processors.JSONProcessor$.loadFrom(JSONProcessor.scala:22)
at report.donut.ResultLoader$CucumberResultLoader.load(ResultLoader.scala:24)
at report.donut.Generator$$anonfun$loadResultSources$1.apply(Generator.scala:83)
at report.donut.Generator$$anonfun$loadResultSources$1.apply(Generator.scala:82)
at scala.collection.immutable.List.foreach(List.scala:381)
at report.donut.Generator$.loadResultSources(Generator.scala:82)
at report.donut.Generator$$anonfun$createReport$2$$anonfun$apply$1.apply(Generator.scala:61)
at report.donut.Generator$$anonfun$createReport$2$$anonfun$apply$1.apply(Generator.scala:61)
at report.donut.performance.PerformanceSupport$class.timed(PerformanceSupport.scala:13)
at report.donut.Generator$.timed(Generator.scala:15)
at report.donut.Generator$$anonfun$createReport$2.apply(Generator.scala:60)
at report.donut.Generator$$anonfun$createReport$2.apply(Generator.scala:59)
at scala.util.Either$RightProjection.flatMap(Either.scala:522)
at report.donut.Generator$.createReport(Generator.scala:59)
at report.donut.Generator$.apply(Generator.scala:33)
at report.donut.Boot$.delayedEndpoint$report$donut$Boot$1(Boot.scala:85)
at report.donut.Boot$delayedInit$body.apply(Boot.scala:8)
at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:381)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at report.donut.Boot$.main(Boot.scala:8)
at report.donut.Boot.main(Boot.scala)
... 6 more
C:\Users\VDI023\Downloads>
The latest version of Cinnamon (0.2.0) uses a newer version of Cucumber (4.2.6). The cucumber.json
file that gets generated puts any output (when using Scenario.write
within an after hook) into the json structure elements.[index].after.[index].output
. Donut does not output this custom output as it is looking in the older location of elements.[index].steps.[index].output
which was in the older version of Cucumber (with Cinnamon 0.1.2).
and
We rely on the after hooks to write custom output when a scenario fails and can no longer see this output with the latest Cinnamon/Cucumber. This issue probably extends to writing to scenarios in before hook too (but haven't checked).
On the Donut Feature summary page, the search function above the feature summary table works only for features currently.
Recently, while presenting Donut as living documentation to a client, they asked if they can search for scenarios. Currently, we can only search for a feature name, but not the scenario names.
Had a case where setting the output from cucumber to UTF-8 was causing donut to fail (we were testing against special characters). For now we managed to solve this error by setting Java's default file encoding in the JAVATOOL_OPTIONS variable to use UTF-8, but it would be useful if it was also a parameter that could be passed in.
First of all, donut is the cool test report I have seen.. Great effort!! Any thought about adding support to "Jbehave" ?
Please can you add Test Automation ID's to the Donut HTML. This will aid in the creation of the UI Acceptance tests.
The initial requirement would be for any interaction elements e.g links, buttons and dropdowns)
As a first trial it would be good to add an ID to the Donut Chart Failures link if possible
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.