allure-framework / allure-ruby-commons

Deprecated, use https://github.com/allure-framework/allure-ruby instead

[DEPRECATED]

Please follow https://github.com/allure-framework/allure-ruby

Allure Ruby Adaptor API

This is a helper library containing the basics for any ruby-based Allure adaptor. Using it you can easily implement the adaptor for your favourite ruby testing library or you can just create the report of any other kind using the basic Allure terms.

Setup

Add the dependency to your Gemfile

 gem 'allure-ruby-adaptor-api'

Advanced options

You can specify the directory where the Allure test results will appear. By default it would be 'gen/allure-results' within your current directory.

    AllureRubyAdaptorApi.configure do |c|
      c.output_dir = "/whatever/you/like"
    end

Usage examples

    builder = AllureRubyAdaptorApi::Builder
    builder.start_suite "some_suite", :severity => :normal
    builder.start_test "some_suite", "some_test", :feature => "Some feature", :severity => :critical
    builder.start_step "some_suite", "some_test", "first step"
    builder.add_attachment "some_suite", "some_test", :file => Tempfile.new("somefile")
    builder.stop_step "some_suite", "some_test", "first step"
    builder.start_step "some_suite", "some_test", "second step"
    builder.add_attachment "some_suite", "some_test", :step_id => "second step", :file => Tempfile.new("somefile")
    builder.stop_step "some_suite", "some_test", "second step"
    builder.start_step "some_suite", "some_test", "third step"
    builder.stop_step "some_suite", "some_test", "third step", :failed
    builder.stop_test "some_suite", "some_test", :status => :broken, :exception => Exception.new("some error")
    builder.stop_suite "some_suite"

    # This will generate the results within your output directory
    builder.build!

allure-ruby-commons's Issues

Add bindings for Label type 'tags' which is a core part of Cucumber.

Hey,

Some folks who are using the allure-cucumber repo are very interested in getting cucumber tags to show in the allure XML report. The add that functionality, it would require allure-core be updated to support a label type called 'tags'. We would then extend that api in allure-ruby-adaptop-api

From this attached issue, I can see the functionality is available in allure-core-java.
allure-framework/allure1#764

Allure Core 2.0 Support

Hey folks,

Figured I would start the conversation about getting ready for Allure Core 2.0. Is there anyone on the yandex team who is gearing up to port the ruby adaptor? If not, please let us know how the community can step up and help.

bump version number to reflect logging level change?

Shouldn't the version number be updated for the change in logging level?

Using tags or labels as in the demo report

I would like to leverage tags and labels usage as like this :

can you guide me for implementation, I can also send a pull request

Unable to generate ruby based allure reports

Failing to generate alllure reports. Trying out a basic example. Am I missing something here?
Sample test code

require 'test/unit'
require 'allure-ruby-adaptor-api'

class StringTest < Test::Unit::TestCase

  def test_length
    s = "Hello, World!"
    assert_equal(13, s.length)
  end

end

AllureRubyAdaptorApi.configure do |c|
  c.output_dir = "/reports"
end

builder = AllureRubyAdaptorApi::Builder
builder.start_suite "StringTest", :severity => :normal
builder.stop_suite "StringTest"
builder.build!

retry usage

I would like to use the cucumber --retry feature. Only thing that is required to publish in xml is to generate each retry result as one test-case . Can you please guide me how to use the library so I can fork allure-cucumber and update

Inspec/ServerSpec integration

Couple of Ruby tools, Inspec and ServerSpec, have occupied a very interesting niche: infrastructure testing that validates target machine for being compliant to set of rules. While both of them are based on RSpec, and generally RSpec adapter should fit in somehow, they are often used under not so common conditions (Test Kitchen that allows only declarative configuration or simply run by people that are not familiar with Ruby enough), what restricts people from using Allure reports - and they would come very handy since security-concerning Inspec check may consist of thousands of assertions, and usually those tools are used against of fleet of machines. That would be terribly lovely to add a cherry on top of infrastructure automation stack Chef guys has made, because currently i can validate may configuration against all possible distros i can imagine, but it's quite difficult to find who exactly has failed (especially if tests are run concurrently, which is typical for test Test Kitchen with tens of target distros).

So, this issue suggests

• Finding a way to add direct support for Inspec/ServerSpec, using RSpec formatter, if possible
• Implementing it
• Possibly adding some changes directly in Inspec and/or ServerSpec (since, by default, declarative-only Inspec configuration may only choose one of built-in formatters)

There is a possibility that i'll do it one day myself, but past experience tells me that's not what i can be sure about.
Feel free to ask me about details of Inspec/Test Kitchen usage, if you need any.

The report file is always attached to 1st step if there are duplicated steps.

I have a scenario that calls one step many times
Ex:
.......(when I do something).......
"Then I should see the "HomePage"
.......(when I do something).......
"Then I should see the "HomePage"
.......(when I do something).......
"Then I should see the "HomePage"

If the step fails I will attach some log file and screenshot to my step.
The problem is if this step fail at the 3rd time called, all my report files will be attached to the 1st step.
This is not correct and cause difficulty in my debugging.
Anyone got the same problem and anyway to fix this?
Thanks.

missing parentheses error

title = opts[:title] || File.basename file

should be

title = opts[:title] || File.basename(file)

confusing type in add_attachment method

I assume the type you are expecting for parameter file in option opts is File. But what does this mean? File type does not have basename instance method. Am I missing something?

Allure/Watir - HTML Report shows as 0 Test Cases

This is the similar issue as mentioned in here. I'm posting it again here as the other thread is closed.

It executes fine when I run the below code from Visual Studio Code with Ruby extension. It also generates the XML file. Below is the ruby code.

require 'aws-sdk'
require 'allure-ruby-adaptor-api'
require 'test/unit'

class StringTest < Test::Unit::TestCase
    def test_length
      s = "Hello, World!"
      assert_equal(13, s.length)
    end
end

builder = AllureRubyAdaptorApi::Builder
builder.start_suite "StringTest", :severity => :normal
builder.start_test "StringTest", "test_length", :severity => :critical
builder.stop_test "StringTest", "test_length", :status => :broken, :exception => Exception.new("some error")
builder.stop_suite "StringTest"
builder.build!

The XML looks like below. I've multiple questions here.

1. Why does the test status shows as broken in the XML below. Per the above code ":status => :broken", broken is supposed to be just the default value, but should be overwritten by the actual test result, in this case - Passed. Is it not right?

2. Why is there a failure message even though the test case passed?

3. When this XML is converted into the HTML report using Command Line, the HTML report is showing as 0 test cases. Why? Actually, when the allure report is generated from the below XML, using the command "allure generate --clean", all the json files generated under allure-report/data folder are empty like ""children" : [ ]". Please let me know what am I mising.

<?xml version="1.0"?>
<ns2:test-suite xmlns="" xmlns:ns2="urn:model.allure.qatools.yandex.ru" start="1505627368257" stop="1505627368257">
  <name>StringTest</name>
  <title>StringTest</title>
  <test-cases>
    <test-case start="1505627368257" stop="1505627368257" status="broken">
      <name>test_length</name>
      <title>test_length</title>
      <failure>
        <message>some error</message>
        <stack-trace/>
      </failure>
      <steps/>
      <attachments/>
      <labels>
        <label name="severity" value="critical"/>
        <label name="thread" value="20044420"/>
        <label name="host" value="GUNDE1"/>
      </labels>
      <parameters/>
    </test-case>
  </test-cases>
  <labels>
    <label name="severity" value="normal"/>
    <label name="thread" value="20044420"/>
    <label name="host" value="GUNDE1"/>
  </labels>
</ns2:test-suite>

I'm using Allure 2.3.5 and allure-ruby-adaptor-api-0.7.0

undefined method `debug' for nil:NilClass in `lib/allure-ruby-adaptor-api/builder.rb:87`

version: 0.6.8
🐛 wrong use @LOGGER, should replace with LOGGER

when example stoping, will raise error:

RSpec::Core::MultipleExceptionError: undefined method `debug' for nil:NilClass

undefined method `debug' for nil:NilClass

undefined method `debug' for nil:NilClass

undefined method `debug' for nil:NilClass

undefined method `debug' for nil:NilClass

undefined method `debug' for nil:NilClass

undefined method `debug' for nil:NilClass

Attachment files overwritten for identical screenshots with rspec-retry

Hi,

when running Allure reports with rspec retry Tests may fail with identical screenshots causing added attachments to overwrite the previously attached file.

This is caused by using SHA256 for determining a file name for an attachment. It would be safer to use a different approach for generating unique attachment filenames like UUID.

I'll add a PR with the required changes.

Replace 'puts' with logger, so user would be able to turn logging off

Add LICENSE

See allure-core for example.

Add time argument to start_step and stop_step methods

I have few steps in cucumber-adaptor which I can't execute in realtime so I would want to be able to pass time as argument to start_step and stop_step methods.

Making it optional won't break anything in adaptors

    def start_step(suite, test, step, time  = Time.now)
        MUTEX.synchronize do
          puts "Starting step #{suite}.#{test}.#{step}"
          self.suites[suite][:tests][test][:steps][step] = {
              :title => step,
              :start => timestamp(time),
              :attachments => []
          }
        end
    end

   def stop_step(suite, test, step, status = :passed, time = Time.now)
        MUTEX.synchronize do
          puts "Stopping step #{suite}.#{test}.#{step}"
          self.suites[suite][:tests][test][:steps][step][:stop] = timestamp(time)
          self.suites[suite][:tests][test][:steps][step][:status] = status
        end
   end

@smecsia - If this looks ok to you then I will send a pull request.