GithubHelp home page GithubHelp logo

cezarypiatek / nscenario Goto Github PK

View Code? Open in Web Editor NEW
67.0 5.0 5.0 481 KB

Dead simple library for annotating steps of test case scenarios.

License: MIT License

C# 8.53% HTML 87.94% CSS 0.24% TypeScript 3.29%
testing dotnet testcase integration-testing component-testing

nscenario's Introduction

NScenario

NuGet

Dead simple, test framework independent, without any magic, a library for annotating steps of test case scenarios. Please read Readable and clear tests for ASP.NET Core services article for better exaplanation and example use cases.

This library was discussed during ASP.NET Community Standup 2021-08-24

How to install

NScenario is distribute as a nuget package NScenario

How to use it

Just create an instance of NScenario.TestScenario class and start annotating your test steps by wrapping it in Step method call. You can create TestScenario instance manually by providing a desired composition of IScenarioStepExecutor instances or simply by calling TestScenarioFactory.Default() method.

Example test can look as follows:

[Test]
public async Task should_activate_community_supporter_license_when_eligible()
{
    using var driver = await LicenseServerTestDriver.Create();
    var scenario = TestScenarioFactory.Default();

    var activationData = new
    {
        email = "[email protected]",
        licenseKey = "WTKP4-66NL5-HMKQW-GFSCZ"
    };

    await scenario.Step("Import supporters", async () =>
    {
        await driver.ImportSupporters("BuyMeCoffee", new[] { "[email protected]", "[email protected]", activationData.email });
    });

    await scenario.Step("Register purchase for supporter email", async () =>
    {
        await driver.RegisterPurchaseWithCoupon("Extension for VS2017", activationData.email, activationData.licenseKey, "OssSupporter");
    });

    await scenario.Step("Activate the license with supporter email", async () =>
    {
        var activationResult = await scenario.Step("Call active license endpoint" () => 
        {
            return await driver.ActivateLicense(activationData.email, activationData.licenseKey);
        });
        
        await scenario.Step("Verify that license activated properly", () =>
        {
            Assert.AreEqual(true, activationResult.Activated);
            Assert.AreEqual("Unlimited", activationResult.Capabilities["VsVersion"]);
        });
    });
}

Console output

SCENARIO: should activate community supporter license when eligible

STEP 1: Import supporters
STEP 2: Register purchase for supporter email
STEP 3: Activate the license with supporter email
    STEP 3.1: Call active license endpoint
    STEP 3.2: Verify that license activated properly

Benefits:

  • Obvious way to enforce step descriptions
  • More readable test scenario
  • Sub-scopes for repeatable steps
  • Readable output that facilitates broken scenario investigation

Console output

Some test runners are hijacking console output and provide a custom stream for logging. By default NScenario is writing scenario description to the console, but this can be overridden by providing a custom TextWriter stream to TestScenarioFactory.Default() method.

Example setup for NUnit

public class MyTests
{
    [Test]
    public void sample_test_case()
    {
        var scenario = TestScenarioFactory.Default(TestContext.Progress);    
    }
}

Example setup for XUnit

public class XUnitOutputAdapter : TextWriter
{
    private readonly ITestOutputHelper _output;
    public XUnitOutputAdapter(ITestOutputHelper output) => _output = output;
    public override void WriteLine(string? value) => _output.WriteLine(value);
    public override Encoding Encoding { get; }
}

public class MyTests
{
    private readonly ITestOutputHelper _output;
    public MyTests(ITestOutputHelper output) => this._output = output;
    
    [Fact]
    public void sample_test_case()
    {
        var scenario = TestScenarioFactory.Default(new XUnitOutputAdapter(_output));
    }
}

More info about capturing console output in XUnit

Global setup for scenario output

Test scenario output can be configured globally by setting TestScenarioFactory.DefaultScenarioOutputWriter.

Example using Module initializer :

using NScenario.OutputWriters;
using NScenario.StepExecutors;

public static class GlobalSetup
{
    [System.Runtime.CompilerServices.ModuleInitializer]
    public static void Setup()
    {
        TestScenarioFactory.DefaultScenarioOutputWriter = new StreamScenarioOutputWriter(TestContext.Progress);
    }
}

Example using SetUpFixture for NUnit

You should put that code under the default namespace:

using NScenario.OutputWriters;
using NScenario.StepExecutors;
using NUnit.Framework;

[SetUpFixture]
public class AllTestsSetup
{
    [OneTimeSetUp]
    public void GlobalSetup()
    {
        TestScenarioFactory.DefaultScenarioOutputWriter = new StreamScenarioOutputWriter(TestContext.Progress);
    }
}

Exporting scenario transcription

You save the test scenario transcription as Markdown (with option to export as HTML) using MarkdownFormatterOutputWriter.

Sample setup with exporting scenario transcription to a file:

using NScenario.OutputWriters;
using NScenario.StepExecutors;
using NUnit.Framework;

[SetUpFixture]
public class AllTestsSetup
{
    private readonly MarkdownFormatterOutputWriter _reportWriter = new (title: "Sample tests with NScenario", currentTestIdentifierAccessor: ()=> TestContext.CurrentContext.Test.ID);

    [OneTimeSetUp]
    public void GlobalSetup()
    {
        TestScenarioFactory.DefaultScenarioOutputWriter = new ComposeScenarioOutputWriter(new IScenarioOutputWriter[]
        {
            //INFO: Configure live reporting to console with NUnit
            new StreamScenarioOutputWriter(TestContext.Progress),
            //INFO: Configure collecting transcription as markdown
            _reportWriter
        });

    }

    [OneTimeTearDown]
    public void GlobalTearDown()
    {
        // INFO: Save the raw Markdown to a file
        _reportWriter.Save("Report.md");
        //INFO: Export the markdown to HTML file
        _reportWriter.ExportToHtml("Report.html");
    }
}

There's also an option to generate a nice html report for all test scenarios. Just invoke TestScenarioFactory.GetAllExecutedScenarios().SaveAsReport("AllReports.html"); in your global teardown to get a report like the one below:

image

This report browser supports links to scenario steps definition. To make it work, you need to set the following msbuild properties (or environment variables):

  • RepositoryUrl
  • SourceRevisionId

Test scenario title

Test scenario title is generated by removing underscores and splitting camel/pascalcase string from the test method name ([CallerMemberName] is used to retrieve that name). This allows for immediate review of the test name (I saw many, extremely long and totally ridiculous test method names. A good test method name should reveal the intention of the test case, not its details). You can always override the default title by setting it explicitly during test scenario creation (especially useful for parametrized test methods):

[TestCase(false)]
[TestCase(true)]
public async Task should_present_basic_scenario_with_explicit_title(bool someFlag)
{
    var scenario = TestScenarioFactory.Default(title: $"some scenario when flag set to '{someFlag}'");

    await scenario.Step("This is the first step", () =>
    {
        // Here comes the logic
    });

    await scenario.Step("This is the second step", () =>
    {
        // Here comes the logic
    });

    await scenario.Step("This is the third step", () =>
    {
        // Here comes the logic
    });
}

NScenario is prefixing scenario title with SCENARIO: prefix and every step is prefixed with STEP. If you are writing step descriptions in other languages than English, you can override those prefixes by specifing them explicitly why calling TestScenarioFactory.Default() method.

var scenario = TestScenarioFactory.Default(scenarioPrefix: "SCENARIUSZ", stepPrefix: "KROK");

Why not XBehave.net

xBehave.net is the XUnit extension so it can be used only with XUnit based tests. In my opinion, it is also quite cryptic (string extension methods called with single letter might not obvious) and a little bit over-complicated. BUT THIS IS MY PERSONAL OPINION

nscenario's People

Contributors

cezarypiatek avatar havret avatar matwitkos avatar osypchuk avatar simoncropp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

nscenario's Issues

Scenario name resets when using child functions

Readme is a bit obsolete - TestScenario do not exist anymore so maybe I am a bit off.

Provided I am building test scenario using TestScenarioFactory.Default() my expectation is that scenario definition is completed. Thus I am expecting the Name to be defined exactly at this moment.

Consider following example:

public class NScenarioDemo
    {
        [Fact]
        public async Task Main_Test_Function()
        {
            var scenario = TestScenarioFactory.Default();
            var response = await scenario.Step("First step", () => { 
                return new HttpResponseMessage(); });
         
            await HelperFunction(scenario, response);
            await scenario.Step("3rd step", () => { });
        }

        private async Task HelperFunction(ITestScenario scenario, HttpResponseMessage result)
        {
            await scenario.Step("Assert result code valid", () => { });
            await scenario.Step("Assert result content is valid", () => { });
        }
    }

My expected result would be something like:
SCENARIO Main_Test_function
STEP 1 First Step
STEP 2 Assert Result code valid
STEP 3 Assert result content is valid
STEP 4 3rd step

However, the actual result is
SCENARIO Main Test Function

STEP 1: First step
SCENARIO HelperFunction

STEP 1: Assert result code valid
STEP 2: Assert result content is valid
STEP 2: 3rd step

as you see:

  • new scenario added out of the blue
  • nesting is not happening.

XUnit: Assert in action is not failing test

Provided that my step action is non-async Step is always succeeding, even when Assert located in it is triggered.

I believe it is not a bug in NScenario but somehow related to xUnit way to implement assertions.
(I tried to debug through implementation but I do not see anything wrong)

Here are more detailed steps to reproduce an issue.
Let's start from defining faulty action with Assert in it:

private Action _AssertAction = () =>
{
    Assert.True(false);
};

I am expecting step which is using such action to fail. Unfortunately, it is succeeding:

[Fact]
public async Task Assert_in_step_action_not_throwing_exception()
{
    var scenario = TestScenarioFactory.Default();
    await scenario.Step("I am assert which should fail", _AssertAction);
}

Again, it is not your fault but I would be curious to hear your feedback, maybe it is easy to fix
Full source code: https://github.com/Osypchuk/NScenario/blob/AssertBug/src/NScenario.XUnitDemo/UnitTest1.cs

Generate report for complex test structure

          HI Cezary,

I trust you are fine.
Imagine you have a set of tests structured in folder and subfolders, which could provide together with the test class names semantic meaning to the report.
What would be the way to proceed to incorporate these structural elements in the md report?
Thanks !
paul.

Originally posted by @paulvanbladel in #10 (comment)

xunit - issue with output

Hi, I checked you code samples, saw that you passed nothing different to your scenario when using xunit, but unfortunately I'm not being able to see the output shown in the readme.

I setup the nunit framework to test it and it worked fine.

Is there any other thing that is need to be passed to the TestScenarioFactory when using xunit?

Option to create md based report

For smooth communication with the business stakeholders, it could be useful to be able to generate from the test output a .md file per scenario. A build server task could publish then the md material to a place where it has valuable meaning for stakeholders.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.