theramis / snapper

Bringing Jest-esque Snapshot testing to C#

Snapper V2

Bringing Jest-esque Snapshot testing to C#

Snapper is a NuGet library which makes snapshot testing very easy in C#. Snapshot testing can simplify testing and is super useful when paired with Golden Master testing or contract testing APIs. It is very heavily based on Jest Snapshot Testing framework.

See https://theramis.github.io/Snapper/ for the full documentation.

What is Snapper?

The best way to describe what Snapper does by going through an example. Imagine you have the following test where are retrieving a user from a service.

[Fact]
public void MyTest()
{
  var myUser = _userService.GetUser(id);
  Assert.Equal("MyUser", myUser.Username);
  Assert.Equal("[email protected]", myUser.Email);
  Assert.Equal("myhash", myUser.PasswordHash);
  ...
  ...
}

As you can imagine the assertion steps in the test can get very long and hard to read. Now lets see what the test would look like if Snapper is used.

[Fact]
public void MyTest()
{
  var myUser = _userService.GetUser(id);
  myUser.ShouldMatchSnapshot();
}

The test above is now asserting the myUser object with a snapshot stored on disk. Snapper helps you create this snapshot at the beginnging (see Quick Start).

This is the basis of snapshot testing. The idea being a baseline is first generated (in this case a json file which is our snapshot) and then everytime the test runs the output is compared to the snapshot. If the snapshot and the output from the tests don't match the test fails!

As you can see using Snapper the test is now much smaller and easier to read but that's not all. Using Snapper brings with it a lot more benefits!

Why use Snapper?

Benefits of using Snapper/snapshot testing vs traditional assertions

• Much easier to read - It's quite common to have a large list of assertions which can be hard to read. Snapper makes your tests a lot shorter and easier to read!
• Very difficult to miss properties to assert - It's hard to validate that all properties have are being asserted using traditional assertions. By using Snapper the whole object asserted which means all properties are always asserted, so there is no chance of missing properties!
• Captures changes to the object being asserted - It's quite common to add new properties to our objects over time. e.g. Adding FirstName to the myUser object above. Using traditional assertions the test would still pass and it's easy to forget to update the test. Using Snapper the test would immediately fail since it's a change in the system and the developer should verify if the change was expected!
• Much quicker to write tests - Writing all those assertions above can be time consuming. With Snapper a json file is generated with the object which the developer can quickly verify!

When to use Snapper?

Use cases where Snapper/snapshot testing really shines

• Contract testing - Testing your contract has not changed is a major part of maintaining any library/API. Snapper is excellent for this! Using Snapper any changes to the contract would immediately fail a test which lets the developer know that they might be breaking a contract they didn't expect.
• Asserting complex objects - Sometimes you have to assert complex objects which can be hard and time consuming to get right. Snapper makes this easy and quick.
• Golden Master testing - Golden master testing is a technique where you capture the behaviour of a system. Snapper is perfect for this as you can easily assert the behaviour without the complex setup and assertions. Snapper would also fail as soon as the behaviour of the system changes

The use cases above are just some of the examples I've found where Snapper is super useful. Feel free to try them in other situation you think would be useful.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Florian Gather 💻 🤔	Tomas Bruckner 💻	Michael Kriese 💻 🤔 🐛	Taylor Kimmett 💻	Patrick Lehner 🐛	Piotr Litwinski 🐛	Warren Ferrell 💻
Aaron Roberts 💻 🤔	jaytulk 💻	Alberto Martín 🐛	Peter Huang 💻	J Teeuwissen 💻

This project follows the all-contributors specification. Contributions of any kind welcome!

snapper's Issues

Approach to Environment.NewLine

Is your feature request related to a problem? Please describe.
It seems as though when one contributor creates a snapshot with Linux, and I create a snapshot with Windows, the snapshots will never be in sync due to different Environment.NewLine settings.

Describe the solution you'd like
Write snapshots to /.snapper/linux/** and /.snapper/windows/**

Developers can then use tools like Windows Subsystem for Linux to generate snappers in alternative environments through the use of docker images and volumes.

Describe alternatives you've considered

1. Manually fixing Environment.NewLine prior to comparing before and after, but this is error prone.
2. Running tool to convert \r\n to \n
3. Stop (trying to) using Snapper :)

Feature Request: when update snapshot is on test should be Inconclusive/Fail

FYI, I haven't updated from Snapper 1 yet.

I'm worried that an inexperienced Developer may commit
[assembly: UpdateSnapshots]
Then test would always pass.

Making the Test inconclusive... or outright failing would stop any mistakes

Serializing non json values

Started using Mongo and considering refactoring some manually written queries to Linq But I want to show that the queries don't change after being refactored. The issue is that the default mongo syntax is not valid json.

Example:
I want to show that the following linq produces the below existing query

from u in userCollection
where u.Id == ctx.UserId && u.CustomerId == ctx.CustomerId && u.IsDeleted == false 
select new { _id = 0, UserId = u.Id, u.Email }

[
        {
            $match : { 
                'CustomerId' : ObjectId("000000000000000000000001"), 
                '_id' : ObjectId("000000000000000000000001"),
                'IsDeleted': false
            }
        },
        { 
            $project : { 
                _id : 0,
                UserId: '$_id', 
                Email : '$Email'
            }
        }
    ]

and as it turns out, it does, but if you try to match that string with command.ShouldMatchSnapshot() you get a MalformedJsonSnapshotException from the ObjectId line. So you need to do new { command }.ShouldMatchSnapshot()
Which will create a snapshot which is not human readable

{"command":"{\n  \u0022aggregate\u0022 : \u0022User\u0022,\n  \u0022pipeline\u0022 : [{\n      \u0022$match\u0022 : {\n        \u0022_id\u0022 : ObjectId(\u0022000000000000000000000001\u0022),\n        \u0022CustomerId\u0022 : ObjectId(\u0022000000000000000000000001\u0022),\n        \u0022IsDeleted\u0022 : false\n      }\n    }, {\n      \u0022$project\u0022 : {\n        \u0022_id\u0022 : 0,\n        \u0022UserId\u0022 : \u0022$_id\u0022,\n        \u0022Email\u0022 : \u0022$Email\u0022\n      }\n    }],\n  \u0022cursor\u0022 : { },\n  \u0022$db\u0022 : \u0022Data\u0022\n}"}

Yaml serialization though produces the following human readable result

command: >-
  {
    "aggregate" : "User",
    "pipeline" : [{
        "$match" : {
          "_id" : ObjectId("000000000000000000000001"),
          "CustomerId" : ObjectId("000000000000000000000001"),
          "IsDeleted" : false
        }
      }, {
        "$project" : {
          "_id" : 0,
          "UserId" : "$_id",
          "Email" : "$Email"
        }
      }],
    "cursor" : { },
    "$db" : "Data"
  }

I think the best solution is to expose ISnapshotStore via an attribute so that users can specify any serialization type they want. Don't mind doing the work but wanted to get people's thoughts first.

Would also serve as a solution to #79
, #83

Specify path for saving snapshots

Since tests sometimes have a pretty long method name and are also deeply nested in folders, PathTooLongExceptions can easily occur.

So it would be good if you can specify a path to store Sanpshots.

Add contributors

False positive SupportedTestMethodNotFoundException with xUnit

This causes invalid SupportedTestMethodNotFoudnException with xUnit:

public class SnapperSnapshotsPerMethodTests
{
        private static async Task Helper(Func<Action<int>, Task> test)
        {
            foreach (var _ in new[] {1, 2})
            {
                await test(o => {});
            }
        }
        
        [Fact]
        [MethodImpl(MethodImplOptions.NoInlining)]
        public async void SnapshotsMatch()
        {

            await Helper(async x =>
            {
                x(1);
                await Task.Delay(100);

                var snapshot = new
                {
                    TestValue = "value"
                };

                snapshot.ShouldMatchSnapshot();
            });
        }
}

This is simplification of my real use case, which is I am using xUnit for API testing. I have a bunch of user roles in the system and I want to iterate all the roles and check that the API call works for all of them.

I wanted to integrate this great library but this is a deal breaker for me.

Is there any workaround?

Have you looked into using CallerMemberName instead of StackTrace (this should be much faster and should not need removing inline optimization)?

Or how about you could specify name of snapshot file in some kind of annotation?

Allow exclusion of fields

I have come back to this library after a while and I swear when I first encountered it back in October 2019, prompted by learning about jest in when learning Vue. Js, there was a capability to exclude (or ignore the value of) certain fields from the snapshot check.

I can't find any notion of this.

I think it would be a useful feature to be able to exclude fields which change.

For example. And the wording and specifics of my method is to illustrate the point and is not that important.

[Fact]
public void MyTest()
{
var myUser = _userService.CreateUser(new User() );
myUser.ShouldMatchSnapshot()
.Exclude((user) => user.Id)
}

I think if you were able to load the snapshot and apply the exclusion and apply the exclusion to the
In memory object and then compare, this might not be too hard to achieve. The typing would be provided by the type available coming into the first extension method and then maybe simply using anonymous types or using reflection you could skip over the field to exclude. I think it might get harder with nested properties though.

EDIT: actually the concept of exclusion Is probably the wrong
Or different thing to what I am after it's more like ignoring the value of a field or defaulting a a changeable field to a fixed value....

I guess a workaround to this would be to do the exclusion/defaulting on the model we want the snapshot of and Then create the snapshot.

So do what I said about exclusion/ignore but in place and then create the snapshot.. I could hide this in my own set of wrapper methods for the time being I guess.

System.ArgumentNullException : Value cannot be null

I know now this error is because Release mode, but i think we should handle this more gracefully.

Error
System.ArgumentNullException : Value cannot be null.
Parameter name: path1
Stacktrace
at System.IO.Path.Combine(String path1, String path2, String path3)
at Snapper.Core.SnapshotIdResolver.ResolveSnapshotId(String partialSnapshotName)
at Snapper.Nunit.NUnitSnapper.MatchChildSnapshot(Object snapshot, String childSnapshotName)
at Snapper.Nunit.EqualToSnapshotConstraint.ApplyTo[TActual](TActual actual)
at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)
at VisualOn.Test.Cases.DataSourceExcelTest.TestExcel()

Support NExpect

Is your feature request related to a problem? Please describe.
I like to use snapshot with the NExpect¹ framework.

Describe the solution you'd like
Add a new library to support NExpect like NUnit and others

Describe alternatives you've considered
Manual write the wrapper for multiple projects.

Additional context

Footnotes

1. https://github.com/fluffynuts/NExpect ↩

`System.NotImplementedException` when calling `ShouldMatchSnapshot`

Issue

I received a bug report (in person) and for some reason on this persons computer when calling ShouldMatchSnapshot() they are receiving a not implemented exception.
I wasn't able to replicate the same bug on my machine even with the exact same code. Neither were a bunch of other people.

We dug into the exception a little bit more on the computer that was receiving this exception. We found that while going through the stack frames one of the methods throws a not implemented exception when getting the custom attributes for it. Here is the line of code which fails

Snapper/project/Snapper/Core/TestMethodResolver/TestMethods/BaseTestMethod.cs

Line 18 in b6f96b5

var attribute = BaseMethod?.CustomAttributes.FirstOrDefault(a =>

The following screenshot shows the method which was causing this issue.

Unfortunately we could not narrow down why this issue was only occurring for this person and on that specific computer.
We also couldn't figure out which method was causing this issue and if there was anything special about it. (We had limited time on the machine which had this issue)

Details

• Xunit + Specflow test
• Snapper v2.0.0-beta3
• Windows 10 OS
• Tried on both VS 2017 and VS 2019

Much better documentation needed

Hi,

Someone asked if we could include Snapper in an open source project I maintain, and I have so far rejected this.

As some basic feedback:

1. nuget.org page says to go to the Project Details page for more information
   
2. the Project Details page repeats what the nuget.org page says, more or less
   
3. This creates a circular experience for people wanting to figure out what value Jest or Snapper brings to the table

As some deeper feedback:

It seems like Jest saves files somewhere to store these snapshots. Yet I am incredibly frustrated that I cannot find any documentation on how Snapper saves these snapshots, and how it even computes the first snapshot to "bootstrap" the fact checking process. What happens if the snapshot is missing? How do these get checked into source control? etc.

As some other feedback:

It's great you are so honest that SnapperV1 was bad, and that you are learning how to build a library, but it also doesn't really make me excited to want to beta test your product if your goal is to learn by doing and you'll make unilateral changes based on those learnings. If you're going to make statements like this, please write up an actual Lessons Learned document describing what was incorrect in V1 and what V2 seeks to address. Yes, it requires discipline to do this and takes time that you could otherwise spend coding. Yes, it is worth doing because it will help clarify your purpose and drive towards what you need to actually spend time on.

Better support for theory tests in XUnit and NUnit

Currently in Xunit and Nunit for theory tests you need to specify a snapshot name so that each execution of the test can be stored in a different snapshot.

Look into the possibility of figuring out which test case is currently running and making a snapshot name based on the test case.

e.g a test like this should make two snapshots automatically and it should know which one is which.

[Theory]
[InlineData(1, 2, 3)]
[InlineData(4, 5, 9)]
public void CanAdd(int value1, int value2, int expected)
{
    var result = value1 + value2;
	XUnitSnapper.Snap({ "result": result});
}

The snaps could be potentially named CanAdd_123 and CanAdd_459 based on the parameter values.

Stack overflow exception Testhost.exe stopped working

Error: The active test run was aborted. Reason: Test host process crashed : Process is terminating due to StackOverflowException.

Configuration:

<PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
    <Platforms>x64</Platforms>
    <IsPackable>false</IsPackable>
</PropertyGroup>

Usage:

        [UpdateSnapshots]
        [Fact]
        public void Set_repaid_tranche_status_return_new_object_with_status_set_to_repaid()
        {
            var tranche = new Tranche(1, Constants.Status.Current, 0, 100m);

            var repaidTranche = tranche.RepaidTranche;

            repaidTranche.ShouldMatchSnapshot();
        }

(feature) Support net45

We would like to use Snapper but we are limited to net45 /net452.

I've made a small diff to add this support. I can also send a pr if you like that idea.

snapper.txt

Plain text serialization

Is your feature request related to a problem? Please describe.
Currently snapper uses Json serialization. This is not well suited for my domain as my objects are F# types and their Json representation is extremely verbose compared to the default auto-generated ToString representation. This makes inspecting snapshots hard.

Describe the solution you'd like
I'd like to use plain text serialization for for my snapshots.

Describe alternatives you've considered
N/A

Additional context
N/A

Snapper.Json.Xunit does not work in Release builds

(This is based on Snapper v1.3.1)

Since the XUnit integration uses StackTraces to find information about the test that is actually using it (see XUnitHelper.GetCallingTestInfo()), this cannot be used in "release" builds (i.e. any release that does not include debug symbols and/or uses extensive optimization).

As the "Remarks" section on the StackTrace documentation states:

StackTrace information will be most informative with Debug build configurations. By default, Debug builds include debug symbols, while Release builds do not. The debug symbols contain most of the file, method name, line number, and column information used in constructing StackFrame and StackTrace objects.

StackTrace might not report as many method calls as expected, due to code transformations that occur during optimization.

Running tests in a release configuration might not be the default use case, but I think it's not very unusual either. Especially in CI servers, where you want to test what you deploy, building and testing the release configuration seems valid to me. (This is, in fact, how this issue bit me.)

It would be great if future versions of Snapper could use more robust ways to automatically gather the information that XUnitSnapper so conveniently provides.

A possible alternative might be the "caller information" attributes where the compiler fills in certain information about the source file containing the call site, in particular [CallerMemberName] and [CallerFilePath].

If release configuration cannot be supported (e.g. too much effort, or technically not feasible in general), it would be great if you could at least specify this as a known limitation in a prominent place (e.g. the README file).

Thanks for the work you put into this library already! It makes snapshot test really quite convenient :)

Add alternative snapshots for decimal values

I have a lot of data objects with decimal values. These are serialized in the snapshots using the default behavior of C#, which keeps all decimal places even if not necessary.

This results in failed snapshots because 10.0 is not the same as 10.000, which is just due to a change in calculations. The result should be the same though for my use case because it's the same value for further calculations.

I haven't found any settings to deal with this problem, neither in Snapper nor in Netwonsoft.Json

It would be nice though to have a way to tell Snapper that it should treat these decimal values the same so my snapshot tests are not breaking just because the value 10 can be represented with more than one decimal value.

Error when old snapshots are left behind

Currently when a test which has a snapshot is deleted, Snapper ignores the created snapshot on the disk.

It's a manual process to determine whether there are any redundant snapshot files left on the disk.

It would be nice if Snapper was somehow able to tell you there are redundant snapshot files and potentially error when this is happening during a CI build.

Add support of self referencing loop

Is your feature request related to a problem? Please describe.
Currently testing of EF Entities is not smooth because navigation properties create reference loops.

Describe the solution you'd like
Add support to configure self reference loop. May be expose Newtonsoft Json config as parameter to ShouldMatchSnapshot()?

Describe alternatives you've considered
None.

Additional context
None.

Enable changing only the snapshot filename

Is your feature request related to a problem? Please describe.
Using tests with dynamic data (e.g. DataRow, DynamicData, DataSource) I want to control the filename of the snapshots, so they do not end up overwriting each other. But I do not want to hardcode any file path, just modify the filename.

Describe the solution you'd like
I would like to use SnapshotId to specify the snapshot filename, but setting snapshotDirectory to null so the default location (_snapshots subdirectory) is used.

Describe alternatives you've considered
I am currently using child snapshots instead, which is rather tedious because the snapshots themselves are rather large and there are many data rows.
There could be a way to specify the folder using path variables or getting the path from the test file, but both are rather tedious and undocumented in Snapper.

Use Custom DSL instead of Inheriting NUnit.Framework.Is

Is your feature request related to a problem? Please describe.
Suppose I had multiple NUnit extensions similar to Snapper, I would not be able to use all Is DSL methods in a single file without using FQDN of the Is Class

ex:
Assert.That(obj, Snapper.Nunit.Is.EqualToSnapshot());

Describe the solution you'd like
I think it would be more declarative and composable to have a custom class, something specific to the domain of Snapper.

ex:
Assert.That(obj, Matches.Snapshot());
Assert.That(obj, Matches.ChildSnapshot("some-snap"));

Great work with this library, I was actually the one who opened nunit/nunit#2392 in the Nunit repo, and this is exactly what I was envisioning!

[BUG] ArgumentException: Object serialized to String. JObject instance expected.

Describe the bug
Snapper throws ArgumentException.

To Reproduce

using System;
using System.Net.Mime;
using Newtonsoft.Json;

namespace Data
{
    internal sealed class ContentTypeConverter : JsonConverter<ContentType>
    {
        public override bool CanRead => true;

        public override bool CanWrite => true;

        public override ContentType ReadJson(JsonReader reader, Type objectType, ContentType existingValue, bool hasExistingValue, JsonSerializer serializer) => new ContentType((string)reader.Value);

        public override void WriteJson(JsonWriter writer, ContentType value, JsonSerializer serializer) => writer.WriteValue(value.ToString());
    }
}

using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
using NUnit.Framework;

var jss = new JsonSerializerSettings {
    ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
    ContractResolver = new CamelCasePropertyNamesContractResolver(),
    Converters = { new Data.ContentTypeConverter() }
};
JsonConvert.DefaultSettings = () => jss;

var ct = JsonConvert.DeserializeObject<ContentType>(JsonConvert.SerializeObject("text/html"));
Assert.That(ct, Snapper.Nunit.Is.EqualToSnapshot());

Error:

 Message: 
    System.ArgumentException : Object serialized to String. JObject instance expected.
Stack Trace: 
    JObject.FromObject(Object o, JsonSerializer jsonSerializer)
    JsonSnapshotStore.StoreSnapshot(SnapshotId snapshotId, Object value)
    SnapperCore.Snap(SnapshotId snapshotId, Object newSnapshot)
    NUnitSnapper.MatchChildSnapshot(Object snapshot, String childSnapshotName)
    EqualToSnapshotConstraint.MatchSnapshot(Object actual)
    EqualToSnapshotConstraint.ApplyTo[TActual](TActual actual)
    Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)
    Assert.That[TActual](TActual actual, IResolveConstraint expression)

Additional context
Snapper Version: 2.2.1
Operation System: Windows Server 2019
.Net Version: net472, netcoreapp3.1
NUnit: 3.12.0

SnapshotDoesNotExistException on Azure Pipelines ubuntu-20.04 Agent

Describe the bug
Unsure if this affects all linux/ubuntu systems, or if it's specific to Azure Pipelines, but I receive the following when running CI Snapper tests

Snapper.Exceptions.SnapshotDoesNotExistException : A snapshot does not exist.  Run the test outside of a CI environment to create a snapshot.

Snapshots work perfectly on Windows

To Reproduce

1. Create a test project using Snapper
2. Create an azure pipeline using an ubuntu-20.04 agent
3. Add the default dotnet test task to the pipeline

Additional context
Snapper Version: 2.3.0
Operation System: ubuntu-20.04
.Net Version: 6

steps:
- task: DotNetCoreCLI@2
  displayName: Test
  inputs:
    command: test
    projects: '**/*.[Tt]ests.*csproj'
    arguments: '--configuration $(BuildConfiguration)'
    testRunTitle: 'Backend Tests'
    workingDirectory: backend

Add inline snapshots

Jest has an API to match inline snapshots.

I can see this being useful for when the snapshot is small.

See https://jestjs.io/docs/en/snapshot-testing#inline-snapshots

Support non json

Is your feature request related to a problem? Please describe.
The current snapshot functionality always uses json to store the snapshot data. When the string is not valid json, it is stored as a json string. This conversion and escaping makes it difficult to read larger chunks of e.g. serialized generated code.

Describe the solution you'd like
Id like a new api to snapshot strings (and potentially other objects) straight to a file, without any json conversion.

Solutions like Verify support this scenario, but i much prefer the snapper workflow.

Storing multiple snapshots in 1 file

The way jest currently works is by having 1 snapshot file for 1 test file. That 1 test file can have multiple tests inside it which means the snapshot file contains multiple different snapshots.

A similar thing could done in c# where all the snaps from one class would be stored together.

Advantage of this is that for tests which have a small object to snapshot you don't have to manage multiple snapshot files and they can all live there.

[BUG] ShouldUpdateSnapshot does not respect SnapshotSettings

Describe the bug
When using ShouldMatchSnapshot passing in a SnapshotSetting in cases when the stack trace cannot be used to detect the test name, the SupportedTestMethodNotFoundException gets thrown.

To Reproduce

Snapper.dll!Snapper.Core.TestMethodResolver.TestMethodResolver.ResolveTestMethod() Line 55	C#
Snapper.dll!Snapper.Core.SnapshotUpdateDecider.ShouldUpdateSnapshotBasedOnAttribute() Line 46	C#
Snapper.dll!Snapper.Core.SnapshotUpdateDecider.ShouldUpdateSnapshot() Line 30	C#
Snapper.dll!Snapper.Core.SnapperCore.ShouldUpdateSnapshot(Snapper.Json.JsonSnapshot existingSnapshot, Snapper.Json.JsonSnapshot newSnapshot) Line 40	C#
Snapper.dll!Snapper.Core.SnapperCore.Snap(Snapper.Json.JsonSnapshot newSnapshot) Line 20	C#
Snapper.dll!Snapper.Snapper.MatchSnapshot(object rawSnapshot, string childSnapshotName) Line 21	C#
Snapper.dll!Snapper.SnapperExtensions.ShouldMatchChildSnapshot(object snapshot, string childSnapshotName, Snapper.SnapshotSettings snapshotSettings) Line 22	C#

Correction I can't work-around this using environment variable, because the logic checks if either attribute or environment variable specifies to update snapshot.

I suggest changing the ShouldUpdateSnapshot() logic to respect the environment variable if it is not null, and short-circuit the decision. i.e. if environment is not set, the check the attribute. If it is explicitly set to false, don't update the snapshots.

Is it worth considering accepting a MethodInfo property in SnapshotSetting of the property?

Additional context
Snapper Version: 2.4.0
Operation System: Windows 11
.Net Version: .net core 7

Allow customising the serializer settings for Snapshots

Is your feature request related to a problem? Please describe.
Allow adjustment of the JsonSerializerSettings in JObjectHelper that is used by JsonSnapshotStore, so that snapshots can be made more readable e.g. when using custom JsonConverters they are taken into account.

e.g. an example of the current behaviour

"myList": [
    {
      "Name": "My SmartEnum",
      "Value": "My Value that should be used instead"
    }
],

after allowing to add converters for instance

"myList": [
      "My Value that should be used instead"
],

Describe the solution you'd like

Not yet full thought through, but maybe

• Allow defining a convention based class e.g. **CustomSnapshotSerialzation** that could expose either the complete JsonSerializerSettings or hooks to be called by the default implementation when adding converters.

What would be great is to at least be able to add the following

 serializerSettings.Converters.Add(new SmartEnumValueConverter<MySmartEnum, string>());

Describe alternatives you've considered

Additional context

Support for Unity Test Framework

Our team is currently trying to adopt Snapper in a Unity project, but there is no support for the Unity Test Framework.

[BUG] Snapshots do not match

Describe the bug
On CI=1 build the snapshot does not match. When update snapshot is set, snapshot does not change.

To Reproduce

Create JObject from Json:

{                                                                               
  "$id": "name=\"Excel\";itemId=\"2\";listId=\"Category\";type=\"Excel\";",     
  "Id": 2,                                                                      
  "CategoryName": "Condiments",                                                 
  "Description": "Sweet and savory sauces, relishes, spreads, and seasonings"   
}                                                                               

Add matcher

NUnit.Framework.Assert.That(obj, Snapper.Nunit.Is.EqualToChildSnapshot("test"));

Error:

  Error Message:
   "
Snapshots do not match
- Snapshot
+ Received


  "Description": "Sweet and savory sauces, relishes, spreads, and seasonings"
}
"

Additional context
Snapper Version: 2.2.1
Operation System: Windows Server 2019
.Net Version: net472, netcoreapp3.1
NUnit: 3.12.0

Remove support for .net 45

Background

Snapper and Snapper.NUnit both currently support .NET framework 4.5.
.net 45 is officially end of life by Microsoft (as of 4 months ago) and no more updates (security or feature updates) will be provided.
See https://endoflife.date/dotnetfx

Problem

A few of the other nuget packages which the Snapper solution uses have been updated and they do not support .net 45 anymore. It's likely that all of the other nuget packages will be following suit.
This means that Snapper cannot update its dependencies.

Suggested solution

Drop support for .net 45.
The oldest .net framework that is still supported is .net 462 and since Snapper already support .net standard 2.0, it is already compatible with .net 462.

Add support for Specflow

Is your feature request related to a problem? Please describe.
Snapper does not have native support for Specflow at the moment. Currently Snapper uses the underlying implementation of Specflow which can be xunit, nunit, etc
This limits the usage of Snapper in Specflow.

Describe the solution you'd like
Native support for Specflow. It should support normal specflow tests as well as Scenario Outline tests (which are essentially XUnit Theory tests).
This could be done via a new nuget package or through extending the main Snapper nuget package.

DateTimeOffset property gives false positive

At work I'm using snapper quite extensively for api contract testing and found a little bit of a problem when trying to assert against a snapshot that contains DateTimeOffset properties.

After a little bit of digging it looks like it's related to Newtonsoft.Json issue reported here JamesNK/Newtonsoft.Json#1110

Given that it's not exactly a bug (as it's expected behavior from Newtonsoft.Json I thought a slight change that would allow to configure stored snapshot deserialisation behaviour would be nice.

Usually code is much easier to follow that words, I put together a PR that addresses the issue ( #43 ).

Alternatively, JsonSerializerSettings could be passed as a parameter to ShouldMatchSnapshot method but guess it might make it a little bit more convoluted.

Having something like that would really make my life easier as currently whole snapshot needs to be stored as a string to workaround the issue....

What do you think about this proposal?

NUnit support

Hi,

Thanks for you great work!
I'm currently trying to extend Snapper to support NUnit. Are you interested in a PR? Or should I create it as a standalone package? If you'd like having it here in your code base, would you mind me moving some stuff from the XUnit implementation which is common to both Testrunners? Also I'd slightly change the SnapperCore interface to allow for easier support of NUnit style constraints.
In case you don't want to natively support NUnit, which would be perfectly fine, I'd release it as a standalone Nuget Package which would depend on Snapper, if you're fine with it.

Custom update snapshot attributes

Is your feature request related to a problem? Please describe.
Currently, the only way to update a snapshot is via the UpdateSnapshot attribute (or the environment variable). Both of these solutions require changing the code, which means that if I accidently leave one or other of those set then the snapshot will be overwritten by any of my team mates who subsequently run the tests.*

Describe the solution you'd like
Allow users to override the UpdateSnapshot attribute with other behaviors (such as opening a diff window) when the snapshots do not match. I don't think that Snapper needs to provide any of those behaviors out of the box - but if we could make it more extensible then users could roll their own.

Describe alternatives you've considered
We are using F# and XUnit for our tests, so I considered using XUnit's ITestOutputHelper to write out the new snapshot, which could then be copied and pasted if necessary. However, since we are using F#, most of our tests are static rather than instance methods, so we can't use the ITestOutputHelper. (Well, we could, but we're keen to avoid that, because we'd rather keep a consistent style).

*I have definitely done this. I'm just going to hang my head in shame :(

Snapshot created with different serialization settings

Describe the bug
When storing snapshots, global settings for NewtonSoft are partially used. Snapper settings should be used instead.

There was a refactor in Snapper that fixed issues with dates and times. However, it missed one line (see image attached):

This is affecting our app, as it is creating invalid snapshots. Could you please fix it for next release?

To Reproduce
Just create a new snapshot with two children (aka: 1 file with two "child" snapshots in it), each of them containing a datetime value. Set global settings different from the ones in JObject.Helper. Create the first snapshot (which should be fine), then create the second (which should be also fine), but this operation forces the already created one to be serialized using the global settings, corrupting it.

Additional context
Snapper Version: 2.3.1
Operation System: Windows 10
.Net Version: 6.0

[DOCUMENTATION] SupportedTestMethodNotFoundException

Second issue here: https://theramis.github.io/Snapper/FAQS.html , states that you can fix the problem if you add the following attribute to your method: MethodImpl(MethodImplOptions.NoInlining).

This didn't work by itself, I had to set DebugType also:

<PropertyGroup Condition=" '$(Configuration)' == 'Release' ">
   <DebugType>full</DebugType>
</PropertyGroup>

While i had it on pdbonly it didn't work. My project is using .NET 4.7.2 and NUnit 3.12. Maybe just add this as a note, spent half day debugging to get it working :)

Support NUnit v4

Is your feature request related to a problem? Please describe.
Currently i get an exception:

System.TypeLoadException : Method 'get_Description' in type 'Snapper.Nunit.EqualToSnapshotConstraint' from assembly 'Snapper.Nunit, Version=2.0.0.0, Culture=neutral, PublicKeyToken=c90abed6faf59688' does not have an implementation.

Describe the solution you'd like
Support NUnit v4

Describe alternatives you've considered
No alternate beside to drop this library

Additional context
none

sign assemblies

We have a project (net45) where signed assemblies are a hard requirement. They got installed to the global assembly cache.

Can we sign the Snapper assemblies? I can send a pr.

Create snapshot on first run if none exists

Is your feature request related to a problem? Please describe.
Currently Snapper returns the error A snapshot does not exist when a snapshot does not exist. The user then has to go and set the UpdateSnapshots attribute and re-run the tests. This is cumbersome and an extra step that can be avoided to improve developer experience.

Describe the solution you'd like
If the environment is not a CI environment and a snapshot does not exist it should create one.

[BUG] EqualToSnapshot takes no parameters

The function Snapper.Nunit.Is.EqualToSnapshot takes a parameter SnapshotName according to the documentation. Unfortunately I cannot use this function, only without overloading.

Operation System: Windows 10
Snapper Version: v2.2.1
Snapper.Nunit: v2.2.1
NUnit: v3.12.0
.Net Version: .NET Framework 4.7.2

Snapper Nunit path resolver can fail

We have a scenario where we package some of our unit tests, which use Snapper, as a nuget to be distributed and consumed.

When doing this, we see an exception thrown by the Snapper Nunit path resolver code, which tries to combine path elements, but fails since it is using StackFrame.GetFileName() which returns null. Actually, this method documents that it can return null.

The code we are referring to is in NUnitTestHelper.cs:

 public static (MethodBase, string) GetCallingTestInfo()
        {
            var stackTrace = new StackTrace(2, true);
            foreach (var stackFrame in stackTrace.GetFrames() ?? new StackFrame[0])
            {
                var method = stackFrame.GetMethod();

                if (IsNUnitTestMethod(method))
                    return (method, stackFrame.GetFileName());

                var asyncMethod = GetMethodBaseOfAsyncMethod(method);
                if (IsNUnitTestMethod(asyncMethod))
                    return (asyncMethod, stackFrame.GetFileName());
            }

            throw new InvalidOperationException(
                "Snapshots can only be created from classes decorated with the [Test] attribute");
        }

So, if stackFrame.GetFileName() returns null, the caller - NUnitPathResolver.ResolvePath() - tries to get its directory using Path.GetDirectoryName(), which itself will return null. The null becomes the first parameter to Path.Combine(), which throws an ArgumentNullException.

As we see it, the only reason to use StackFrame.GetFileName() is to obtain the directory of the actual unit test assembly. But this could be done instead by using the declaring type of the unit test method found, and getting its assembly. So, the change would replace:

return (method, stackFrame.GetFileName());

with

return ( method, method.DeclaringType.Assembly.Location );

...and the same for the async case. Or even put the Path.GetDirectoryName() at this level instead of the caller, since only the directory is relevant.

What do you think?