ppi-ag / deep-sampler Goto Github PK
View Code? Open in Web Editor NEWDeepSampler is a stubbing framework for compound tests
Home Page: https://ppi.de
License: MIT License
DeepSampler is a stubbing framework for compound tests
Home Page: https://ppi.de
License: MIT License
Hey all,
I just wanted to let you know, that proxied services cannot be recorded when using something like "target(my.package.MyService)" in a Pointcut when the check "!within(is(FinalType))" is present, see:
I had to create a local copy without that condition to get it working.
Best regards
Michel Pittelkow
We already have a rough custom theme for docsify, but it needs some improvements.
Sometimes methods change their parameter values (i.e. "return parameters" in clean code terms), even though this is considered to be a code smell. DeepSampler has no means to handle that, yet. A simple solution could be a postprocessor that runs after a stubbed method, or after a Sample has been retrieved from a JSON-file. That postprocessor could then implement arbitrary logic to enhance the return parameter.
Samples in a JSON-file still need a matching Sampler. If the file contains a Sample for a method that doesn't have a corresponding Sampler, DeepSampler would silently filter the Sample from the file. This results in an unspecific error because a Sample is not used in the test and the original method is called. This error is currently hard to find. It would be better to throw an Exception if the file contains a Sample that has no corresponding Sampler. This Exception should explain which Sample had no Sampler in an easy to unterstand manner.
The code in question is here:
Hi all,
I am using version 1.1.0 and therefore had to add some functionality, that in parts is already added for version 2.x.
Part of this is the ability to configure a base repository for stored files, which I just noticed you already implemented for the next version.
Another addition I added is the ability to configure if the name of the testMethod should be used in the generated json file name.
Let me explain, why I added this:
For convenience I added another configuration value to the Annotation UseSamplerFixture, which is then used inside the JUnitPluginUtils. Let me show you with the following:
public static void saveSamples(final Method testMethod) {
final SaveSamples saveSamples = testMethod.getAnnotation(SaveSamples.class);
if (saveSamples == null) {
return;
}
final JsonSourceManager.Builder persistentSampleManagerBuilder = loadBuilder(
saveSamples.persistenceManagerProvider());
final String fileName = getFilePath(saveSamples, getFixture(testMethod), testMethod);
PersistentSamplerHolder.source(persistentSampleManagerBuilder.buildWithFile(fileName));
addExtensions(testMethod);
PersistentSamplerHolder.getSampler().record();
}
private static String getFilePath(LoadSamples loadSamples, UseSamplerFixture fixture, Method testMethod) {
return getFilePath(loadSamples.file(), fixture, testMethod);
}
private static String getFilePath(SaveSamples saveSamples, UseSamplerFixture fixture, Method testMethod) {
return getFilePath(saveSamples.file(), fixture, testMethod);
}
private static String getFilePath(String path, UseSamplerFixture fixture, Method testMethod) {
String basePath = fixture != null ? fixture.repositoryBase() : "";
String file = StringUtils.defaultIfEmpty(path,
getDefaultJsonFileNameWithFolder(testMethod, withMethod(fixture)));
return Paths.get(basePath, file).toString();
}
private static boolean withMethod(final UseSamplerFixture fixture) {
return Optional.ofNullable(fixture).map(anno -> anno.withMethod()).orElse(false);
}
private static String getDefaultJsonFileNameWithFolder(final Method testMethod, boolean withMethod) {
return testMethod.getDeclaringClass().getName().replace(".", "/")
+ (withMethod ? ("_" + testMethod.getName()) : "") + ".json";
}
Let me hear your thoughts on this.
Cheers,
Michel
Since DeepSampler is modularized and customizable we need an article that describes how to build a custom AOP-Provider.
We have three methods to change the Scope:
The first one is an abstract Version that uses Enums to set the scope and it sets the scope for both, SampleRepository and ExecutionRepository.
Number 2 and 3 are on a lower level of the API and are used to set the actual scope-implementation to the repositories that need scoping.
Users cannot understand which of these methods should be used to set the scope. Therefore we need to clarify the API.
Deserialized SampleDefinitions don't honor custom matchers even though they are known by the SampleDefinition that is defined by Samper.of(...). So currently deserialized Sampler use the EqualsMatcher only.
AOP comes with a downside: If an aspect is not called due to a bug, this bug is not visible, since no code is executed, that could notice the bug and throw an Exception, or print an error message.
As a result, DeepSampler tends to do nothing, if some configuration errors exist. This is quite hard to debug. This could be eased by logging.
Since DeepSampler is an API that must not force the usage of a specific logging API, we must find a possibility to allow a form of logging, that fits into an existing logging environment, and comes without additional dependencies.
If the persistence is managed by Annotations in JUnitTests it is not possible to add Extensions for the PersistentBeanFactory. The
PersistentSampleManagerProvider does not give any access to that.
The plugin interface is not consistently. It is mostly designed around the actual use-case. Futhermore it is in my point of view not save to add a additional plugin. The plugins are hardcoded and have to be in a specific order.
There had to be a standartizied Plugin Interface and a not order dependend possibility to add new plugins.
If DeepSampler is used inside of an ApplicationServer and the definition of Samples and saving recorded data to a file is should be done in separated Threads, we need a possibility to keep the SampleRepository alive across Thread borders.
Usually we enforce thread safety using ThreadLocals. This should be configurable.
We need an article that explains the Sampler API in detail. Since this is the core of DeepSampler we should design this article thoroughly.
The article should provide in-depth-information but it should also be easy to search in it, so that it can be used to quickly look things up.
The search-plugin of docsify should be able to find things in this article.
The article should not describe stubbing using the recorder and player, that will be another chapter.
Hi,
after I finished working on SaveSamples-part of a service execution, I started with the LoadSamples-part.
This quickly throw me an Matcher-Error for a call like Sample.of(service.doStuff(Matchers.equalTo("MyString")));
as the EqualsMatcher does not allow a type without an equals-implementation.
As a workaround I then used Matcher.matcher(param -> "MyString".equals(param))
.
When taking a look at e.g. hamcrest as matcher library, this is working with String as well.
I thought about implementing a hamcrest-to-ParameterMatcher wrapper but wanted to let you know first as you maybe want to do something about this inside deepsampler.
Best regards,
Michel Pittelkow
If a stubbed method is called multiple times with the same parameter each call is saved in the json file leading to redundant data. A filter option would be nice to get cleaner files.
Right now we sometimes have quite long and hard to read method names in the JSON-Files. This becomes tedious if the method names are slightly refactored, so that the names in the JSON files need to be changed too. If the change is not done together with the original refactoring but instead as part of a bugfix a few days later, or by another person that doesn't now anything about the refactoring, it becomes hard to find , or even notice the difference.
The names are this extensive, because they must be distinct. Maybe this can be circumnavigated by checking if the name already exists in the SampleRepository. If this is not the case a short name can be used. Otherwise the extensive distinct name must be used.
Is this possible?
When DeepSampler tries to find a sample for a stubbed method in a JSON-file, the recorded parameters from the JSON-file are compared to the actual parameters, with which the stubbed method is actually called during the test. The comparison, is by default, done using equals()-methods. However, some parameter objects don't override equals() and it is sometimes not possible to add an equals()-method to the object. For cases like this, the PersistentMatchers#combo() method was introduced. It allows to add a replacement for the equals() method. But the method was designed from the internal perspective instead of from the user's perspective, hence the method is hard to understand. The reason for that is the internal distinction between matchers that are used during recording and replay. Combo-matchers allow to define both variants freely. But this is seldomly usefull, so the added complexity is not helpful.
Sometimes Samples are not found because the expected parameters have changed. Currently DeepSampler would treat this situation the same way as if no Sampler (no fitting method) was found at all. But in reality the first situation may occur because the tested code has changed. In this case DeepSampler would simply call the stubbed code. It would be better if DeepSampler would throw an Exception if the a Sample for a called Method exists, but no fitting parameter could be found.
When DeepSampler wants to deserialize an Object, the concrete type of the object must be known, so that a new instance of that type can be created.
There are two cases where different deserialization approaches are used:
PersistentBean
s by DeepSamplers
PersistentBeanConverter`.In the first case type information is completely handled by the persistence framework (Jackson).
In the second case DeepSampler uses the declared return type of the method that is supposed to return the persistent object. Now, there are circumstances where this doesn't work. In general, this is always the case if the actually returned object is a sub type of the declared return type. E.g. in many cases this happens if the declared return type is an interface.
That's why we need to add a type information to PersistentBean
in all cases where the original type cannot be determined by the declared return type of the sampled method.
Hi,
Annotation can be used at class level, javaDoc is telling the same. But JUnitPluginUtils ignores this and just tries to load from the TestMethod.
See:
final UseSamplerFixture fixture = testMethod.getAnnotation(UseSamplerFixture.class);
if (fixture == null) {
return;
}
Best regards,
Michel Pittelkow
Hi,
I came across the situation that many of our bean classes throw an exception when loading. Storing works fine.
This happens because of final fields inside the bean class and no fitting constructor. This behavior would mean, that I have to create a lot of boilerplate code for de-/serializing these objects when using deepsampler.
This brought to my mind, that maybe getAllFields should be extensible. With that I could e.g. write my own logic on which fields should be serialized in a more generic way, without having to declare BeanFactoryExtensions for all cases.
If this is not the way deepsampler should work -> Alternative:
Please let me know if this makes sense for you or if I'm missing something here.
Best regards,
Michel Pittelkow
Currently Optional
s cannot be persisted. An attempt to do this results in the following Exception:
de.ppi.deepsampler.persistence.error.PersistenceException: The type class java.util.Optional includes at least one final field. Therefore we tried to automatically detect a constructor accepting all field values, but weren't able to find any. If you still want to transform the bean you have to implement a BeanFactoryExtension which is able to construct the desired type class java.util.Optional.
Since Optional
is part of standard Java and quite common, DeepSampler should bring it's own BeanFactoryExtension, so that no custom implementations are necessary.
Java 16 introduced records
https://www.baeldung.com/java-record-keyword
The persistence will most likely need some alterations to allow serialization and deserialization for records.
IF we want to load JSONS and define Samplers for void methods, we currently have to do something like this:
Sampler.of(dao.loadStuff());
load Json
Sampler.of(dao.save()).doesNothing()
So the definition of Samplers must be before and after loading the json. This is a bug.
I believe this is caused by
PersistentbeanFactory::initiateUsingMatchingConstructor throughs an Exception if it is not possible to instantiate an Object using a constructor and if the object's fields are final. The corresponding Exception's message is hard to unterstand (bad english).
Currently we use the same API for marking methods and for creating stubs:
// Mark method so it can be used to record/load the method calls
Sample.of(testee.myTestMethod()); // no is/answer -> marked for persistence
Sample.of(testee.myTestMethod()).is("Hello World"); // creating stub to be returned when a real call appears. -> no persistence
This might be confusing when using both ways in a single test. So we should divide these apis to make transparent to the user what is happening.
The BeanFactory must ignore Collection-classes.
Additionally, there must be a check for embedded objects.
Currently java.sql.Timestamp needs a custom BeanExtension. I expect that this is used quite offten, so we should ad this to the standard bean mapper.
We need an article that describes how the persistence using default JSON-recorder and -player works.
Currently we can only define another Sourcemanager using the low-level-api as in
PersistentSampler.source(JsonSourceManager.builder().buildWithFile(pathToFile))
This also means, that persistence related annotations cannot be used at all, if another SourceManager must be used. Therefore we need a new annotation that configures the SourceManager.
Hi,
when loading Data that contains an array as a response an InstantiationException is thrown.
One workaround might be creating an extension for that, but this might be core functionality the factory should provide.
Instead of a stracktrace, a simple Unit Test to clarify:
@Test
@DisplayName("createValueFromPersistentBean can handle Arrays")
void createValueFromPersistentBean() {
PersistentBeanFactory factory = new PersistentBeanFactory();
PersistentBean bean = new DefaultPersistentBean();
bean.setValues(Collections.emptyMap());
Assertions.assertDoesNotThrow(() -> factory.createValueFromPersistentBean(bean, IOneInterface[].class));
}
interface IOneInterface {
void sampleMethod();
}
best regards,
Michel Pittelkow
BeanConverterExtension
has grown in the past. Although this was necessary in order to persist complex objects (i.e. generic Map
s or List
s), we believe that the majority of future extensions will be simpler. Therefore we need a simplified version of BeanConverterExtension
. For instance, many Extansion might be able to work without generics, so ParameterizedType
might be excludable in the simplified version.
Samplerepository::findAllForMethod
applies the mathers for the parameter values even though the type and/or the method did not match. This might lead to ClassCastExceptions if the type of the actual parameter value and the type of the method's parameter don't fit together.
We need an impressum, a privacy declaration, and a cookie notice.
Jackson saves byte[] in base64 format. According to Stackoverflow this might have been done to save space on disk. But in our case we often want to compare the byte[] in our .json to a byte[] in the debugger. This is e.g. the case if a byte[] parameter has changed so that a Sample is not found anymore.
So we should provide a custom serializer/deserialzer for Jackson that encodes byte[] in a readable form. This serializer/desrialzer should be default.
If JsonSourceManager.builder().buildWithFile(path)
path does not contain a folder (so only a filename) we get a NullPointerException in JsonRecorder::record or more precisly in Files::provider.
In Version 1.1.0 enumerations are converted to PersistentBeans. Recording enums in this way is possible, but the results don't make sense. Loading and replaying such enums is not possible. An Excwption is thrown since DeepSampler tries to call a constructor on individual enums.
A simple workaround is a PersistenceBeanConverter (PersistenceFactoryExtension in older versions) that skips the conversion, so that the underlying json-api (Jackson by default) handles the enum.
If we define a target path for saving/loading samples we currently have to define the path including the filename. Also we have to define this in every single test-method. In reality we mostly want to define one path once without the filename, as the filename can easily be generated.
Goal:
Hi,
just came across an issue with charset/encoding of recorded data:
reads the files with UTF8 but in
We use Cp1252 most of the time and when loading the data from the json file, it's corrupted then.
Maybe I'm missing a configuration that I can adjust.
Regards,
Michel
Sometimes methods change parameter values instead of returning results as return values (i.e. return parameter in the terms of clean code). We need som means to record and replay that using JSON-persistence.
The EqualsMatcher checks if the object that is beeing matched has a equals() that is not just the implementation from Object. But it expects that the equals() is implemented in the top subclass. If equals() has been implemented on some parent class that is not Object, the check would fail even though a custom equals() method exists. This is too strict, so the check should allow every equals() as long as it is not the one from Object.
comons-lang is propably a unessecary dependency which might leyd to problems if a customer doesn't have the proper version in a mirrored maven repo server
We are unable to persist Maps which have non primitve types (i.e. their wrapper types) as key. We need a new BeanConverterExtenstion
that is able to convert and revert Maps of the type Map<Object, Object> in a way that the Objects are converted to PersistentBean
s.
We are currently able to persist Map<T, Object> where T is any wrapper of a primitve type. This is done by MapPrimitiveKeyExtension
. This can be used as a template.
https://github.com/ppi-ag/deep-sampler/blob/645e0ab8f4775d194d6c90a54a94cd3e72cf8ca9/deepsampler-persistence/src/main/java/de/ppi/deepsampler/persistence/bean/ext/MapPrimitiveKeyExtension.java
MapPrimitiveKeyExtension
converts the key to a String, so that the key can be used as a property name in JSON:
{
"key": "value"
}
There is no nice way to convert complex objects to Strings that could be used as a key in the same way, so we need to find another way to express the key in a persistable way. One way to do it, would be to convert the Map in a List which has a tuple as entry. The tuple would then contain the key and the value.
Verify should be able to verify calls depending in the parameters. We have a testcase where verify counts all calls of a method together even though the calls were done using different parameters.
The testcase can be found in the branch bug-verify:
We need to implement scopes for the ExecutionRepository aswell, so we have consistent behavior throughout the framework.
CollectionExtension
is using generic type parameters to determine the type of the objects in the collection. This type is used to instantiate the entries of the Collection during deserialization.
Currently CollectionExtension
runs on a NullPointerException
during recording, if a subtype of Collection is saved, that doesn't provide a generic type argument. The NullPointerException
doesn't give any explanation whats wrong, and how to solve the problem. This needs to be changed.
There are several thinkable solutions:
isProcessable(...)
CollectionExtension
so that users can provide the type on their own without the need to rewrite the whole extenstion.Spring AOP replaces intercepted types by a proxy object that cannot be casted to the concrete Beans type if the Bean implements an interface. This yields an ClassCastExceptions.
Despite it is not advisable to cast interfaces to conrete types, this may happen in every application so there is a means nessecary to filter which classes should be intercepted and which not.
DeepSampler can be used without JUnit. We need an articel that shows how this is done:
We need to write a chapter on how to install DeepSampler in all supported (by default) environments:
We have a fist starting point here:
https://ppi-ag.github.io/deep-sampler/#/installation/
The article should also give an overview about DeepSampelrs various modules. In most cases users will only have to work with the top dependencies and everything else will be loaded automaticly as transient dependencies by maven/gradle. But sometimes some deeper knowledge might be usefull, at least, if someone wants to build extensions.
We need a tutorial that explains how a JUnitTest using DeepSampler would look like. The artciel explains
The tutorial should have examples for all 4 environment variations using the docsify-tabs-plugin.
Startingpoint is here:
https://ppi-ag.github.io/deep-sampler/#/setup-junit/
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.