GithubHelp home page GithubHelp logo

nidi3 / raml-tester Goto Github PK

View Code? Open in Web Editor NEW
71.0 10.0 14.0 1.21 MB

Test if a request/response matches a given raml definition

License: Apache License 2.0

Java 97.04% RAML 2.96%
raml java raml-tooling testing

raml-tester's Introduction

raml-tester

Build Status codecov License

Test if a request/response matches a given raml definition.

Versioning

Version Contents
0.8.x Stable version, uses RAML parser 0.8.x and supports only RAML v0.8
0.9.x Development version, uses RAML parser 1.x and supports RAML v0.8 and parts of v1.0
1.0.x As soon as RAML v1.0 support is stable

Add it to a project

Add these lines to the pom.xml:

<dependency>
    <groupId>guru.nidi.raml</groupId>
    <artifactId>raml-tester</artifactId>
    <version>0.8.8</version>
</dependency>

If you are stuck with java 1.6, use the compatible version by adding the following line:

    <classifier>jdk6</classifier>

Use in a spring MVC test

@RunWith(SpringJUnit4ClassRunner.class)
@WebAppConfiguration
@ContextConfiguration(classes = Application.class)
public class SpringTest {

    private static RamlDefinition api = RamlLoaders.fromClasspath(SpringTest.class).load("api.raml")
            .assumingBaseUri("http://nidi.guru/raml/simple/v1");
    private static SimpleReportAggregator aggregator = new SimpleReportAggregator();

    @ClassRule
    public static ExpectedUsage expectedUsage = new ExpectedUsage(aggregator);

    @Autowired
    private WebApplicationContext wac;

    private MockMvc mockMvc;

    @Before
    public void setup() {
        mockMvc = MockMvcBuilders.webAppContextSetup(wac).build();
    }

    @Test
    public void greeting() throws Exception {
        Assert.assertThat(api.validate(), validates());

        mockMvc.perform(get("/greeting").accept(MediaType.parseMediaType("application/json")))
                .andExpect(api.matches().aggregating(aggregator));
    }
}

The ExpectedUsage rule checks if all resources, query parameters, form parameters, headers and response codes defined in the RAML are at least used once.

The RamlMatchers.validates() matcher validates the RAML itself.

api.matches() checks that the request/response match the RAML definition.

See also the raml-tester-uc-spring project.

Use in a Java EE / JAX-RS environment

@RunWith(Arquillian.class)
public class JaxrsTest {

    private static RamlDefinition api = RamlLoaders.fromClasspath(JaxrsTest.class).load("api.raml")
            .assumingBaseUri("http://nidi.guru/raml/simple/v1");
    private static SimpleReportAggregator aggregator = new SimpleReportAggregator();
    private static WebTarget target;

    @ClassRule
    public static ExpectedUsage expectedUsage = new ExpectedUsage(aggregator);

    @Deployment(testable = false)
    public static WebArchive createDeployment() {
        return ShrinkWrap.create(WebArchive.class).addClass(Application.class);
    }

    @ArquillianResource
    private URL base;

    @Before
    public void setup() throws MalformedURLException {
        Client client = ClientBuilder.newClient();
        target = client.target(URI.create(new URL(base, "app/path").toExternalForm()));
    }

    @Test
    public void greeting() throws Exception {
        assertThat(api.validate(), validates());

        final CheckingWebTarget webTarget = api.createWebTarget(target).aggregating(aggregator);
        webTarget.request().post(Entity.text("apple"));

        assertThat(webTarget.getLastReport(), checks());
    }
}

The RamlMatchers.checks() matcher validates that the request and response conform to the RAML.

Use in a pure servlet environment

public class RamlFilter implements Filter {
    private final Logger log = LoggerFactory.getLogger(getClass());
    private RamlDefinition api;

    @Override
    public void init(FilterConfig filterConfig) throws ServletException {
        api = RamlLoaders.fromClasspath(getClass()).load("api.yaml");
        log.info(api.validate().toString());
    }

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
            throws IOException, ServletException {
        final RamlReport report = api.testAgainst(request, response, chain);
        log.info("Raml report: " + report);
    }

    @Override
    public void destroy() {
    }
}

Or see the raml-tester-uc-sevlet project.

Use together with Apache HttpComponents

public class HttpComponentsTest {
    @Test
    public void testRequest() throws IOException {
        RamlDefinition api = RamlLoaders.fromClasspath(getClass()).load("api.yaml");
        Assert.assertThat(api.validate(), validates());

        RamlHttpClient client = api.createHttpClient();
        HttpGet get = new HttpGet("http://test.server/path");
        HttpResponse response = client.execute(get);

        Assert.assertThat(client.getLastReport(), checks());
    }
}

Or see the raml-tester-uc-servlet project.

Use together with RestAssured

public class RestAssuredTest {
    @Test
    public void testWithRestAssured() {
        RestAssured.baseURI = "http://test.server/path";
        RamlDefinition api = RamlLoaders.fromClasspath(getClass()).load("api.yaml");
        Assert.assertThat(api.validate(), validates());

        RestAssuredClient restAssured = api.createRestAssured();
        restAssured.given().get("/base/data").andReturn();
        Assert.assertTrue(restAssured.getLastReport().isEmpty());
    }
}

If you are using RestAssured 3.0, call api.createRestAssured3().

Use as a standalone proxy

When used as a proxy, any service can be tested, regardless of the technology used to implement it. See the raml-proxy project.

Use with Javascript

There is special support for javascript.

See raml-tester-js for details and raml-tester-uc-js for examples.

FailFast

You can configure the RamlDefinition to throw an exception in case a violation is found.

@Test(expected = RamlViolationException.class)
public void testInvalidResource() {
    RestAssured.baseURI = "http://test.server/path";
    RamlDefinition api = RamlLoaders.fromClasspath(getClass()).load("api.yaml");
    Assert.assertThat(api.validate(), validates());

    RestAssuredClient restAssured = api.failFast().createRestAssured();
    restAssured.given().get("/wrong/path").andReturn();
    fail("Should throw RamlViolationException");
}

raml-tester's People

Contributors

ddossot avatar ismail-s avatar martinpaoloni avatar nidi3 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

raml-tester's Issues

Feature request: Usage registers usage of different actions on a resource.

For Usage to register usage of different actions on a resource. Right now only usage of a resource (regardless of action) is registered.
So I'm thinking a getUnusedResources version that returns untested http actions per resource.
(What I'm looking for is getting a list of unused actions on each resource).

NPE in SecurityExtractor when using Basic Authentication

This is related to #24

With this definition, which is valid in RAML:

securitySchemes:
  - basic:
      type: Basic Authentication

I receive:

java.lang.NullPointerException
    at guru.nidi.ramltester.core.SecurityExtractor.queryParameters(SecurityExtractor.java:46)
    at guru.nidi.ramltester.core.RamlChecker.checkQueryParameters(RamlChecker.java:146)
    at guru.nidi.ramltester.core.RamlChecker.checkRequest(RamlChecker.java:112)
    at guru.nidi.ramltester.core.RamlChecker.check(RamlChecker.java:68)

Until #24 gets done, I think raml-tester should ignore the schemes it can't check (maybe log a warning about it) instead of dying with an NPE.

Issue in mime sub-type compatibility test

I'm facing a problem where:

application/vnd.unbounce.test.v1+json

is not detected as being compatible with:

application/json

although it should.

Because of this issue, I receive:

No SchemaValidator found for media type 'application/vnd.unbounce.test.v1+json'

Playing with guru.nidi.ramltester.util.MediaType, I indeed see that:

MediaType
    .valueOf("application/vnd.unbounce.test.v1+json")
    .isCompatibleWith(MediaType.valueOf("application/json"))

returns false, which prevents the RestassuredSchemaValidator to be picked.

It would be great if this issue could be fixed.

!include ignored in resource type parameters

If I pass an included file to a resource type, like this:

exampleCollection: !include example/sites.json

then when validating the RAML, instead of reading the file example/sites.json, it just behaves as if the !include keyword wasn't there, and tries to parse the filename as if it were JSON.

Here is the relevant part of the stacktrace:

  [1] com.fasterxml.jackson.core.base.ParserMinimalBase._reportError (ParserMinimalBase.java:533)
  [2] com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken (ReaderBasedJsonParser.java:2,462)
  [3] com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue (ReaderBasedJsonParser.java:1,621)
  [4] com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken (ReaderBasedJsonParser.java:689)
  [5] com.fasterxml.jackson.databind.MappingIterator.hasNextValue (MappingIterator.java:240)
  [6] com.github.fge.jackson.JsonNodeReader.readNode (JsonNodeReader.java:142)
  [7] com.github.fge.jackson.JsonNodeReader.fromReader (JsonNodeReader.java:127)
  [8] com.github.fge.jackson.JsonLoader.fromReader (JsonLoader.java:179)
  [9] com.github.fge.jackson.JsonLoader.fromString (JsonLoader.java:192)
  [10] com.jayway.restassured.module.jsv.JsonSchemaValidator.matchesSafely (JsonSchemaValidator.java:214)
  [11] com.jayway.restassured.module.jsv.JsonSchemaValidator.matchesSafely (JsonSchemaValidator.java:75)
  [12] org.hamcrest.TypeSafeMatcher.matches (TypeSafeMatcher.java:65)
  [13] guru.nidi.ramltester.validator.RestassuredSchemaValidator.validate (RestassuredSchemaValidator.java:97)
  [14] guru.nidi.ramltester.core.RamlValidatorChecker.exampleSchema (RamlValidatorChecker.java:275)
  [15] guru.nidi.ramltester.core.RamlValidator.mimeType (RamlValidator.java:117)
  [16] guru.nidi.ramltester.core.RamlValidator.action (RamlValidator.java:102)
  [17] guru.nidi.ramltester.core.RamlValidator.resource (RamlValidator.java:85)
  [18] guru.nidi.ramltester.core.RamlValidator.validate (RamlValidator.java:66)
  [19] guru.nidi.ramltester.RamlDefinition.validate (RamlDefinition.java:122)

This is with raml-tester 0.8.4.

Allow schema in POST/PUT body

I tried today the newest version of raml-tester. While running tests I got an exception for all tests, which test POST/PUT endpoints:

No schema allowed on action(PUT /url/toEndpoint/{Id}) mime-type('application/json'), No formParameters given on action(PUT /url/toEndpoint/{Id}) mime-type('application/json')], responseViolations=[]}

According RAML spec 0.8 it should be possible to define schemas in request body like this:

post:
  body:
    application/json:
      schema: |
        {
          "$ref":"schemas/<<resourcePathName | !singularize>>.json"
        }

RamlViolationException is thrown, if content-type header is null

When testing response of POST-Request and endpoint only returns HTTP-Status 200 (OK) without response, a RamlViolationException is thrown by the following code:

Class: RamlTestRunner.java
Method: public void testResponse(Action action, HttpResponse response)

violations.addViolationAndThrow(response.getContentType() == null, "Response has no Content-Type header");

According HTTP/1.1-Method definition chapter 9.5 a POST-Request might not result in a resource, so I think it would be a good idea to expand the check according the definition like this:

violations.addViolationAndThrow((action.getType() != RequestMethod.POST || response.getStatus() != HttpStatus.OK)  && response.getContentType() == null, "Response has no Content-Type header");

External RAML includes with absolute URI not found

We have a RAML file which includes an external RAML schema by specifying an absolute URI.

resourceTypes: !include http://host.com/private/common/types_v1.0.raml

This causes an exception:

guru.nidi.ramltester.loader.RamlLoader$ResourceNotFoundException: java.io.IOException: Http response status not ok: HTTP/1.1 404 Not Found
    at guru.nidi.ramltester.loader.SimpleUrlFetcher.fetchFromUrl(SimpleUrlFetcher.java:35)
    at guru.nidi.ramltester.loader.UrlRamlLoader.fetchResource(UrlRamlLoader.java:51)
    at guru.nidi.ramltester.loader.RamlLoaderRamlParserResourceLoader.fetchResource(RamlLoaderRamlParserResourceLoader.java:34)
    at org.raml.parser.tagresolver.IncludeResolver.resolve(IncludeResolver.java:67)
    at org.raml.parser.visitor.TemplateResolver.resolveInclude(TemplateResolver.java:178)
    at org.raml.parser.visitor.TemplateResolver.init(TemplateResolver.java:116)
    at org.raml.parser.visitor.RamlDocumentBuilder.preBuildProcess(RamlDocumentBuilder.java:140)
    at org.raml.parser.visitor.YamlDocumentBuilder.build(YamlDocumentBuilder.java:86)
    at org.raml.parser.visitor.YamlDocumentBuilder.build(YamlDocumentBuilder.java:94)
    at org.raml.parser.visitor.YamlDocumentBuilder.build(YamlDocumentBuilder.java:77)
    at guru.nidi.ramltester.RamlLoaders.load(RamlLoaders.java:168)
    at guru.nidi.ramltester.RamlLoaders.loadFromUri(RamlLoaders.java:92)

With this message in our server log.

/reader/raml/http://host.com/private/common/types_v1.0.raml

This is probably related to #13

NPE in SecurityExtractor when using `null` security scheme

Per RAML spec:

To indicate that the method may be called without applying any securityScheme, the method may be annotated with the null securityScheme.
...
securedBy: [null, oauth_2_0]

When adding null to the array of security types, I get:

java.lang.NullPointerException
    at guru.nidi.ramltester.core.SecurityExtractor.queryParameters(SecurityExtractor.java:46)
    at guru.nidi.ramltester.core.RamlChecker.checkQueryParameters(RamlChecker.java:146)
    at guru.nidi.ramltester.core.RamlChecker.checkRequest(RamlChecker.java:112)
    at guru.nidi.ramltester.core.RamlChecker.check(RamlChecker.java:68)

Testing apps with a servlet-per-resource fails

We've been using (or trying to use) raml-tester for two different projects. It's working great on one project, but we're having troubles with the second project. The first project uses Jersey, so there's a single servlet, mapped to /api, with all the resources served up via it, eg /api/someApiResource. This works fine.

The project we have trouble with is a legacy project, that has a servlet per resource. Because of this, there is no path-info on requests to the resources. That's causing problems with this code from ServletRamlRequest:

    @Override
    public String getRequestUrl(String baseUri) {
        return baseUri == null
                ? request().getRequestURL().toString()
                : (baseUri + request().getPathInfo());
    }

This results in "baseUrinull" as the requestUrl. Because of this, raml-tester is unable to match any of the resources being requested by our API tests.

I'm not sure what the best solution to this is, but I wondered if returning request().getRequestURL().toString() when path-info is null would do the trick safely?

    @Override
    public String getRequestUrl(String baseUri) {
        String pathInfo = request().getPathInfo();
        return baseUri == null || pathInfo == null
                ? request().getRequestURL().toString()
                : (baseUri + pathInfo);
    }

I've verified that the above change passes a mvn clean install, at least, and seems to fix our usecase.

resource.getResolvedUriParameters() does not resolve params from ancestor above parent

If getResolvedUriParameters is run on the resource child2 below, no parameters is returned. If it is run from child1, the id parameter is returned.

/{id}:
    uriParameters:
      id:
        type: integer
        required: true
        example: 7393
    /child1:
      /child2:
        get:
          responses:
            200:
              body:

It looks like this line:

Map<String, UriParameter> uriParams = new HashMap<String, UriParameter>(parentResource.getUriParameters());

Should be:

Map<String, UriParameter> uriParams = new HashMap<String, UriParameter>(parentResource.getResolvedUriParameters());

Bogus unusedResources error when a path has no method

With this RAML structure:

/users:
  /self:
    get:
  /{user_id}:
    get:

I get the following error:

  "unusedResources": [
    "/users"
  ]

after running tests on the API.
As you can see, there's no method defined at /users level, only in the levels below can the get method be found. I think that in this case, RAML Tester should not consider this resource as being unused because it can't be used at all...

RamlLoaders.loadFromUri() throws a guru.nidi.ramltester.loader.RamlLoader$ResourceNotFoundException

Specifying a URI http://localhost:8080/reader/raml/api_v1.raml fails to read the RAML file and throws an exception.

guru.nidi.ramltester.loader.RamlLoader$ResourceNotFoundException: org.apache.http.client.ClientProtocolException: URI does not specify a valid host name: localhost:8080/reader/raml/api_v1.raml
    at org.apache.http.impl.client.CloseableHttpClient.determineTarget(CloseableHttpClient.java:94)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
    at guru.nidi.ramltester.loader.SimpleUrlFetcher.fetchFromUrl(SimpleUrlFetcher.java:33)
    at guru.nidi.ramltester.loader.UrlRamlLoader.fetchResource(UrlRamlLoader.java:51)
    at guru.nidi.ramltester.loader.RamlLoaderRamlParserResourceLoader.fetchResource(RamlLoaderRamlParserResourceLoader.java:34)
    at org.raml.parser.visitor.YamlDocumentBuilder.build(YamlDocumentBuilder.java:76)
    at guru.nidi.ramltester.RamlLoaders.load(RamlLoaders.java:168)
    at guru.nidi.ramltester.RamlLoaders.loadFromUri(RamlLoaders.java:92)

After digging through the code it looks like the http:// part is stripped off which may be the cause of this issue. Specifying http:// twice like so http://http://localhost:8080/reader/raml/api_v1.raml by-passes the issue but causes another issue which I'm going to log separately.

I'm using 0.0.6 version.

SimpleUrlFetcher locks with multiple !includes

I have the following raml file:

title: raml with links 
baseUri: http://deadleg.github.io/bugs
version: v0.6

/first:
  get:
    body:
      application/json:
        schema: !include json1.json
/second:
  get:
    body:
      application/json:
        schema: !include json2.json
/thrid:
  get:
    body:
      application/json:
        schema: !include json3.json

When using a UrlLoader or GithubLoader, the thread get stuck in a WAITING state when trying to grab the last !include:

Raml raml = new RamlDocumentBuilder(new RamlLoaderRamlParserResourceLoader(new UriRamlLoader(new UrlRamlLoader("http://deadleg.github.io/bugs")))).build("test.raml");
Raml raml = new RamlDocumentBuilder(new RamlLoaderRamlParserResourceLoader(new UriRamlLoader(new GithubRamlLoader("Deadleg/Deadleg.github.io")))).build("bugs/test.raml");

With thread dump:

java.lang.Thread.State: WAITING
      at sun.misc.Unsafe.park(Unsafe.java:-1)
      at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
      at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:138)
      at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:306)
      at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64)
      at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:192)
      at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:185)
      at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:107)
      at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276)
      at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263)
      at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
      at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
      at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
      at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
      at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
      at guru.nidi.ramltester.loader.SimpleUrlFetcher.fetchFromUrl(SimpleUrlFetcher.java:43)
      at guru.nidi.ramltester.loader.GithubRamlLoader.fetchResource(GithubRamlLoader.java:62)
      at guru.nidi.ramltester.loader.UriRamlLoader.fetchResource(UriRamlLoader.java:70)
      at guru.nidi.ramltester.loader.RamlLoaderRamlParserResourceLoader.fetchResource(RamlLoaderRamlParserResourceLoader.java:34)
      at org.raml.parser.tagresolver.IncludeResolver.resolve(IncludeResolver.java:64)
      at org.raml.parser.visitor.NodeVisitor.resolveTag(NodeVisitor.java:162)
      at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:139)
      at org.raml.parser.visitor.NodeVisitor.visitMappingNode(NodeVisitor.java:86)
      at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:218)
      at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
      at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
      at org.raml.parser.visitor.NodeVisitor.visitMappingNode(NodeVisitor.java:86)
      at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:218)
      at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
      at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
      at org.raml.parser.visitor.NodeVisitor.visitMappingNode(NodeVisitor.java:86)
      at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:218)
      at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
      at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
      at org.raml.parser.visitor.NodeVisitor.visitMappingNode(NodeVisitor.java:86)
      at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:218)
      at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
      at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
      at org.raml.parser.visitor.NodeVisitor.visitDocument(NodeVisitor.java:209)
      at org.raml.parser.visitor.YamlDocumentBuilder.build(YamlDocumentBuilder.java:90)
      at org.raml.parser.visitor.YamlDocumentBuilder.build(YamlDocumentBuilder.java:102)
      at org.raml.parser.visitor.YamlDocumentBuilder.build(YamlDocumentBuilder.java:78)

From what I can see, the issue isn't with this library, but rather with the raml-java-parser IncludeResolver.resolve method. The resolver grabs the input stream for each include, but as far as I can see it doesn't close it (nor consume the response contents). If I explicitly call for the input stream to close, the issue goes away.

Since raml-java-parser doesn't have a dependency on HttpComponents, it may be easier (and safer) to switch out this projects dependency on it to something else.

Allow X- headers to be ignored

This is more a feature request than an issue.

RAML Tester reports lots of issues with X- headers like the following (coming via the proxy):

    "Header 'X-Runtime' on action(GET /accounts) is not defined",
    "Header 'X-Instance-Id' on action(GET /accounts) is not defined",
    "Header 'X-Server-Instance' on action(GET /accounts) is not defined",
    "Header 'X-UA-Compatible' on action(GET /accounts) is not defined",
    "Header 'X-Request-Id' on action(GET /accounts) is not defined",
    "Header 'X-Powered-By' on action(GET /accounts) is not defined"

These headers are not in the RAML spec and won't be: they are added by servers and middlewares and are not really part of the API spec.

So it would be great to be able to tell RAML Tester to ignore X-* headers.

Nested URL-parameters not yet supported

Testing nested URL-parameters not possible

raml-example:

/book:
  get:
    responses:
      200:
        body:
          application/json:
  /{bookId}:
    /author:
      get:
        responses:
          200:
            body:
              application/json:

Bogus warning if JSON example file contains string instead of object

If an example file contains just a string, which is legal JSON, instead of an object, a bogus warning is logged:

WARN  | 2015-10-01 18:26:16,662 | guru.nidi.ramltester.RelativeJsonSchemaAwareRamlDocumentBuilder - Could not parse json file example/authenticate.json. Relative $refs inside might not work.
com.fasterxml.jackson.databind.JsonMappingException: Can not instantiate value of type [map type; class java.util.LinkedHashMap, [simple type, class java.lang.Object] -> [simple type, class java.lang.Object]] from String value ('foo'); no single-String constructor/factory method
 at [Source: "foo"
; line: 1, column: 1]
    at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
    at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:875)
    at com.fasterxml.jackson.databind.deser.ValueInstantiator._createFromStringFallbacks(ValueInstantiator.java:281)
    at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromString(StdValueInstantiator.java:284)
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:334)
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26)
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3731)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2724)
    at guru.nidi.ramltester.RelativeJsonSchemaAwareRamlDocumentBuilder.onScalar(RelativeJsonSchemaAwareRamlDocumentBuilder.java:56)
    at org.raml.parser.visitor.NodeVisitor.visitScalar(NodeVisitor.java:256)
    at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:222)
    at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
    at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
    at org.raml.parser.visitor.NodeVisitor.visitMappingNode(NodeVisitor.java:86)
    at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:218)
    at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
    at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
    at org.raml.parser.visitor.NodeVisitor.visitMappingNode(NodeVisitor.java:86)
    at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:218)
    at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
    at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
    at org.raml.parser.visitor.NodeVisitor.visitMappingNode(NodeVisitor.java:86)
    at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:218)
    at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
    at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
    at org.raml.parser.visitor.NodeVisitor.visitMappingNode(NodeVisitor.java:86)
    at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:218)
    at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
    at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
    at org.raml.parser.visitor.NodeVisitor.visitMappingNode(NodeVisitor.java:86)
    at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:218)
    at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
    at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
    at org.raml.parser.visitor.NodeVisitor.visitMappingNode(NodeVisitor.java:86)
    at org.raml.parser.visitor.NodeVisitor.visit(NodeVisitor.java:218)
    at org.raml.parser.visitor.NodeVisitor.visitResolvedNode(NodeVisitor.java:180)
    at org.raml.parser.visitor.NodeVisitor.doVisitMappingNode(NodeVisitor.java:151)
    at org.raml.parser.visitor.NodeVisitor.visitDocument(NodeVisitor.java:209)
    at org.raml.parser.visitor.YamlDocumentBuilder.build(YamlDocumentBuilder.java:90)
    at org.raml.parser.visitor.YamlDocumentBuilder.build(YamlDocumentBuilder.java:102)
    at org.raml.parser.visitor.YamlDocumentBuilder.build(YamlDocumentBuilder.java:78)
    at guru.nidi.ramltester.RamlLoaders.load(RamlLoaders.java:188)

NullPointerException when the response does not contain a body

guru.nidi.ramltester.httpcomponents.HttpComponentsUtils#buffered(org.apache.http.HttpResponse) causes a NullPointerException when the response does not contain a body, as when a 204 is returned. Looks like this is incorrect:

        final HttpEntity entity = response.getEntity();  //entity may be null
        if (!entity.isRepeatable()) {

Raml:

    post:
      body:
        ...
      responses:
        204:

queryParameters under securitySchemes is not recognized

queryParameters defined under securityschemes is not picked up, so that test fails due to url parameter which is not present in raml.

Example:

Raml:

securitySchemes:
    - session:
        description: |
            Session authentication.
        describedBy:
            queryParameters:
                sessionId:
                    description: |
                       A session ID.
                    type: string
            responses:
                401:
                    description: |
                        An invalid session ID was given or the session is expired.
        type: x-session
...
/something/{id}:
  uriParameters: 
    id: 
      type: integer
      required: true
  get:
    securedBy: [session]

Request:
http://localhost:8080/something/706?sessionId=1001

Resulting report:
RamlReport{
requestViolations =[Query parameter 'sessionId' on action(GET /something/{id}) is not defined]
responseViolations=[]}

Accept header check doesn't support `*/*`

We've tried upgrading to RAML tester 0.8.2 but have many tests failing because the new Accept header test doesn't cater to */*.

Example (only relevant parts are kept):

{
    "requestHeaders": {
        "Accept": [
            "*/*"
        ]
    },
    "responseHeaders": {
        "Content-Type": [
            "application/json; charset=UTF-8"
        ]
    },
    "responseViolations": [
        "Response Content-Type 'application/json; charset=UTF-8' is not compatible with Accept header '*/*'"
    ]
}

The specification of the Accept header allows complex expressions like: Accept: audio/*; q=0.2, audio/basic. Thus, IMO, if RAML Tester decides to enforce Accept/Content-Type validity, it has to fully support Accept's spec (FWW I'm using the Java helper in this library for that intent in some of my apps).

Header names are case sensitive

We have the following API definition:

    put:
      headers:
        If-Match:
          description: |
            xxx
          type: string
          required: true
      body:
        schema: xxx
      responses:
        200:
          description: xxx
          headers:
            ETag:
              type: string
              description: xxx
          body:
            schema: xxx

As you can see we specify If-Match and ETag because they play an important role on this resource and we want to attach a clear documentation to these request and response headers.

When we test our API, we keep getting this kind of violations:

Unused request headers: ("If-Match in PUT /xxx")
Unused response headers: ("ETag in PUT /xxx -> 200")

But our test requests and server responses do have these headers. I think this is due to the fact that these headers are in the DefaultHeaders structure: https://github.com/nidi3/raml-tester/blob/master/src/main/java/guru/nidi/ramltester/core/RamlChecker.java#L39-L40

No check for non-existing properties in schemas or responses

Properties in responses that are missing in the raml-schemas does not result in errors in the RamlReport. Example response:

{
"id": 9706
"name": "Bob"
}

With raml:

          schema: |
              {
                  "properties": {
                      "id": {
                          "type": "integer"
                      }
                  },
                  "type": "object"
              }

Should result in a validation error I think.

Also, properties in schemas which does not exist in the response are validated as OK.
Example response:

{
"id": 9706
}

Example raml:

    responses:
      200:
        body:
          schema: |
              {
                  "properties": {
                      "name": {
                          "type": "string"
                      },
                      "id": {
                          "type": "integer"
                      }
                  },
                  "type": "object"
              }

Cannot resolve relative schema references in JSON schemas

We have a RAML file that references some JSON schema files, and one of the JSON schema files references another JSON schema file. The latter reference cannot be resolved by the JSON schema validator - even when we pass in a custom loader to SchemaValidators.withloader to resolve such references (the custom loader is never called).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.