GithubHelp home page GithubHelp logo

sozu-proxy / sozu-integration-tests Goto Github PK

View Code? Open in Web Editor NEW
2.0 2.0 0.0 139 KB

Integration tests for sozu with https://www.testcontainers.org/

Java 86.84% Dockerfile 2.70% JavaScript 7.78% Shell 2.68%

sozu-integration-tests's People

Contributors

geal avatar ldoguin avatar notbad4u avatar

Stargazers

avatar avatar

Watchers

avatar avatar avatar avatar avatar

sozu-integration-tests's Issues

Allow to specify a config.toml for a sozu container

For now, all the sozu containers use the same config.toml and maybe for some tests we 'll need some specific global options that will collide with other tests. So we must be able to specify the config.toml that will be use by sozu during the test.

That need:

  • Modify the sozu constructor
  • Modify configure() by adding a mount volume or copy the config.toml in the container
  • Fix the tests
  • (best effort) Split the current config.toml in multiple config file

new test: remove backend between requests

  • set up an application with one backend
  • start a server for that backend, and another server
  • those servers indicate their id in the response
  • the client sends a request (in keep alive)
  • check that we receive a response, and that it comes from the first backend
  • change the application's configuration in sozu to remove the first backend and replace it with the second one
  • the client sends a request
  • check that we receive a response, and that it comes from the second backend

add a README

indicate what the project does, how to run it, how to add tests, how to update containers

new test: websocket

  • set up an application that can upgrade from a HTTP request to a websocket stream
  • the client does a HTTP request with upgrade to websocket
  • we verify that communication can happen both ways and that all the data is coming through

example client:

var WebSocket = require('ws');

/*
var ws = new WebSocket('ws://echo.websocket.org:8080/', {
  origin: 'http://websocket.org'
});
*/

var ws = new WebSocket('ws://lolcatho.st:33450/', {
  origin: 'http://lolcatho.st:33450',
  protocolVersion: 8
});

ws.on('open', function open() {
  console.log('connected');
  ws.send(Date.now().toString(), {mask: true});
});

ws.on('close', function close() {
  console.log('disconnected');
});

ws.on('message', function message(data, flags) {
  console.log('Roundtrip time: ' + (Date.now() - parseInt(data)) + 'ms', flags);

  setTimeout(function timeout() {
    ws.send(Date.now().toString(), {mask: true});
  }, 500);
});

example server code:

var WebSocketServer = require('ws').Server;
//var ws = new WebSocket('ws://lolcatho.st:1026/');
var ws = new WebSocketServer({ port: 1026, origin: 'http://lolcatho.st:8080' });

ws.on('open', function open() {
  console.log("client opened");
  ws.send('opened');
});

ws.on('message', function(data, flags) {
  // flags.binary will be set if a binary data is received.
  // flags.masked will be set if the data was masked.
  console.log("message: "+message);
  ws.send(data);
});

ws.on('connection', function connection(c) {
		console.log("got connection");
  c.on('message', function message(data) {
		console.log("message: "+data);
    // Broadcast to everyone else.
    /*ws.clients.forEach(function each(client) {
      if (client !== c) client.send(data);
    });*/
    c.send(data);
  });
});

new test: path begin

  • set up two applications
    • one with only a domain name for its frontend (this one may not have any backend server)
    • one with the same domain name, and a path prefix
  • the client does a request with the domain name and the path prefix
  • the second application should be the one responding

Bug: Address already in use

Maybe the configuration of ipv6 for the daemon break the tests.
Here is the stack trace I'll take a look

HaproxyContainerTest STANDARD_ERROR
    SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
    SLF4J: Defaulting to no-operation (NOP) logger implementation
    SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

HaproxyContainerTest STANDARD_OUT
            ℹ︎ Checking the system...
            ✔ Docker version should be at least 1.6.0
            ✔ Docker environment should have more than 2GB free disk space
            ✔ File should be mountable

NodeBackendContainerTest > testCorrectResponseFromNodeBackend STANDARD_ERROR
    Nov 02, 2018 11:53:13 AM NodeBackendContainerTest testCorrectResponseFromNodeBackend
    INFO: Running node backend at http://localhost:32843

SozuContainerTest > testStickySessions FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

SozuContainerTest > testRetryPolicy FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

SozuContainerTest > testHttpsredirect FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

SozuContainerTest > shouldGet100ContinueStatusCode FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

SozuContainerTest > shouldWorkWithWebsocket FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

SozuContainerTest > testPathbeginWithKeepAlive FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

SozuContainerTest > testPathbegin FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

SozuContainerTest > shouldNotPanicWhenHostIsEmpty FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

SozuContainerTest > testRemoveBackendBetweenRequests FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

SozuContainerTest > testCorrectResponseFromSozu FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

Gradle Test Executor 1 finished executing tests.

> Task :test

SozuContainerTest > testHttpCircuitBreaker FAILED
    org.testcontainers.containers.ContainerLaunchException: Container startup failed

        Caused by:
        org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception

            Caused by:
            org.testcontainers.containers.ContainerLaunchException: Could not create/start container

                Caused by:
                java.lang.reflect.UndeclaredThrowableException

                    Caused by:
                    java.lang.reflect.InvocationTargetException

                        Caused by:
                        com.github.dockerjava.api.exception.DockerException: {"message":"Address already in use"}

13 tests completed, 11 failed

new test: circuit breaker

  • set up an application with a backend, but do not start that backend
  • the client sends a request
  • check that sozu sends back a 503 response after some attempted backend connections

testStickySessions failed

The test testStickySessions failed with the log:

SozuContainerTest > testStickySessions FAILED
    java.lang.NullPointerException
        at SozuContainerTest.testStickySessions(SozuContainerTest.java:243)

Dump TCP packets of a sozu container

That can be interesting to have a feature to dump the packets of a sozu container.
A way to do that is by adding tcpdump at the sozu image and use execCommand method.

running in CI

how could this project be run regularly, on the master branch's modifications and on pull requests? Could we wrap it under gitlab runners on Clever Cloud's infrastructure? ping @k33g

Allow to create docker network bridge dynamically

We must create a local bridge network named "my-net" to run the test suite :
docker network create --driver=bridge --subnet=172.18.0.0/16 my-net
It's because we have to know the ipv4 address backends to register them in the sozu config.toml

testcontainers provide this feature but it's not documented for the moment (issues/509) . They have unit tests to show how to use this feature.

new test: sticky sessions

  • set up an application with two backends, and sticky sessions (possibly with a specific sticky id)
  • the two backends send their id in the response
  • the client sends a request (no keep-alive)
  • we check that we got a sticky session cookie, and we store the id of the backend that answered
  • the client sends a second request, with the session cookie
  • we check that we got a response from the same backend
  • we remove all the backends and set up new ones
  • the client sends a third request, with the session cookie
  • we check that we got a response from a different backend, and a new session cookie

new test: chunked response

  • the client does a GET / request
  • the server sends a file as chunked response
  • the client receives all the packets and verify the file hash (we should have received the entire file)

it might be interesting to have variations in chunk size and frequency (example: send some 1 byte chunks, or wait a few milliseconds before sending the next chunk)

new test: 100 continue

The 100 HTTP status code needs special handling, because the server will do a first validation and send a response, then the client will send the rest of the response. It can be used when the client wants to send a huge file but check authorization before sending the actual data.

References:
https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.1.1
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/100

example server code:

const http = require('http');
const net = require('net');
const url = require('url');

const server = http.createServer()

// listen for checkContinue events
server.on('checkContinue', function(req, res) {
  console.log("check_continue")
  req.checkContinue = true;
  handlePostFile(req, res); // call express directly to route the request
});

// example request handler
function handlePostFile(req, res) {
  console.log("handle post file")


  // was this a conditional request?
  if (req.checkContinue === true) {
    req.checkContinue = false;
    // send 100 Continue response
    res.writeContinue();
    
    console.log("wrote continue");
    // client will now send us the request body
  }
  
  // collect request body
  var body = '';
  req.once('readable', function() {
    var chunk;
    console.log("readable()");

    while ((chunk = req.read()) !== null) {
      console.log("read");
      body += chunk;
    }

    // do something with request body
  });
}

server.listen(1029)

new test: retry policy

  • we set up an application with two backends, but only one of the backend server is actually up
  • the client does a serie of requests (no keep-alive)
  • we check that sozu correctly reports one of the backend as down after a few requests

new test: proxy protocol and x-forward-proto

  • set up an application with:
    • one backend
    • a sozu in tcp mode with send_proxy=true
    • a sozu behind the send_proxy with expect_proxy=true and in http mode
  • the backend send back the forwarded headers present in the request.
  • the client sends a request (no keep-alive)
  • we check that we got a response from the backend with the forwarded headers expected. The headers should be:
    • X-Forwarded-Proto: http
    • X-Forwarded-For: <identifier>
    • X-Forwarded-Port: <port>
    • Forwarded: proto=http;for=<identifier>;by=<host>
  • check in the access log of sozu-tcp|http the client ip and the port is present

handling container updates

apparently testcontainers downloads a container once, and does not check for updates afterwards. We should either have a command to update the container we use, or have the tool check for updates before running

new test: path begin and keep alive

  • set up two applications
    • one with only a domain name for its frontend
    • one with the same domain name, and a path prefix
  • the client does a request with the domain name (but not the path prefix of the second app), making sure it's in keep-alive
  • we verify that the response comes from the first application and is correct
  • the client does a new request, on the same connection, with the domain name and the path prefix
    • we verify that the response comes from the second application and is correct

new test: early response

  • the client wants to send a huge file (we can just make up an infinite stream of data)
  • the server answers early with a 401 (it does not wait until it gets the whole file to send the response)
  • we check that the client receives the response while it is still sending data

this is a way for some servers to stop big uploads if the client did not send a Expect: 100-continue in its request headers

capture stdout

there's probably a way to get what's written on stdout in the sozu container, but also in the client and backend containers. This output should be displayed in case the test fails

new test: HTTP -> HTTPS redirect

  • set up an app with two frontends
    • one in HTTP, with HTTPS redirection
    • one in HTTPS
  • the client does a HTTP request
  • verify that the proxy answers with a 301 to the HTTPS version
  • the client does a HTTPS request
  • verify that the server gets the correct protocol in the Forwarded-* headers

ipv6?

is it possible to perform some tests over ipv6? To check that sozu works properly with ipv6 addresses

Send command to sozu

For some integration test e.g. #15 we have to send command through the sozu socket unix in the container.
A solution could be to add sozuctl binary in the sozu Dockerfile and use docker exec endpoint to make commands.

new test: load balancing

  • set up an application with two backends, each with different weights
  • each backend is set up to indicate its id (number or something else) in the response
  • the client does a serie of requests (no keep-alive, otherwise the session would just reuse the backend connection)
  • the client counts how many requests were answered by which backend
  • we check that it corresponds to the weights

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.