microservices-demo / load-test

A load-test script & container for Sock Shop

DEPRECATED: Load / Integration Tests

These tests simulate actual end user usage of the application. They are used to validate the overall functionality and can also be used to put simulated load on the system. The tests are written using locust.io

Parameters

• [host] - The hostname (and port if applicable) where the application is exposed. (Required)
• [number of clients] - The nuber of concurrent end users to simulate. (Optional: Default is 2)
• [number of requests] - The total number of requests to run before terminating the tests. (Optional: Default is 10)

Running locally

Requirements

• locust pip install locustio

./runLocust.sh -h [host] -c [number of clients] -r [number of requests]

Running in Docker Container

• Build docker build -t load-test .
• Run docker run load-test -h [host] -c [number of clients] -r [number of requests]

load-test's Issues

Locust fails with unknown flag

Running the load test gives an error: no such option: --num-request message and exits.

Reproduction

1. Install locust as per instructions pip install locustio
2. Execute runLocust.sh with arguments

or, using Docker

1. Build docker image as per instructions docker build -t load-test .
2. Execute docker run load-test with arguments

Expected Behavior

Locust executes.

Actual Behavior

The following error:

$ ./runLocust.sh -h sock-shop.example.com -c 5 -r 10                                                                    
Default Locust file: locustfile.py
Will run locustfile.py against sock-shop.example.com. Spawning 5 clients and 10 total requests.
Usage: locust [options] [LocustClass [LocustClass2 ... ]]

locust: error: no such option: --num-request
done

or, using Docker

$ docker run load-test -h sock-shop.example.com -c 5 -r 10                
Locust file: /config/locustfile.py
Will run /config/locustfile.py against sock-shop.example.com. Spawning 5 clients and 10 total requests.
Usage: locust [options] [LocustClass [LocustClass2 ... ]]

locust: error: no such option: --num-request
done

Environment

Tool	Version
Python	Python 2.7.15rc1
Locust	Locust 0.11.0
Docker	18.09.2

Run script fails on HTTPS endpoints

When load testing a sock shop secured by HTTPS, there is no way to give the https:// address as the script places the prefix http:// on the host.

As a workaround, a user can execute locust directly, for example:

locust --host=https://sock-shop.example.com -f locustfile.py --clients 10 --hatch-rate=5 --run-time=1m --no-web --only-summary

Add travis to auto pushing

Add travis and auto docker.io pushing to this repo

Error when running load test

Hi
I am encountering this error when I am running the load-test container.
standard_init_linux.go:228: exec user process caused: no such file or directory

The command I am running: docker run --rm --net=host weaveworksdemos/load-test -h localhost -r 1000 -c 5

This started after I ran docker system prune -a -f to clean up space on my VM that docker was using.
Some details regarding my environment:

status":406,"error":"Not Acceptable" message":"Payment declined: amount exceeds 100.00

While running load-tests, I see lot of error responses in front end logs -

Order response: {"timestamp":1543518975262,"status":406,"error":"Not Acceptable","exception":"works.weave.socks.orders.controllers.OrdersController$PaymentDeclinedException","message":"Payment declined: amount exceeds 100.00","path":"/orders"}
Order response: {"statusCode":406,"body":{"timestamp":1543519009697,"status":406,"error":"Not Acceptable","exception":"works.weave.socks.orders.controllers.OrdersController$PaymentDeclinedException","message":"Payment declined: amount exceeds 100.00","path":"/orders"},"headers":{"x-application-context":"orders:80","content-type":"application/json;charset=UTF-8","date":"Thu, 29 Nov 2018 19:16:49 GMT","x-envoy-upstream-service-time":"18","server":"envoy","connection":"close","transfer-encoding":"chunked"},"request":{"uri":{"protocol":"http:","slashes":true,"auth":null,"host":"orders","port":80,"hostname":"orders","hash":null,"search":null,"query":null,"pathname":"/orders","path":"/orders","href":"http://orders/orders"},"method":"POST","headers":{"accept":"application/json","content-type":"application/json","content-length":232}}}

Where is the amount populated while making calls from load-test microservice? and how to fix these errors?

Loadtest not deploying to sock-sock namespace properly

Deployed the sock-shop to a Kubernetes cluster in the usual way:

kubectl apply -n sock-shop -f deploy/kubernetes/manifests

Zipkin and the loadtest are returning the following errors:

namespace "zipkin" created
the namespace from the provided object "loadtest" does not match the namespace "sock-shop". You must pass '--namespace=loadtest' to perform this operation.
the namespace from the provided object "zipkin" does not match the namespace "sock-shop". You must pass '--namespace=zipkin' to perform this operation.

Load test is looking for edge router

Installing on bare metal server, and try to run the loadtest with the following:

docker run -ti --rm --name=LOAD_TEST weaveworksdemos/load-test -h edge-router -r 100 -c 2 host IP: node-port

returns: Locust file: /config/locustfile.py
edge-router is not accessible

Load-test fails, especially with large number of clients

I'm seeing that the load-test is rather unstable. Especially when running a larger number of clients (say 20000 to make it fail faster). I think it's more an issue with the orders and user services, but figure I should open this issue against the load-test, since running the load-test is the only way to see this issue.

When running the load test with say 20000 clients. There are two sets of errors I see. The first is a reoccurring log in error. Kind of seems like there's a sequencing issue in the load-test script. The user doesn't get logged in before a checkout is performed? The logs show something like this:

...
Error: User not logged in.
500
Error with log in: true
...

These errors appear as soon as you start running the load-test. It's very hard to miss. The front-end service produces the errors.

The second error that happens is that the front-end service crashes and restarts. It looks to crash because it receives an empty response from the orders service. The issue happens within 5-10 minutes, sometimes faster. It also happens faster when the load-test is executed from a remote machine, not locally on the machine that is running the weavesocks demo.

When this crash happens you see a log that looks something like this:

Received response: "{"error":"hystrix: max concurrency","status_code":500,"status_text":"Internal Server Error"}\n"
/usr/src/app/api/orders/index.js:67
var customerlink = jsonBody._links.customer.href;
^

TypeError: Cannot read property 'customer' of undefined
at Request._callback (/usr/src/app/api/orders/index.js:67:47)
at Request.self.callback (/usr/src/app/node_modules/request/request.js:186:22)
at emitTwo (events.js:87:13)
at Request.emit (events.js:172:7)
at Request. (/usr/src/app/node_modules/request/request.js:1081:10)
at emitOne (events.js:77:13)
at Request.emit (events.js:169:7)
at IncomingMessage. (/usr/src/app/node_modules/request/request.js:1001:12)
at IncomingMessage.g (events.js:260:16)
at emitNone (events.js:72:20)
at IncomingMessage.emit (events.js:166:7)

npm ERR! Linux 4.4.36-8-default
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "start"
npm ERR! node v4.2.6
npm ERR! npm v3.5.2
npm ERR! code ELIFECYCLE
npm ERR! [email protected] start: node server.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script 'node server.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the weave-demo-frontend package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node server.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs weave-demo-frontend
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls weave-demo-frontend
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! /usr/src/app/npm-debug.log

Note, I did try a newer version of nodeJS and npm, but the error still appears like above. This suggests it's probably an issue with the demo.

To run the services, I used "docker-compose up" in the following directory:
/microservices-demo/deploy/docker-compose

Example command to use with the load-test:
./runLocust.sh -h <host_name> -r 20000 -c 20000

406 Client Error: Not Acceptable for url:...../orders

By running the test via
./runLocust.sh -h localhost:30001 -r 100 -c 1,

we occasionally (not every time) got
POST /orders: "HTTPError(u'406 Client Error: Not Acceptable for url: http://localhost:30001/orders',)"

It is occasional, not every time...
Could anyone point out how to fix this?

Loadtest doesn't run

In a local setup the command:

docker run -ti --rm --name=LOAD_TEST --net=dockercompose_default weaveworksdemos/load-test -h localhost edge-router -r 100 -c 2

returns,

localhost is not accessible

Is it normal to see some 500 errors for logins in front-end log?

When running the load-test service with the default options (-d 60 -r 200 -c 2), I get good results in the user-sim log itself with just a few sporadic errors.

However, I see quite a few HTTP 500 errors for logins in the front-end service's log looking like:

[stdout] Received login request
[stdout] 500
[stdout] Error with log in: true
[stdout] �[0mGET /login �[33m401 �[0m126.906 ms - -�[0m

Note that some logins do succeed. For instance:

[stdout] Received login request
[stdout] {"user":{"firstName":"","lastName":"","username":"barronkelly","id":"58e79f1b0d8eec0009da0e1d","_links":{"addresses":{"href":"http://user.apcera.local:8080/customers/58e79f1b0d8eec0009da0e1d/addresses"},"cards":{"href":"http://user.apcera.local:8080/customers/58e79f1b0d8eec0009da0e1d/cards"},"customer":{"href":"http://user.apcera.local:8080/customers/58e79f1b0d8eec0009da0e1d"},"self":{"href":"http://user.apcera.local:8080/customers/58e79f1b0d8eec0009da0e1d"}}}}
[stdout] 58e79f1b0d8eec0009da0e1d
[stdout] Merging carts for customer id: 58e79f1b0d8eec0009da0e1d and session id: Ygumay5VY1doTJS18T-l_Mwc0gwsfcW1
[stdout] Carts merged.
[stdout] Sent cookies.
[stdout] �[0mGET /login �[32m200 �[0m152.222 ms - 13�[0m

I'm wondering if the login errors I see in the front-end log are intentional and part of the test. If not, why are the errors not reflected in the user-sim log and statistics?

Thanks,
Roger Berlind

Review comprehensive nature of loadtest

Right now it often seems to be missing whole chunks for long periods of time (or even forever).

Insure that the endpoint tests are comprehensive across all services and steady.

How to simulate GET /login method with locust?

• Describe the expected behaviour and the actual behaviour

self.client.get("/login", headers={"Authorization":"Basic %s" % base64string})

The codes of locustfile.py which simulates the login method to sock-shop seems not work with locust return:

GET /login: "HTTPError('401 Client Error: Unauthorized for url: http://example-sock-shop/login')"  

And I checked the client.js file of sock-shop, the corresponding part of login codes is:

beforeSend: function (xhr) {
            xhr.setRequestHeader("Authorization", "Basic " + btoa(username + ":" + password));
        }

which means the operation of locust is correct, but why does it not work?

Besides, I have already change the "username: password" str to base64 bytes

Invalid credentials for docker

The credentials to push to docker hub are invalid.

https://travis-ci.org/microservices-demo/load-test/builds/234798953#L684

JSON decoding error

After deploying the microservices-demo on ECS (with the cloud formation template) and making suer that the Stack finished creating, I proceeded to load-test the deployment with docker run weaveworksdemos/load-test -h host -c 2 -r 1000

It works for the most part, but I continuously get this error:

[2016-11-16 11:01:22,451] 4c4c9be0f3bf/ERROR/stderr: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/locust/core.py", line 267, in run
    self.execute_next_task()
  File "/usr/local/lib/python2.7/site-packages/locust/core.py", line 293, in execute_next_task
    self.execute_task(task["callable"], *task["args"], **task["kwargs"])
  File "/usr/local/lib/python2.7/site-packages/locust/core.py", line 305, in execute_task
    task(self, *args, **kwargs)
  File "/config/locustfile.py", line 31, in purchaseItem
    self.createCustomer()
  File "/config/locustfile.py", line 83, in createCustomer
    self.cust_id = customer.json()["id"]
  File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 826, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded