High memory usage - 9,6GiB about docker-documentserver

[jkaberg]

By passing these environment variables to the container, I worked around the issue (now using ~300mb of ram).

      - NGINX_WORKER_PROCESSES=1
      - NGINX_WORKER_CONNECTIONS=1024

I'd still urge you to change the defaults tho.

[jkaberg]

Looking into this further reveals,

root@69ccdaf9dcd5:/# ps aux  | awk '{print $6/1024 " MB\t\t" $11}'  | sort -n
0 MB            COMMAND
0.09375 MB              /usr/lib/erlang/erts-5.10.4/bin/epmd
0.484375 MB             bash
0.835938 MB             /bin/sh
0.839844 MB             /bin/sh
0.867188 MB             inet_gethost
0.988281 MB             awk
1.625 MB                inet_gethost
2.52734 MB              ps
3.10547 MB              bash
3.12891 MB              /usr/bin/redis-server
3.30469 MB              postgres:
3.30469 MB              postgres:
3.71094 MB              bash
3.73047 MB              postgres:
4.14453 MB              nginx:
4.60156 MB              postgres:
4.94531 MB              nginx:
5.71875 MB              postgres:
14.5273 MB              /usr/bin/python
20.0703 MB              /usr/lib/postgresql/9.3/bin/postgres
22.2656 MB              statsd
26.4922 MB              node
37.5625 MB              node
37.6211 MB              node
40.5391 MB              /usr/bin/nodejs
45.1094 MB              /usr/bin/nodejs
45.3672 MB              node
49.5547 MB              /usr/bin/nodejs
97.168 MB               /usr/lib/erlang/erts-5.10.4/bin/beam.smp
404.887 MB              nginx:
405.301 MB              nginx:
405.352 MB              nginx:
405.367 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
405.371 MB              nginx:
root@69ccdaf9dcd5:/#

Any idea @agolybev?

[agolybev]

Hi @jkaberg, thank you for report.
By default nginx run one process per each processor core inside the container. I'll think about whether this is necessary.

[agolybev]

@jkaberg, we'll try testing one nginx worker per docker container by default: 990a0ab

[agolybev]

This is output from one of our production servers:

$ ps aux  | awk '{print $6/1024 " MB\t\t" $11}'  | sort -n | grep nginx
0.191406 MB		nginx:
0.28125 MB		nginx:
15.0977 MB		nginx:
15.1484 MB		nginx:
16.2422 MB		nginx:
16.3828 MB		nginx:

I really don't know why the nginx inside your docker container uses too much memory.

[jkaberg]

@agolybev the output of the grep on my server results in 24 "cores" (12 core + 12 hyperthreads)

root@89c393a54433:/# grep processor /proc/cpuinfo | wc -l
24
root@89c393a54433:/# ps aux  | awk '{print $6/1024 " MB\t\t" $11}'  | sort -n
0 MB		COMMAND
0.09375 MB		/usr/lib/erlang/erts-5.10.4/bin/epmd
0.835938 MB		/bin/sh
0.839844 MB		/bin/sh
0.9375 MB		awk
0.964844 MB		inet_gethost
1.57422 MB		inet_gethost
2.08594 MB		sort
2.48047 MB		ps
3.14844 MB		bash
3.25 MB		postgres:
3.33984 MB		/usr/bin/redis-server
3.46484 MB		postgres:
3.72656 MB		bash
4.19141 MB		nginx:
4.51953 MB		postgres:
5.18359 MB		postgres:
5.34375 MB		nginx:
5.78125 MB		postgres:
8.55469 MB		nginx:
14.5352 MB		/usr/bin/python
20.0781 MB		/usr/lib/postgresql/9.3/bin/postgres
23.2461 MB		statsd
27.1484 MB		node
38.0312 MB		node
38.1289 MB		node
43.0586 MB		/usr/bin/nodejs
45.5156 MB		/usr/bin/nodejs
47.3711 MB		node
57.5117 MB		/usr/bin/nodejs
96.3086 MB		/usr/lib/erlang/erts-5.10.4/bin/beam.smp

That might give a hint to the high memory usage.

Btw, could you push a new image to Docker Hub? The current one is 1+month old

[agolybev]

Hi @jkaberg, I cann't push new image to Docker Hub right now. It will be released in Document Server v5.0.1 in the near future.

[Abijeet]

In case anyone is searching, I had similar issue where the nginx process was taking up 22% of the 2GB RAM (450MB), 1 core server that I had.

Setting these in the docker-compose.yml helped:

      - NGINX_WORKER_PROCESSES=1
      - NGINX_WORKER_CONNECTIONS=20

Which is fine in my case since the server will not be used by more than 3 people at the same time. This dropped my nginx RAM usage to 17MB.