hopsoft / docker-graphite-statsd

Docker image for Graphite & Statsd

Table of Contents

• Docker Image for Graphite & Statsd
  • Get Graphite & Statsd running instantly
  • Quick Start
    • Includes the following components
    • Mapped Ports
    • Mounted Volumes
    • Base Image
  • Start Using Graphite & Statsd
    • Send Some Stats
    • Visualize the Data
  • Secure the Django Admin
  • Change the Configuration
  • Statsd Admin Management Interface
  • Secure Grafana
  • Connect Grafana to Graphite
  • A Note on Volumes
  • Memcached config
  • Additional Reading
  • Contributors

NOTICE: Check out the official Graphite & Statsd Docker image at https://github.com/graphite-project/docker-graphite-statsd

Docker Image for Graphite & Statsd

Get Graphite & Statsd running instantly

Graphite & Statsd can be complex to setup. This image will have you running & collecting stats in just a few minutes.

Quick Start

docker run -d\
 --name graphite\
 --restart=always\
 -p 80:80\
 -p 81:81\
 -p 2003-2004:2003-2004\
 -p 2023-2024:2023-2024\
 -p 8125:8125/udp\
 -p 8126:8126\
 hopsoft/graphite-statsd

This starts a Docker container named: graphite

That's it, you're done ... almost.

Includes the following components

• Nginx - reverse proxies the graphite dashboard
• Graphite - front-end dashboard
• Carbon - back-end
• Statsd - UDP based back-end proxy
• Grafana - front-end dashboard (more refined than Graphite)

Mapped Ports

Host	Container	Service
80	80	nginx - grafana
81	81	nginx - graphite
2003	2003	carbon receiver - plaintext
2004	2004	carbon receiver - pickle
2023	2023	carbon aggregator - plaintext
2024	2024	carbon aggregator - pickle
8125	8125	statsd
8126	8126	statsd admin

By default, statsd listens on the UDP port 8125. If you want it to listen on the TCP port 8125 instead, you can set the environment variable STATSD_INTERFACE to tcp when running the container.

Mounted Volumes

Host	Container	Notes
DOCKER ASSIGNED	/opt/graphite/conf	graphite config
DOCKER ASSIGNED	/opt/graphite/storage	graphite stats storage
DOCKER ASSIGNED	/etc/grafana	grafana config
DOCKER ASSIGNED	/etc/nginx	nginx config
DOCKER ASSIGNED	/opt/statsd	statsd config
DOCKER ASSIGNED	/etc/logrotate.d	logrotate config
DOCKER ASSIGNED	/var/log	log files

Base Image

Built using Phusion's base image.

• All Graphite related processes are run as daemons & monitored with runit.
• Includes additional services such as logrotate.

Start Using Graphite & Statsd

Send Some Stats

Let's fake some stats with a random counter to prove things are working.

while true; do echo -n "example:$((RANDOM % 100))|c" | nc -w 1 -u 127.0.0.1 8125; done

Visualize the Data

Open Graphite in a browser.

• http://localhost:81/dashboard
• http://localhost:81/render?from=-10mins&until=now&target=stats.example

Secure the Django Admin

Update the default Django admin user account. The default is insecure.

• username: root
• password: root
• email: [email protected]

First login at: http://localhost:81/account/login Then update the root user's profile at: http://localhost:81/admin/auth/user/1/

Change the Configuration

Read up on Graphite's post-install tasks. Focus on the storage-schemas.conf.

1. Stop the container docker stop graphite.
2. Find the configuration files on the host by inspecting the container docker inspect graphite.
3. Update the desired config files.
4. Restart the container docker start graphite.

Note: If you change settings in /opt/graphite/conf/storage-schemas.conf be sure to delete the old whisper files under /opt/graphite/storage/whisper/.

---

Important: Ensure your Statsd flush interval is at least as long as the highest-resolution retention. For example, if /opt/statsd/config.js looks like this.

flushInterval: 10000

Ensure that storage-schemas.conf retentions are no finer grained than 10 seconds.

[all]
pattern = .*
retentions = 5s:12h # WRONG
retentions = 10s:12h # OK
retentions = 60s:12h # OK

Statsd Admin Management Interface

A management interface (default on port 8126) allows you to manage statsd & retrieve stats.

# show all current counters
echo counters | nc localhost 8126

More info & additional commands.

Secure Grafana

Update the default Grafana admin account. The default is insecure.

• username: admin
• password: admin
• email: admin@localhost

First login at: http://localhost Then update the admin user's profile at: http://localhost/admin/users/edit/1

Connect Grafana to Graphite

Visit http://localhost/datasources/new Then configure the Graphite data source with the URL http://localhost:81

A Note on Volumes

You may find it useful to mount explicit volumes so configs & data can be managed from a known location on the host.

Simply specify the desired volumes when starting the container.

docker run -d\
 --name graphite\
 --restart=always\
 -v /path/to/graphite/configs:/opt/graphite/conf\
 -v /path/to/graphite/data:/opt/graphite/storage\
 -v /path/to/statsd:/opt/statsd\
 hopsoft/graphite-statsd

Note: The container will initialize properly if you mount empty volumes at /opt/graphite, /opt/graphite/conf, /opt/graphite/storage, or /opt/statsd

Memcached config

If you want Graphite to use an existing Memcached server, set the following environment variables:

docker run -d\
 --name graphite\
 --restart=always\
 -p 80:80\
 -p 2003-2004:2003-2004\
 -p 2023-2024:2023-2024\
 -p 8125:8125/udp\
 -p 8126:8126\
 -e "MEMCACHE_HOST=127.0.0.1:11211"\  # Memcached host(s) comma delimited
 -e "CACHE_DURATION=60"\              # in seconds
 hopsoft/graphite-statsd

Additional Reading

• Introduction to Docker
• Official Statsd Documentation
• Practical Guide to StatsD/Graphite Monitoring
• Configuring Graphite for StatsD

Contributors

Build the image yourself.

1. git clone https://github.com/hopsoft/docker-graphite-statsd.git
2. docker build -t hopsoft/graphite-statsd .

docker-graphite-statsd's Issues

Build fails

Even with my pull request modification to fix the spacing on the Dockerfile:35, the file does not exists and build fails.

graphite 1.0.1 support

would be nice to have since according to documnetation it is the production ready version,

local_settings from ENV

Would be really handy if you could provide config overrides in /opt/graphite/webapp/graphite/local_settings.py through the environment as well, would go nicely in docker-compose.yml for instance.

mounted /opt/graphite/storage/ but no data source response

no storage mount:

$ kubectl exec -it grafana-3653661013-nh6vw --namespace grafana -c graphite -- bash
root@grafana-3653661013-nh6vw:/# cd /opt/graphite/storage/
root@grafana-3653661013-nh6vw:/opt/graphite/storage# ls
carbon-aggregator-a.pid  carbon-cache-a.pid  graphite.db  index  whisper

dashboard responds fine

with /opt/graphite/storage mounted:

$ kubectl exec -it grafana-3653661013-6d37x --namespace grafana -c graphite -- bash
root@grafana-3653661013-6d37x:/# cd /opt/graphite/storage/
root@grafana-3653661013-6d37x:/opt/graphite/storage# ls
carbon-aggregator-a.pid  carbon-cache-a.pid  conf  graphite.db  storage  whisper

Graphite encountered an unexpected error while handling your request. & Template init failed [object object]

graphite web crashes no table auth_user

After launching the container graphite web will return an exception no table auth_user, I suspect that syncdb failed for some reason.

To reproduce launch a container with following command:

sudo docker run \
    --name="graphite" \
    -d -i -t \
    -p 80:80 \
    -p 2003:2003 \
    -p 8125:8125/udp \
    -v /var/log/graphite:/var/log \
    -v /data/graphite/storage:/opt/graphite/storage \
    -v /data/graphite/conf:/opt/graphite/conf \
    hopsoft/graphite-statsd /opt/hopsoft/graphite-statsd/start

To pindown the problem, it happens when you mount /opt/graphite/storage as data volume on the host. Changing /data/graphite/storage:/opt/graphite/storage to /opt/graphite/storage is a work around to fix the problem.

This is the trace when i visit http://localhost

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 101, in get_response
    request.path_info)
  File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 252, in resolve
    sub_match = pattern.resolve(new_path)
  File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 158, in resolve
    return ResolverMatch(self.callback, args, kwargs, self.name)
  File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 164, in _get_callback
    self._callback = get_callable(self._callback_str)
  File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 124, in wrapper
    result = func(*args)
  File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 91, in get_callable
    lookup_view = getattr(import_module(mod_name), func_name)
  File "/usr/local/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/opt/graphite/webapp/graphite/browser/views.py", line 20, in <module>
    from graphite.util import getProfile, getProfileByUsername, defaultUser, json
  File "/opt/graphite/webapp/graphite/util.py", line 69, in <module>
    defaultUser = User.objects.get(username='default')
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 132, in get
    return self.get_query_set().get(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 344, in get
    num = len(clone)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 82, in __len__
    self._result_cache = list(self.iterator())
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 273, in iterator
    for row in compiler.results_iter():
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 680, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", line 234, in execute
    return Database.Cursor.execute(self, query, params)
DatabaseError: no such table: auth_user

StatsD configured with legacy namespace

While it is supposed to be a breaking change, I think that namespace should be configured with non legacy namespace.

https://github.com/etsy/statsd/blob/master/docs/namespacing.md

Wrong ports description

https://github.com/hopsoft/docker-graphite-statsd#mapped-ports

2003/2004 are carbon receivers (carbon cache)
2023/2024 are carbon aggregators

mount /opt/graphite wont work, or am I doing it wrong ?

As suggested in README: -v /path/to/ebs/graphite:/opt/graphite
if an empty directory on host machine mounted to /opt/graphite in container, graphite won't work.

/usr/bin/python: can't open file '/opt/graphite/bin/carbon-cache.py': [Errno 2] No such file or directory
...

I tried to mount only /opt/graphite/storage, but there are graphite.db and graphite.db-journal already created in the image, which should be preserved.

Finally made it to work by mounting only /opt/graphite/storage/whisper. But I'm not sure if this is the only directory that consumes disk-space.

Also if mount /the/mount/point from the host to /var/log in container, /the/mount/point/nginx and /the/mount/point/graphite should be created on host before running the container.

port 8125 on localhost

Hi

For some reason port 8125 it's not opened on the host.

  graphite:
    container_name: graphite
    image: hopsoft/graphite-statsd
    ports:
      - "8080:80"
      - "8125:8125/udp"
      - "2003-2004:2003-2004"
      - "2023-2024:2023-2024"
      - "8126:8126"

All the other ports are exposed on the host except 8125.

The mounted volume /opt/graphite contains the graphite application and libraries, not just config and logs

What is the reason for making /opt/graphite a docker volume instead of just the storage and log directories by themselves? The graphite binaries are installed in /opt/graphite, not just the config files and logs. If you mount that volume on the host, or in a docker volume container, what happens when you update the image to graphite 0.9.14? How will the scripts in /opt/graphite/bin and /opt/graphite/lib be updated?

Missing carbon-relay

You guys forget to add carbon-relay process to the docker

carbon LOGROTATION doesnt adds up well with logrotate

docker-graphite-statsd/conf/opt/graphite/conf/carbon.conf

Line 34 in 2141fe1

ENABLE_LOGROTATION = True

i always ends en with tons of logs 4-8GB of logs that is higher than my whisper database.

since carbon do move logs file from file.log to file.log.YYYY_MM_DD
logrotate does too from file.log to file.log.X=[1-10]

`-rwxr-xr-x 1 _graphite _graphite 199M nov. 15 11:48 listener.log
-rwxr-xr-x 1 _graphite _graphite 853M nov. 15 02:49 listener.log.1
-rwxr-xr-x 1 _graphite _graphite 271M nov. 10 23:59 listener.log.2017_11_10

`

Graphite is broken

On the latest version, when I try to access port 80, I get:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 101, in get_response
    request.path_info)
  File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 252, in resolve
    sub_match = pattern.resolve(new_path)
  File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 158, in resolve
    return ResolverMatch(self.callback, args, kwargs, self.name)
  File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 164, in _get_callback
    self._callback = get_callable(self._callback_str)
  File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 124, in wrapper
    result = func(*args)
  File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 91, in get_callable
    lookup_view = getattr(import_module(mod_name), func_name)
  File "/usr/local/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/opt/graphite/webapp/graphite/browser/views.py", line 20, in <module>
    from graphite.util import getProfile, getProfileByUsername, defaultUser, json
  File "/opt/graphite/webapp/graphite/util.py", line 69, in <module>
    defaultUser = User.objects.get(username='default')
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 132, in get
    return self.get_query_set().get(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 344, in get
    num = len(clone)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 82, in __len__
    self._result_cache = list(self.iterator())
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 273, in iterator
    for row in compiler.results_iter():
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 680, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", line 234, in execute
    return Database.Cursor.execute(self, query, params)
DatabaseError: no such table: auth_user

Reduce Docker image size and layers, re-organize

When I pulled the image to a new machine with disk space constraints, I noticed that it contained a large number of layers and had a pretty large space requirement.

So I applied a number of optimizations to a fork.

You can see the effects per optimization here: https://hub.docker.com/r/hochzehn/docker-graphite-statsd/tags/

Here are the uncompressed numbers (after pull / local build):

statsd_8   704.5 MB
statsd_7   704.5 MB
statsd_6   704.5 MB
statsd_5   765.4 MB
statsd_4   768.3 MB
statsd_3   768.3 MB
statsd_2   769 MB
statsd_1   795.5 MB
master     854.2 MB

They are built on top of each other, i.e. latest is current master branch, 1-no-recommends builds on top of master, 2-single-apt-step on top of 1-no-recommends, etc.

You can find the commits here: https://github.com/hochzehn/docker-graphite-statsd/commits/8-cleanup - one commit per optimization. As I realize now, I should have probably labelled them with 1 to 8 as well.

Why did I not create a pull request, you might wonder? I wanted to put the optimizations up for discussion first, as they require a pretty heavy restructuring of Dockerfile (compare). My largest consideration is splitting installation and configuration of the various components.

Also, some further optimizations could be made, e.g. by dropping vim from the image.

So what do you think? The overall effect is pretty cool, I think, and worth the loss in readability. Clustering up e.g. git installations might even make version changes like #57 a little easier.

mistake in storage-schemas.conf

The following snippet https://github.com/hopsoft/docker-graphite-statsd/blob/master/conf/opt/graphite/conf/storage-schemas.conf#L15

Is wrong.

[default_1min_for_1day]
pattern = .*
retentions = 10s:6h,1min:6d,10min:1800d

We need to keep original section name [stats] and leave default as is at the end of the file.

SNMP Collectd counters are not getting displayed on Graphite UI

I am using collectd to collect my snmp counters and send it to graphite docker.
SNMP counters are coming but graphs are not coming
My snmpd.conf file is

sec.name source community

com2sec notConfigUser default public
com2sec MyRad 172.16.134.222 xxxx
com2sec MyRad 172.16.111.195 xxxx

Second, map the security name into a group name:

groupName securityModel securityName

#group localgroup v1 local
group notConfigGroup v1 notConfigUser

#group localGroup v2c local
group notConfigGroup v2c notConfigUser
group MyRadGroup v2c MyRad
group MyRadGroup v1 MyRad

Third, create a view for us to let the group have rights to:

Make at least snmpwalk -v 1 localhost -c public system fast again.

name incl/excl subtree mask(optional)

#view systemview included .1.3.6.1.2.1.25.1.1
view MyRadView included .1

Finally, grant the group read-only access to the systemview view.

group context sec.model sec.level prefix read write notif

access notConfigGroup "" any noauth exact systemview none none
access MyRadGroup "" any noauth exact MyRadView MyRadView none

###############################################################################

System contact information

It is also possible to set the sysContact and sysLocation system

variables through the snmpd.conf file:

syslocation Unknown (edit /usr/local/share/snmp/snmpd.conf)
syscontact Root root@localhost (configure /etc/snmp/snmp.local.conf)

Added for support of bcm5820 cards.

pass .1.3.6.1.4.1.4413.4.1 /usr/bin/ucd5820stat

###############################################################################

Further Information

See the snmpd.conf manual page, and the output of "snmpd -H".

master agentx
agentXSocket tcp:172.16.134.222:1070

My write graphite plugin configuration in collectd.conf file is

Host "172.16.111.195" Port "2003" Protocol "tcp" LogSendErrors true Prefix "collectd" Postfix "collectd" StoreRates true AlwaysAppendDS false EscapeCharacter "_"

and snmp plugin is

<Data "total_auth_req_list">
Type "Total_Auth_Requests_List"
Table false
Instance ""
Values ".1.3.6.1.2.1.67.1.1.1.1.5.0"

<Host "localhost.localdomain">
Address "172.16.134.222"
Version 2
Community "xxxx"
Collect "total_auth_req_list"
Interval 300

IF I do snmpwalk inside docker container I am getting the counters
snmpwalk -v2c -c xxxx 172.16.134.222 .1.3.6.1.2.1.67.1.1.1.1.5.0

But graph is not getting plotted, I am getting a blank graph.
Thanks in advance

Django admin from docker environment variables

Hi all,

Could you add an ability to automatically change/set django admin/password in docker-composer env variables?

read(net): Connection refused

I'm running Mac OS X 10.11.4 with the new "Docker for Mac (beta)"...

docker run -d --name graphite --restart=always hopsoft/graphite-statsd
beef991eceb6bf482592a00b3eb5b22786d3a6fe00c9810a94ad1bf090e60262

We can see it's running:

CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS                                                                    NAMES
beef991eceb6        hopsoft/graphite-statsd   "/sbin/my_init"          31 seconds ago      Up 29 seconds       80/tcp, 2003-2004/tcp, 2023-2024/tcp, 8126/tcp, 8125/udp                 graphite

I then run the example that suggests testing data going to StatsD:

echo -n "example:$((RANDOM % 100))|c" | nc -w 1 -u localhost 8125
read(net): Connection refused

Grafana and collectd

For personal tests I added grafana and collectd to this useful docker image. Do you think it is worth to push a pull request?

mount empty volumes at /opt/graphite, /opt/graphite/conf, /opt/graphite/storage, or /opt/statsd doesn't work properly

Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 101, in get_response
resolver_match = resolver.resolve(request.path_info)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 340, in resolve
sub_match = pattern.resolve(new_path)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 224, in resolve
return ResolverMatch(self.callback, args, kwargs, self.name)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 231, in callback
self._callback = get_callable(self._callback_str)
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 31, in wrapper
result = func(*args)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 98, in get_callable
mod = import_module(mod_name)
File "/usr/local/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
import(name)
File "/opt/graphite/webapp/graphite/browser/views.py", line 22, in
from graphite.util import getProfile, getProfileByUsername, defaultUser, json
File "/opt/graphite/webapp/graphite/util.py", line 81, in
defaultUser = User.objects.get(username='default')
File "/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 143, in get
return self.get_query_set().get(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 398, in get
num = len(clone)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 106, in len
self._result_cache = list(self.iterator())
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 317, in iterator
for row in compiler.results_iter():
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 775, in results_iter
for rows in self.execute_sql(MULTI):
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 846, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", line 366, in execute
six.reraise(utils.DatabaseError, utils.DatabaseError(*tuple(e.args)), sys.exc_info()[2])
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", line 362, in execute
return Database.Cursor.execute(self, query, params)
DatabaseError: no such table: auth_user

Saving graph permanently in Graphite even after container is removed.

Hi I have spawned a docker container for Graphite and using Telegraf for collecting metrics. The command I have used is ->
docker run -d --name graphite --restart=always -p 80:80 -p 2003-2004:2003-2004 -p 2023-2024:2023-2024 -p 8125:8125/udp -p 8126:8126 -v /opt/graphite/data:/opt/graphite/storage graphiteapp/graphite-statsd
I am logged in to Graphite with root, and I can save the graphs which gets stored in User Graphs and My Graphs folder in the Tree.
Now If I remove the container of Graphite, I cannot see the saved Graph too.
Is there any way I can map my "My Graphs or User Graph" folder to our local and save the graph permanently.
I am stuck on this since long. It would be really helpful of you for your support.

Can I run statsd on both an UDP and TCP at port 8125

Secure Graphite Dashboard

It would be good to be able to secure the Graphite Dashboard, even if only HTTPAuth, and ideally have the password setable as a Docker env var.

Thoughts? Approach?

nginx configuration broken

Hi hopsoft,

I was installing your docker under FreeNAS Corral and it didn't work. Seemed permission stuff at the begining, as the nginx config dir was always empty no matter my efforts and tests, but following the leads I arrived to your page and I believe to have found the mistake in the Dockerfile.

After cloning the statsd git repo you're not changing the WORKDIR like with the other tools you're cloning, and that's why I believe that all the config gets screwed after that point.

Just my guess, if you could please look into it I'll be glad.

Regards, Angel

P.S: I imagine that only new installs suffer from this problem, as far as I've understood. Could have happened in the last build 5 days ago???

PID File Cleanup

Hi,

It seems that, if the container dies unexpectedly (e.g. power loss), it will not restart properly.

The root cause seems to be some PID files left in /opt/graphite/storage/, specifically carbon-cache-a.pid.

It seems that the PID checker code is not robust enough to know that the process died, so it refuses to start it up again.

Since we're running in a Docker container, perhaps an earlier part of the /etc/my_init.d/01_conf_init.sh file could remove those PID files?

Error when mounting volumes

Hi,

When trying to run the container with mounted volumes as mentioned in the README, I get the following errors:

/usr/bin/python: can't open file '/opt/graphite/bin/carbon-aggregator.py': [Errno 2] No such file or directory
/usr/bin/python: can't open file '/opt/graphite/webapp/graphite/manage.py': [Errno 2] No such file or directory

I'm trying to run like so:

sudo docker run -d \
  --name graphite \
  -v /opt/graphite:/opt/graphite \
  -p 80:80 \
  -p 2003:2003 \
  -p 8125:8125/udp \
  hopsoft/graphite-statsd 

When looking into the /opt/graphite on my host - it's empty

Thanks

Too old grafana version

http://docs.grafana.org/installation/debian/#apt-repository
The Dockerfile doesn't contain creating a file /etc/apt/sources.list.d/grafana.list.
The Latest version of the Dockerfile installs Grafana 2.6.0, and latest Grafana version is 6.x.
It would be great to support newer versions 👍

Overriding /opt/statsd will cause statsd daemon to fail.

In the Readme file is suggested to mount a volume to the /opt/statsd directory to override some options.

This is achieve via passing the -v option on the docker run with -v /path/to/statsd:/opt/statsd\.

However this will cause the complete overriding of the statsd directory and will cause the following issue:

module.js:340
    throw err;
          ^
Error: Cannot find module '/opt/statsd/stats.js'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:902:3

The right option to do is to mount specific configuration file overriding single files for example:
-v /path/to/statsd/config.js:/opt/statsd/config.js.

Enable 2003 UDP Port

I want to send plaintext metrics to UDP port 2003 instead of TCP 2003. It is supported in graphite. Can you tell me how to enable this?

Lose carbon data after 24h

Hi everyone,

Somehow I cannot keep the carbon data (cpu usage, memory used...) more than one or two days after the carbon data was created.

It works perfectly well for statsd - I have monitored the response time and number of requests/errors on my endpoints more than a week ago and can still have access to it. On the other hand, the very same day that I sent this data, carbon data was also available, but now it is not visible any more...

Is there any fix I can do?

Thank you in advance :)

Questions following my first real Docker experience

A few quick questions. I know graphite/statsd pretty well but not Docker.

I had to slightly modify the start script to include export DOCKER_HOST=tcp://localhost:4243 and remove sudo which I think is not needed on a Mac.

After running the start script, all seems well:
> docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 885cecdcf6f4 hopsoft/graphite-statsd:latest /opt/hopsoft/graphit 8 seconds ago Up 5 seconds 0.0.0.0:80->80/tcp, 0.0.0.0:2003->2003/tcp, 0.0.0.0:8125->8125/udp graphite

If I understand this correctly it means the ports 80/tcp, 2003/tcp and 8125/udp are exposed from the container to the image/VM as 80/2003 & 8125.

However even after exposing 80/tcp to my host computer via VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port80,tcp,,80,,80" nothing answers at port 80.

With the container started, are the nginx, carbon & statsd servers automatically started as well? Should this be working as I describe it or am I making this too complicated?

docker container can't run, alaways restart

take a few minutes is seem like a joke ,it already cost my 6 hours to run it .

hahahah .
mac os x ,

docker run -d \ --name graphite \ --restart=always \ -p 80:80 \ -p 2003-2004:2003-2004 \ -p 2023-2024:2023-2024 \ -p 8125:8125/udp \ -p 8126:8126 \ hopsoft/graphite-statsd

*** Killing all processes...
Traceback (most recent call last):
File "/sbin/my_init", line 346, in
main(args)
File "/sbin/my_init", line 265, in main
export_envvars()
File "/sbin/my_init", line 88, in export_envvars
f.write(value)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 7-9: ordinal not in range(128)

Multiple Carbon Cache Daemons?

First of all, thanks so much for this project!

Looks like only a single carbon-cache.py instance is started. I'm starting to experience performance issues with querying graphite, and write timeouts when clients attempt to connect to graphite.

I am running this image on a 4-CPU machine, and if I understand correctly, the best practice is to run one instance of carbon-cache.py per CPU.

Is there a way to do that?

no need to pip install whisper? or remove the steps of clone and 'python ./setup.py install' for whisper?

python dependencies

RUN pip install django==1.3
python-memcached==1.53
django-tagging==0.3.1
whisper==0.9.12\ ## this line can be removed since it's installed from source in later steps
twisted==11.1.0
txAMQP==0.6.2

How can I restart the carbon-cache service with this VM?

I've made alterations to the storage-schemas.conf and now I want to apply them, how can I do it?

Revision to docker-graphite-statsd

Hi Hopsoft,

Just letting you know that I rewrote your image in a perhaps more structured fashion.. I linked back to your repo and given you credit :).

https://github.com/CastawayLabs/graphite-statsd

Best,
Matej

Graphite \ server time and the wrong time

[root@host-10-240-44-1 ~]# docker exec -i -t 6d6b745f7133 bash
root@6d6b745f7133:/usr/local/src/carbon# date
Tue Oct 13 22:18:55 CST 2015

Unable to receive any stats.

I'm using docker-compose to build this, here's my config:

  graphite-statsd:
    image: hopsoft/graphite-statsd
    ports:
      - "5000:80"
      - "2003-2004:2003-2004"
      - "2023-2024:2023-2024"
      - "8125:8125/udp"
      - "8126:8126"

I'm able to access the graphite interface at http://localhost:5000 however, using the test command in the readme, I'm unable to send any stats to graphite.

If I run (echo "counters" | nc localhost 8126) > counters then I see my stats. The issue is that I'm unable to see any of this from within Graphite.

Is there a step I'm not seeing in the Readme to getting this up and running?

If I purposefully misconfigure the port, I get connection refused every millisecond, so I know the

Any plans to adapt to version 0.10 of graphite?

Accessing from non-localhost address fails

If I hit :80 from any IP other than localhost, things fall apart with the following sort of error messages in /var/log/nginx/access.log:

2014/07/30 21:00:06 [crit] 50#0: *197 open() "/var/lib/nginx/fastcgi/9/00/0000000009" failed (13: Permission denied) while reading upstream, client: , server: , request: "GET /content/js/ext/ext-all.js HTTP/1.1", upstream: "fastcgi://127.0.0.1:8080", host: "", referrer: "http:///composer/?"
2014/07/30 21:00:06 [crit] 50#0: *196 stat() "/opt/graphite/webapp/content/content/js/completer.js" failed (13: Permission denied), client: , server: , request: "GET /content/js/completer.js HTTP/1.1", host: "", referrer: "http:///composer/?"

Cannot map data volume for storage

First thanks for creating this docker image!

I'm trying to map my storage directory so that I can rebuild the docker container if necessary. I've tried making the following changes in the bin/start file:

-v /opt/graphite/storage:/opt/graphite/storage \

My understanding is that this would map the storage to the host directory of the same name. Nothing appears in any of the graphite logs, but accessing the website results in a lot of authentication errors.

Have you tried this, or seen anything similar?

Regards,
Roy.

ports are not exposed in dockerfile

In order to bind to a port on the host you should define which ports are exposed of the container. I previously launched the container with -p 127.0.0.1:2003:2003 but this didn't work, after I changed the dockerfile to include EXPOSE 80 2003 it was working as it should.

Pull request incoming

Graphie showing time in UTC

The container should use host's timezone.
It's default timezone is UTC.

Deploy on DigitalOcean VPS

Hey, thank you for creating such a convenient docker container!

While I got everything up and running using VirtualBox and boot2docker locally, installation on a remote DigitalOcean VPS failed for me.

What I did:

• install docker
• clone this repository
• run docker-graphite-statsd/bin/start

Let's say my VPS IP is 123.123.123.123

Visiting http://123.123.123.123/ shows the black graphite web dashboard headline, but the main content is not loaded.

I suspect it is some kind of issue with nginx or django configuration, but I can't seem to figure it out.

Contents of container's /var/log/nginx/errors.log:

2014/05/30 07:55:32 [crit] 49#0: *1 stat() "/opt/graphite/webapp/content/" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET / HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:32 [crit] 49#0: *1 stat() "/opt/graphite/webapp/content/browser/header/" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /browser/header/ HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:32 [crit] 49#0: *4 stat() "/opt/graphite/webapp/content/composer/" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /composer/? HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:33 [crit] 49#0: *1 stat() "/opt/graphite/webapp/content/content/img/carbon-fiber.png" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /content/img/carbon-fiber.png HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:33 [crit] 49#0: *1 stat() "/opt/graphite/webapp/content/content/js/ext/resources/css/ext-all.css" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /content/js/ext/resources/css/ext-all.css HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:33 [crit] 49#0: *4 stat() "/opt/graphite/webapp/content/content/js/ext/adapter/ext/ext-base.js" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /content/js/ext/adapter/ext/ext-base.js HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:33 [crit] 49#0: *1 stat() "/opt/graphite/webapp/content/content/js/ext/ext-all.js" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /content/js/ext/ext-all.js HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:33 [crit] 49#0: *10 stat() "/opt/graphite/webapp/content/content/js/browser.js" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /content/js/browser.js HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:33 [crit] 49#0: *11 stat() "/opt/graphite/webapp/content/content/js/composer_widgets.js" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /content/js/composer_widgets.js HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:33 [crit] 49#0: *13 stat() "/opt/graphite/webapp/content/content/js/composer.js" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /content/js/composer.js HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:33 [crit] 49#0: *15 stat() "/opt/graphite/webapp/content/content/js/completer.js" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /content/js/completer.js HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:55:33 [crit] 49#0: *1 open() "/var/lib/nginx/fastcgi/1/00/0000000001" failed (13: Permission denied) while reading upstream, client: 200.100.100.100, server: , request: "GET /content/js/ext/ext-all.js HTTP/1.1", upstream: "fastcgi://127.0.0.1:8080", host: "123.123.123.123"
2014/05/30 07:56:06 [crit] 49#0: *11 stat() "/opt/graphite/webapp/content/favicon.ico" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /favicon.ico HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:56:06 [crit] 49#0: *11 stat() "/opt/graphite/webapp/content/favicon.ico" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /favicon.ico HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:57:45 [crit] 49#0: *20 stat() "/opt/graphite/webapp/content/" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET / HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:57:45 [crit] 49#0: *20 stat() "/opt/graphite/webapp/content/browser/header/" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /browser/header/ HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:57:45 [crit] 49#0: *23 stat() "/opt/graphite/webapp/content/composer/" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /composer/? HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:57:45 [crit] 49#0: *23 stat() "/opt/graphite/webapp/content/content/js/ext/ext-all.js" failed (13: Permission denied), client: 200.100.100.100, server: , request: "GET /content/js/ext/ext-all.js HTTP/1.1", host: "123.123.123.123"
2014/05/30 07:57:45 [crit] 49#0: *23 open() "/var/lib/nginx/fastcgi/2/00/0000000002" failed (13: Permission denied) while reading upstream, client: 200.100.100.100, server: , request: "GET /content/js/ext/ext-all.js HTTP/1.1", upstream: "fastcgi://127.0.0.1:8080", host: "123.123.123.123"

Probably I am missing something basic here... grateful for any help!

Unable to bring up the service on the VM created by Vagrant

Host: Windows 7
I cloned the repo from github and did a 'vagrant up'. The VM started without errors.

I did a 'vagrant ssh' and executed the following.
sudo docker run -d
--name graphite
-p 80:80
-p 2003:2003
-p 8125:8125/udp
hopsoft/graphite-statsd

First time, it threw an error that stated vaguely that the port is already in use. I tried changing the port 80 to 8080, 8081 and a few others but it kept saying that the port is already in use. I could not bring up the service using docker. Am I missing something?

Fail to connect MySQL

Hi all,

I'm having a problem while trying to use MySQL for graphite.
I have changed local_settings.py like this:
...
DATABASES = {
'default': {
'NAME': 'graphite',
'ENGINE': 'django.db.backends.mysql',
'USER': 'graphite_db',
'PASSWORD': '123456',
'HOST': '192.168.99.100',
'PORT': '3306'
}
}
...
And when I try to build Dockerfile, and the below message showed:

spawn python /opt/graphite/webapp/graphite/manage.py syncdb
OperationalError: (1044, "Access denied for user 'graphite_db'@'%' to database 'graphite'")
expect: spawn id exp5 not open
while executing
"expect "Username" {
send "root\r"
}"
(file "/usr/local/bin/django_admin_init.exp" line 10)
The command '/bin/sh -c /usr/local/bin/django_admin_init.exp' returned a non-zero code: 1

P/s: MySQL server is running on docker too. I've try to connect MySQL from Docker Container via /bin/bash when docker run and I can connect to MySQL server.
Is there any problem with my config?
I'm running Docker on my Mac (Docker machine IP is: 192.168.99.100)
Thank you in advance.
Sorry for my bad english.

Can't use any other port than 80

When setting up this docker image I didn't want to use port 80 as it clashes with IIS on our windows machines, but changing docker run command to other port made it not work at all.

So my question is, how do I set up this image to work on any other port than 80?

Graphite and Statsd configurations not loaded

I have replaced conf/opt/graphite/conf/storage-schemas.conf and conf/opt/statsd/config.js with my own files. However, neither are available in the container instance after running docker build /path/to/Dockerfile with and without the --no-cache option.

This seems fishy because of these relevant commands in the Dockerfile.
ADD conf/opt/graphite/conf/*.conf /opt/graphite/conf/
ADD conf /etc/graphite-statsd/conf

Am I missing something? At the present, I'm forced to manually copy the files from the host to the container.

Only last 6 hrs shown in Grafana, not able to write graphs with Graphite dashboard.

Hello, I have two issues.

First of all I'm not able to graph anything in my graphite dashboard - with exactly the same date settings and target, data is shown in Grafana properly. I can't understand why. I can't view my graphite data with graphite dashboard, but when I connect the same graphite into Grafana - Grafana shows the data.

Second issue is that my Grafana is showing only data for last 6hrs. After picking bigger range, for example "last 24hrs" - it shows nothing. No data from last 6 hrs, not data from 24 hrs. This is weird.

Any suggestions? Thanks in advance!

TCP interface

statsd from v0.8.0 introduced the support for a TCP listener. This is useful in a number of cases, and I think it'd be good to have it in this Docker image.

For now I've forked the repository,changed the Dockerfile so it clones statsd v0.8.0 instead of v0.7.2, and updated the statsd configuration in conf/opt/statsd/config.js. Ideally it would be nice to be able to specify on which interface statsd should listen when the container is run.

Any thoughts how this could be implemented? Let me know if you feel that a PR for that would be useful.