icinga2_graphite_web_treeThe Graphite feature in Icinga 2 is pretty cool and simple – just enable the feature and point it to your Graphite Carbon Cache listening address. Graphite 0.9.14 was released just a few days ago :)

We’ve received lots of nice feedback on this but also several feature requests which involve breaking changes. Be it the default escaping of metrics (the dash becoming an underscore), turning off additional meta data or a different tree schema.

While working on a Graphite module for Icinga Web 2 we’ve evaluated all these changes and came to the conclusion that we need to change the Graphite tree and layout.



Prefix for hosts:


Prefix for services:


Metrics are written as follows underneath the perfdata level.


With enable_send_thresholds = true (default is false) Icinga 2 will add the following threshold values.


With enable_send_metadata = true (default is false) Icinga 2 will add


Sending thresholds and meta data must be enabled as this additional data caused problems in the past in large scale environments.



Time-series databases generally store metrics but no additional meta data as for example PNP4Nagios does to select specific templates for graph representation. PNP uses additional XML files generated on each update. We’ve come up with a solution for that by changing the host and service prefix and adding the CheckCommand for proper template selection by user interfaces.

Furthermore, all services are located underneath “services” on the host to allow easier selection from the applications pulling the data from the Graphite Web API.


Metric escaping

The following characters are escaped with an underscore in prefix labels: whitespace, dots, /, \. Performance data labels won’t escape the dot allowing a more selective representation of multiple metric levels returned by the plugin. If your host or service name contains dots, they still will be escaped in order to forbid multiple tree levels and ensure that external applications may properly select the objects.



There is no direct migration path although you can still use the old schema. To prevent unwanted data corruption the new schema is located underneath “icinga2” while the old schema is using “icinga”. In order to restore the old legacy schema, you’ll need to adopt the GraphiteWriter configuration:

object GraphiteWriter "graphite" {

  enable_legacy_mode = true

  host_name_template = "icinga.$host.name$"
  service_name_template = "icinga.$host.name$.$service.name$"

Note: The legacy mode will be removed in future feature releases.


Test-drive Icinga 2 and Graphite

The easiest way is to use 2 Docker Containers: One for Graphite and icinga/icinga2. I’ve found docker-graphite-statsd which works like a charm also for Icinga 2 development tests.

docker run -d --name graphite --restart=always -p 9090:80 -p 2003:2003 hopsoft/graphite-statsd

The most recent icinga2 Docker container build provides additional options to enable and configure the Graphite feature by passing these environment variables:


These two containers must be linked together in order to let Icinga 2 write to port 2003 and Graphite read from port 2003 in two different containers. graphite is the name of the previously started Graphite container.
The following example uses the Docker IP address assigned on OSX ( Adjust this for your needs where the Graphite Container is actually listening on.

docker run -d -ti --name icinga2 -p 3080:80 --link graphite:graphite -e ICINGA2_FEATURE_GRAPHITE=1 -e ICINGA2_FEATURE_GRAPHITE_HOST="" -e ICINGA2_FEATURE_GRAPHITE_PORT=2003 icinga/icinga2

Navigate to (adjust for your container) and select “icinga2” from the tree navigation. You can watch the screencast over here :)