While at first sight we’ve only included the monitoring plugins and their CheckCommand definitions, we’ve been working closely with community contributors and team members to make it easier for everyone to contribute their own command definitions.
There are certain advantanges sending your definitions upstream:
- everyone can use and update/fix them
- one single place for configs and documentation
- developers may suggest updates and help with best practices
- you don’t need to care about copying the command definitions to all your instances (because the icinga2 package already provides them)
There is a detailed wiki howto available. Looking forward to your contributions!
It has been nearly 3 years when I attended the 6th TF-NOC meeting in Dublin, presenting Icinga to all NREN participants (research and education networking organisations). At that time I was working at ACOnet at the University of Vienna which slightly changed into NETWAYS and Nuremberg later that year. After-all this task force is community/developer friendly so I was allowed to follow their actions, discussions and upcoming meetings. At last years OSMC Ernst Heiri from SWITCH was asking if I could spare some time to come present the latest Icinga 2 development in Stuttgart this year.
Well, time goes by, and then there was a discussion on open source log collection & monitoring tools going on, where I’ve stepped into remarking the things we’ve talked about in the Icinga 2 & Graylog talk at OSMC 2014, and much more. Considering that I also know a little Logstash from my Icinga 2 training sessions, attending the 12th TF-NOC meeting for an in-depth Icinga 2 presentation and also sharing my ideas on log monitoring in open panels was pure fun.
There are changes in terms of naming going on – Terena & Dante joined forces under the Géant Association, while the Task Force (TF) is being migrated to a Special Interest Group (SIG). Therefore this meeting was in the end called “SIG NOC meeting“. Apart from the confusing stuff (my presentation slides show the wrong logo ;)) I’ve got the opportunity to talk about Icinga 2 nearly 2 hours.
The presentation catches up with plenty of things you might already know but is rather long-ish with ~80 slides. It also required me to do some heavy re-factoring of the Icinga Vagrant boxes used for the demo (no more custom VM as seen 3 years ago!). The feedback on the talk was awesome, and also shed some light into NREN network operators (NOC teams) and their problems. Multi-path dependencies, network topology scan & simulation or even Shibboleth as auth provider for Icinga Web 2 were those to mention, next to discussions on existing issues or just some real-life hints.
During the two days one thing was clear – integration of tools in your existing stack is more important than ever. While changes might happen slowly, one should consider evaluating how the tool does fit into the environment, including interfaces and apis for data exchange (take for instance, Icinga 2 + Puppet + Logstash + Graphite). On another note: How to deal with alerts and connect that to the ticket systems – where the RT integration in Icinga Web 2 proofed itself as a pretty good idea Or – why would one need Rackview, HP Openview and Icinga, how about only using one tool (Icinga)?
The meeting was pretty much Splunk dominated in terms of logs and their presentation was more of a sales one – they didn’t see my Icinga 2 presentation before which emphasizes on the integration part. Logstash/ELK and Graylog are far beyond Splunk in my humble opinion – at least from what I’ve seen during the meeting. Adding my own knowledge on several aspects on combining log monitoring with other tools (even the fancy sflow-to-elasticsearch project) from my managed service colleagues at NETWAYS worked out pretty well. I am confident that the participants will consider a closer look deeply into Icinga 2 as well as ELK / Graylog in the future.
It really was a pleasure being there, I am looking forward to future meetings – and even more NRENs already using Icinga 2
While you may have seen a lot of updates in our 2.x development head, Icinga 1.x is still alive and being patched and bug-fixed. Some smaller features have also been incorporated into 1.13.0 so consider upgrading your existing installation.
- Remove deprecated event_profiling_enabled from icinga.cfg
- Remove deprecated broker_module from icinga.cfg (use module object configuration instead)
- Add module config examples in modules/ directory (livestatus, mod_gearman, pnp4nagios, flapjack)
- Move contrib/downtimes to tools/downtimes and add ‘make install-downtimes’
- Feature #1867: Recurring Downtimes
- Feature #6353: deprecate icinga.cfg:broker_module; add more module examples
- Feature #8007: Implement an option to disable transactions
- Feature #8139: Add functions for registering file descriptors closed on fork()
- Feature #8140: Add Check Result List Mutex for NEB modules
- Feature #8426: Remove constraint from *dependencies tables
- Feature #8440: Enhance idomod logging
- Bug #6263: Race condition in init.d scripts’ stop
- Bug #6762: Icinga crashes when “args” attribute is not specified for modules
- Bug #7004: GET form param has no effect on cmd.cgi acks (again)
- Bug #8202: Cool tip text for refresh of hosts and services says “I’m so lonely up here. Where should I go?”
- Bug #8441: require the ‘config_file’ argument in idomod modules configuration
- Bug #8445: cmd.cgi use_ack_end_time param does not enable tickbox in form
Download icinga-1.13.0 here.
- Ewoud Kohl van Wijngaarden found a way for an SQL injection in Icinga Web’s API. An authenticated user could inject SQL code via a crafted JSON filter (#7924, CVE-2015-2685)
We recommend to update your installation to 1.13.0 as the features are minimal invasive.
Notable changes and features
- The log now contains the ip address of a user login failed, or the user just logged in and out (#7357)
- We implemented a command log that contains any command that is send to the Icinga core by an user – written to a separate log file command-20XX-XX-XX.log (#7893)
- (Bug) Acknowledgments where sent without a proper sticky declaration. This problem has been fixed and host or service acknowledgments are now sticky by default – what it should and was intended to be. (#5838 #7003) Please review our documentation if you are not sure what sticky means.
- Grids can now display customvariables. Because customvariables are customised on every installation, this feature is disabled by default. See doc/grids_and_customvars.md for further information.
- When using Kerberos authentication in a web server a user could receive all credentials when he had a role that had no credentials set (#7892) In our tests that only happens with Kerberos users.
- When a user could not be imported during login the database exception was not generated correctly (#8301)
- Don’t contact more authentication providers than necessary during login. Thanks to Victor Hahn (#8341)
- Fixed the irritating error during application state reset (#8523) The state was always cleared, but an error popped up for the user.
Download icinga-web-1.13.0 here.
Starting early on Sunday morning meeting in the NETWAYS office in Nuremberg, the first shift moved to Hannover reaching out for CeBit 2015 preparations. The Open Source Park is located in hall 6, and we were excited to have our booth directly attached to the Open Source Forum where all the cool presentations took place as well.
Monday morning should’ve started quiet and calm, but it wasn’t – users and visitors stopping by and looking at the fancy Icinga Web 2 demo. Listening to our stories on how Icinga 2‘s apply-services-based-on-patterns works and all the benefits with and around it. The first day already was overwhelming and so did it go on the second day on Tuesday. So much #monitoringlove at CeBit is really incredible, and we certainly enjoyed showing everything in detail
We also had the pleasure giving two radio interviews – make sure to join the Open Source Magazin CeBit special on Monday, 23.3.2015 and the Midnight Gaming CeBit Podcast (German only).
Bernd joined later in the evening, having prepared his talk in the Open Source Forum on Wednesday, Enterprise Open Source Monitoring with Icinga 2. Never seen so many people attending a talk in the Open Source Forum and the presentation really was a blast. After the talk the Icinga booth was a bit crowded The second shift with Eric, Johannes and co-workers from NETWAYS already joined us and so Marius and myself where leaving Wednesday, tired but happy
While our feedback was overwhelming, it did not stop yet. People coming over and saying “Thanks for Icinga, we love it” is the best feedback there is! And rumors do tell the party on Wednesday evening was great too, and Thursday was a bit heavy. But still, following the Icinga social media streams the appreciation is massive and really keeps us going forward!
See you next year, hopefully
Follow Icinga on Twitter, Facebook & Google+ – some impressions below.
Thanks for your kind feedback on the initial v2.3.0 release – we’ve found a few bug reports and already implemented fixes. There’s a regression with the disk plugin check command in 2.3.0 – warning and critical thresholds now require the percent sign explicitly. The documentation has been fixed – make sure to update your configuration accordingly.
Cluster heartbeat does not disconnect clients on slow bandwidth, process groups are killed on timeouts instead of only the parent process and the String type provides a new method called ‘contains‘.
Update: There was a problem with the fix in #8687 which is why we’re now at v2.3.2.
Upgrade now and enjoy Icinga 2
Please note that this version fixes the default thresholds for the disk check which were inadvertently broken in 2.3.0; if you’re using percent-based custom thresholds you will need to add the ‘%’ sign to your custom attributes.
- Feature 8659: Implement String#contains
- Bug 8540: Kill signal sent only to check process, not whole process group
- Bug 8657: Missing program name in ‘icinga2 –version’
- Bug 8658: Fix check_disk thresholds: make sure partitions are the last arguments
- Bug 8672: Api heartbeat message response time problem
- Bug 8673: Fix check_disk default thresholds and document the change of unit
- Bug 8679: Config validation fail because of unexpected new-line
- Bug 8680: Update documentation for DB IDO HA Run-Once
- Bug 8683: Make sure that the /var/log/icinga2/crash directory exists
- Bug 8684: Fix formatting for the GDB stacktrace
- Bug 8687: Crash in Dependency::Stop
- Bug 8691: Debian packages do not create /var/log/icinga2/crash