Monitoring vhosts with Icinga 2 and Icinga Web 2

Since Icinga 2 runs stable and is considered mature (1.x still works, but is a pain to configure) I’ve taken 2015 for starting to use it on my productive systems. The server which hosts also serves several other vhosts, overall it is hosted in the NETWAYS cloud (thanks!), one situation where you love your managed service guys & Opennebula, Foreman, Puppet.

Afterall I wanted to monitor at least some web and dns services for all vhosts (more to come though). Web services are just a simple http reachability check, while dns verifies that the A record of the given vhost also returns the server’s ip address. That is mandatory to see whether the dns records are still intact, or some misconfiguration is happening.

Therefore I’ve created a file ‘hosts.conf’ containing one host so far, introducing the ‘vhosts’ dictionary.

# cat hosts.conf

object Host "srv-mfriedrich" {
  check_command = "hostalive"

  address = ""

  vars.vhosts[""] = {
  vars.vhosts[""] = {
  vars.vhosts[""] = {

The services.conf file is rather generic – it takes all hosts with the custom attribute ‘vhosts’ and loops over that dictionary, creating new service object. Each service is prefixed with either “http-” or “dns-” depending on the generated check then. Read more about apply-for-rules.

I’m using the shipped Icinga 2 plugin check commands “http” and “dns” and only set the expected custom attributes.

 # cat services.conf

apply Service "http-" for (http_vhost => config in host.vars.vhosts) {
  import "generic-service"

  check_command = "http"

  vars += config
  vars.http_vhost = http_vhost

  notes = "HTTP checks for " + http_vhost

  assign where host.vars.vhosts

apply Service "dns-" for (dns_lookup => config in host.vars.vhosts) {
  import "generic-service"

  check_command = "dns"

  vars += config
  vars.dns_lookup = dns_lookup

  notes = "DNS checks for " + dns_lookup

  assign where host.vars.vhosts

Note: Apply For was introduced in Icinga 2 v2.2.0 and is the preferred way of configuring the magic stuff.

icinga2_legendiary_01 icinga2_legendiary_02 icinga2_legendiary_03

Added some notifications and users, in that example it’s just everything for vhosts and myself. I don’t like to get notified when someone forgets to configure the ‘address’ attribute required for the checks, so the notifications are not generated for these objects.

The ‘mail-{host,service}-notification’ templates are shipped with Icinga 2 in conf.d/templates.conf, similar to ‘generic-{host,service-user}’ templates. The notification templates do reference the mail notification command, but I don’t really care about it. The only important thing is to set the User object’s ’email’ attribute.

# cat users.conf 
object User "michi" {
  import "generic-user"

  email = ""

# cat notifications.conf 

apply Notification "vhost-mail-host" to Host {
  import "mail-host-notification"

  users = [ "michi" ]

  assign where host.vars.vhosts
  ignore where !host.address //prevent wrong configuration being notified

apply Notification "vhost-mail-service" to Service {
  import "mail-service-notification"

  users = [ "michi" ]

  assign where host.vars.vhosts
  ignore where !host.address //prevent wrong configuration being notified

icinga2_legendiary_04Using the new dynamic Icinga 2 language could become rather complex. But still, for simple vhost monitoring it even saves you a lot of typing. And keep in mind – the notification rules are applied based on patterns. I don’t have to worry about contacts assign to hosts/services as I always struggled with in Icinga 1.x or Nagios.

In the end, I’m also using Icinga Web 2‘s git master. It’s still beta, but works far better than Classic UI or Web 1.x. So you’ll see, it’s time for Icinga 2* and a bright future. Next up – Graphite, Graylog2 and automated Puppet deployments of remote checker clients/satellites.

Playing with Icinga 2 and graphite

If you’ve attended the OSMC 2013 and the Icinga presentation you might have seen it already, but for all new readers – Icinga 2 got native support for writing metrics to graphite carbon-cache. There’s not much to do than

  • have Icinga 2 installed & some checks configured
  • have graphite up & running
  • enable the GraphiteWriter feature

I’m using a Vagrant box for graphite where I am running a puppet module to install graphite from sources, but patching it for realtime performance – so you might assign that a little more disk space then.

The Icinga 2 Vagrant box will install the latest and greatest snapshot rpms built from git next, so we are bleeding edge here – if you encounter any bugs, please report them to

The graphite vagrant box will listen on the forwarded port 20003 on localhost’s ip address. Feel free to modify the virtualbox portforwarding though – it’s just a different port not to harm any local installs.

Now get into the Icinga 2 Vagrant box and enable the GraphiteWriter feature.

$ vagrant ssh
$ sudo -i
# icinga2 feature enable graphite

Now uncomment host and port, and modify it to your carbon cache listener. Restart Icinga 2 to apply changes.

# vim /etc/icinga2/features-available/graphite.conf

 * The GraphiteWriter type writes check result metrics and
 * performance data to a graphite tcp socket.

library "perfdata"

object GraphiteWriter "graphite" {
  host = "",
  port = 20003

# service icinga2 restart

The Vagrant graphite box is accessible at http://localhost:8081.

Home exercise: Set “check_interval = 1s” in your services, and watch graphite in realtime (patched auto-refresh). If you need some detailed insights on graphite itself, you may checkout my employer’s trainings.

Nagios – More drama for the mama

It all starts with a bang

Well, obviously long time before I actually knew about anything like Linux or Nagios. Shortcut – First appearance 1999, trademark issues with the name ‘netsaint’ in 2002, renamed to Nagios which is some acronym for whatever (google it). Versions 1.x and 2.x, simplicity with a plugin api capable for everyone, fetched attraction by many users. We can read that on Google, history, done.

Update 2016-01-15: It is still going on. Now with the Nagios::Plugin Perl CPAN module and legal threats. More to read over here.

Update 2014-01-24 09:29 CET:It seems that the drama continues before it ended, so if you came here reading about “Michael Friedrich”, “Netways” and trademarks, as Ethan G. outlined in his comment on a rhel bug, here’s some additional vita (the rest can be found on XING and LinkedIn). 30, Austrian, 2009-2012 Admin/Developer at the University of Vienna, 2012-2013 Senior Consultant at Netways, 2013-present Application Developer at Netways. Lead Icinga Core Developer since May 2009.
If you’re wondering why last names are abbreviated – I prefer not to expose the possibly offended characters too much. It’s become a psycho drama.

Once you’ve read all through, you can happily start at the very beginning here to read again. Look for the 5th ace and let me know. I’ll happily grant you a beer or two, if you find it.

It’s a drama loop now.

Chapter I: How to (not) deal with problems & fork you

I figure that at some point, when companies have had put Nagios into their monitoring stack, selling consulting, workshops, whilst the original author just wrote code, it got problematic. Especially when people are demanding too much, and provide nothing in return. I do have that feeling sometimes with Icinga users as well, but that makes us human people to tell everyone about it, figure out the input both sides may provide, and reject a feature request, if it does not work out. Still, to remark – listen to the community, and answer.

Start your own business, company, etc is completely legit. The problem arising – community requires love & feedback. If there is none, you really should consider giving up your one-man-show and open up for more developers. While Ethan G. claims that Icinga is “my fork”, it apparently is not. We generally do not own projects. We instead appreciate being part of it. I joined the team one month after they’ve forked from Nagios, looking for a way to integrate Oracle into the *DOUtils backend. The dead nagios-devel mailinglists had made it clear already – a fresh fork with motivated people would allow me to participate and bring my patches upstream.

Basically that fork just happened when everyone’s local fork (Nagios with patches) was growing too big and unmaintainable. It’s still reasonable to do so, also because Ethan G. also suggested forking to users in the past being annoyed by the not-very-reponsive leadership ten years ago.

Chapter II: How to (not) deal with problems & be a drama queen

In Europe, especially Germany, it’s quite common that trademark trolls register trademarks for existing open source projects, and forcing them to rename their projects, and lose their corporate identity and name within the community but also the search engine’s index. I do believe it when Julian says (and told me in person) that he decided to register (and buy) the Nagios trademark in Germany to prevent that from happening. And it all worked out quite well.

  • NagVis was born in 2004, designed by German developers.
  • In 2006, PNP4Nagios was created. The most famous Perfdata Graphing Addon based on RRDTool. Hosted in Germany.
  • The Monitoring Portal was created in late 2003, operated by independent Germans. This is were the community meets. Statistic from 23.1.2014: “12,226 Members – 29,644 Threads – 197,306 Posts (average 51.04 Posts per day)”
  • Monitoringexchange (former Nagiosexchange) operated by Netways offers 2000+ plugins and addons

Trademarks are not free. They need to be paid. So Ethan should’ve been grateful not to pay those fees in Germany afterall, and getting supported by companies caring about the community, letting them work on their spare time projects, and not meaning any harm.

Instead, the bitching about trademarks started. First off, it was about (now, then related to the famous Nagios conference (now Open Source Monitoring Conference) and other brands which were demanded to be handed over to Ethan himself. It ended all with misleading information, a smear campain on twitter.

Announcements by Nagios Enterprises: I II

Statement by Julian

At that time Icinga was already forked, and gained attraction by the existing Nagios community. Furthermore, Icinga is supported by Netways with server hosting and development manpower. Which obviously played a role in this fight, but it’s not that clear.

The outcome for the Nagios community in the end was

  • People started to know Icinga
  • Nagios Business Process addon lost its domain, and it took a while to let search engines adopt
  • the Nagios Conference was called Open Source Monitoring Conference and attracts more participants looking at Nagios & forks, OpenNSM, Cacti, Zabbix, etc
  • Confusion between and
  • Bookmark update fun

In the end, we also had the Nagios drama mailinglists announced.

Chapter III: How to (not) deal with problems & act like a child

At some point, Jean Gabes proposed his rewrite of Nagios in Python to the Nagios Developer’s list. While it was to be expected that Nagios developers wouldn’t just switch from C to Python, the reactions after making Shinken a “Nagios compatible rewrite” were a bit strange and childish. Especially in terms of approaching other people. Or attacking even.

Only quoting a passage here:

blah, blah, blah,
You heard about the drama on the mailing list but yet you are still busy creating more. If you truly believed that flaming is nonsense then why do you initiate it? It is very humble of you to request the flaming to stop after you have had your inaccurate dramatic statements posted. I suppose you thought your statements would not be challenged because we usually do not respond to ridiculous rantings of a seemingly angry person who jumps to inaccurate conclusions and makes rash assumptions.
Your reason for forking lacks substance. Nagios is an engine. A very powerful engine that has gained in popularity because Ethan Galstad has been a very successful programmer and gatekeeper of the project. If you are not happy with the progress of the project, so what! You are just one person . There are millions of Nagios users that are very happy with the Nagios engine. We do not feel the need to cater to one or a few angry people that are jealous of Ethans success. Your lack of respect for Ethan is appalling, knowing that Ethan spent 10 years of his life developing Nagios and cultivating the community.
You “forkers” try to justify your actions by saying it is for the good of the community. The rest of us know the real reason is to satisfy your own egos (really twisted). You must feel unethical about what you are doing or you would not have tried to blame us for your actions. IMO, you forked because you and Gerhard lack the skills and creativity to start your own project. You would rather “steal” someone else’s work. Further evidence of your lack of creativity is posted on your own website where even your tag line is stolen from Nagios.
It is time to stop the negative, inaccurate, flaming drama directed at Ethan and Nagios. Do your own hard work and stop attacking Ethan and the work that he has done for the last 10 years. Please share my response with your partner in crime, Gerhard Loser.

That doesn’t sound very constructive, and contains plently of personal attacks. Gerhard’s last name actually is spelled “Laußer”.

Chapter IV: How to (not) deal with problems & compare yourself

At some point, your customers ask you about your competitors. Or community members even. How do you compare to XY? We had that with Icinga a lot, and therefore created our version of it. While it may make Icinga look good, we’ve learnt one thing: The previous version compared Icinga with Nagios Core and Nagios XI (the commercial entity). While Icinga is free in every single line of code, Nagios XI isn’t. We decided to skip that part, and removed that column from the Icinga comparison chart.

After a while, we came across this interesting OpenNMS article talking about Nagios FUD. And we figured, that Nagios Enterprises had created a comparison chart for Nagios vs Icinga themselves. But actually that one lacks of any substance, comparing Icinga Core with Nagios XI (the commercial one), but only flagging it Nagios. At that time it was unclear that “Nagios” means “Nagios Commercial Version”, while “Nagios Core” follows the free open-core model.

Either way, the pdf also contains false information about IP violation done by Icinga and Netways. Both never have violated any intellectual property, but probably that’s the only arguments to tell managers and buyers that Icinga is a risk, while apparently it’s just pure hate against the fork, spreading FUD themselves all over the internet. How_Nagios_Compares_To_Icinga of that pdf, copyright both pdf and the quote below owned by NE.

About Icinga
Icinga is a young project designed as incompatible Nagios fork by German company Netways GmbH. Icinga suffers from high developer turnover and poses legal risks to organizations that deploy it.

Use Caution
Both Icinga and Netways GmbH have a history of intentionally violating international intellectual property laws. You may be putting yourself and your organization at risk if you deploy or implement Icinga in your organization.
Knowingly using or working with products or companies that violate intellectual property laws and treaties can have significant legal and financial repercussions.

After reading this bullshit, and not liking the Matrix either, the developer in me decided to invest several days to create a comparison from a developer’s point of view. Since many developers don’t necessarily mean bad project quality. And quite frankly, being the release manager since 0.8.3 4,5 years ago, it’s still annoying to see that they advertise their lies publicly.

Btw – there are other comparisons with Centreon, Groundwork, Opsview and Zabbix.

Chapter V: How to (not) deal with problems & censor the internet

Wikipedia is an open-minded community, enforcing free speech in their articles. The english article about Nagios contains references to its forks. Some people obviously influenced or working for Nagios Enterprises (whois said so) have had a different opinion on that, and tried to censor that Nagios article by removing all unwanted strings and urls. Or, they’ve added subjective meanings into it, like “Nagios XI is the best monitoring tool in the world”. That did not end well for them after several attempts of fraud on that wikipedia article. It got even worse – Wikipedia banned Nagios Enterprises being sockpuppets.

And – trademarks join the drama queens the first time. The French Nagios community was formed by Olivier J. at former also talking about Icinga being a Nagios Fork. It seems that Ethan G. and Mary S. (wife) didn’t like that much, and attacked Olivier in a way, well, no words for that. But go figure, transferring the domain and changing the community portal to was the only wise choice. And it’s still a good community portal for everyone, not only Nagios. And you may not censor their articles, but read them – Chrome offers auto-translate anyways.

Chapter VI: How to (not) deal with problems & treat community members like shit

In early 2012 there was an interview done by with Ethan G. and Michael L. – both sides added their arguments, comments, etc. While the article was great, I felt that it required some more additional content (some of it is found in this article as well now).

A short time after that article, Michael L., the former owner of got a letter from ICANN that Nagios Enterprises has claimed the trademark on the domain, and that the “Uniform Domain Name Dispute Resolution Policy” now applies. .org TLD disputes are directly handled by ICANN, while cTLD disputes for example for .de are handled by the local domain registry “DENIC” (and not even that, but this is handled under civil law in Germany). That ICANN procedure is pretty annoying and for non-native English speaking people a mess without lawyers.

At that time, it was already known thanks to the “Julian incident” that trademarks are now handled differently by Nagios. So the domain “” had been prepared long time ago, the primary record was changed, and Google did get a hit. The final announcement left hope that the community would go one like before. And we weren’t mistaken – it’s even more international after all those years.

Sending a patch to Nagios to in the name of the Icinga Development Team resulted in “Changelog clarifications” by Ethan G., and after claiming the copyright on that patch, on a ban on their tracker. Meanwhile on the nagios-* mailinglists the string “*” was banned (my previous employer) – i know that because I couldn’t write to nagios-plugdevel either. Holger unblocked me when we were debugging the root cause.

Chapter VII: How to (not) deal with problems & work with a zombie community

Have a Nagios core developer maintain the patch queue & redesign the Nagios Core 4 – awesome. Kicking that developer out of the dev team just because he didn’t test the cgis properly – priceless. So yeah, 2013 was when it all went shits with Nagios. Fun fact: Andreas chose the Open Source Monitoring Conference (the one which had to be renamed due to trademark claims by NE) to literally hijack the “Nagios future” presentation, telling the audience that he got removed, and forked Nagios 4 into Naemon. Well, he actually wrote 99% of it, so no big deal.

So it really feels like that the (former) Nagios community now meets at conferences, presenting new features and having lots of beers with community members. And it doesn’t matter if it’s called Nagios, Icinga, Shinken, Naemon or anything else. At OSMC 2013, we had a great time with Zabbix & OpenNMS developers too, it felt like one big happy family. Not celebrating that Nagios got forked again – only having a great time together, and leaving each other with visions & ideas.

So it’s not about Nagios anymore, and the community changes as well. is bloated with do-it-yourself, student, pro and commercial editions so people won’t easily find the download urls anymore. Everything is about business and selling a product. And the difference between Nagios and Nagios XI is not clear – rather close the webpage and ask for alternatives these days.

Content in that picture is copyright by NE.

The final chapter

Many people still use Nagios. They are annoyed by bugs, or broken releases. Missing features like real distributed monitoring, or stable apis are bad, but there are other tools on the market solving those. You may still make Nagios write to graphite somehow. But at some point, #monitoringsucks will apply.

In any way, installing a Nagios/Icinga/Shinken/Naemon/Centreon/Opsview Core will require plugins. Small nifty executables, be it compiled source binaries, or just perl/python/etc scripts returning some output and an exit code. If not, your first look onto the webinterface won’t be green. Green means “Everything is OK, don’t panic”. As a matter of fact, every single admin gets nervous if there’s something not green. Especially after the first install.

That being said – without the enormous effort of the Monitoring Plugins Development Team (former Nagios Plugins Development Team) all those core check engines would be nothing. Not even a single customer would buy your support without plugins making this work out of the box. We (in terms of the Icinga team) include the Fedora EPEL repositories in our Vagrant demo boxes for Icinga 2 pulling the nagios-plugins package.

Last week, wednesday 15.1.2014 at 22:38 to be exact, the following dialog happened on IRC (#nagios-devel

22:24:40 < dnsmichi> is that the iframe forwarder thing?
22:27:37 < emias> dnsmichi:
22:28:11 < emias> dnsmichi: And if you query that web server with "Host:", you'll get a modified version of our home page.
22:28:25 < dnsmichi> modified?
22:28:34 < emias> dnsmichi: I'm pretty sure yhey're taking over the domain.
22:28:42 < emias> "they're"
22:29:11 < dnsmichi> i guess. they are trying everything to hide projects talking about nagios forks.
22:29:57 < emias> dnsmichi: Yes. Ethan had sent an email a while back, seems ge wasn't happy with my response.
22:30:00 < emias> s/ge/he/
22:31:59 < emias> Soooo!  Are we totally sure?  Then I guess we should put up and post to the list as soon as possible.
22:32:35 < dnsmichi> No. But 2014 is a good year to change things
22:32:41 < emias>
22:32:54  * emias wanted to go to bed early today ...
22:33:02 < dnsmichi> Like leaving sourceforge
22:33:47 < emias> Well that's done.
22:34:25 < emias> As we moved everything to our own server (+ GitHub), switching to would be very easy now technically.
22:35:46 < emias> http://xxxxxx/archive/np-new.html
22:36:00 < emias> That's the content I currently get from their server.
22:36:33 < emias> I think I'm sure enough.
22:38:10 < dnsmichi> ehm
22:38:24 < dnsmichi> i just figured
22:38:35 < dnsmichi> that webserver actually serves
22:38:49 < dnsmichi> i miss icinga, neamon, shinken on the main website
22:39:11 < emias> dnsmichi: That's what I'm talking about.
22:39:40 < dnsmichi> yeah, but not only via telnet Host trickery, but live in my browser
22:40:10 < dnsmichi> can you edit the main page and look if your changes are applied?
22:40:37 < emias> heh
22:41:36 < dnsmichi> i think you have been hijacked.

So basically, at that time, the websites were identical, after the Nagios Plugins Development Team was required to rename their project. I may be biased and have reacted emotional in that situation, but my german speaking brain told me, that copying a website, and changing the dns records would mean: Put your own (censored) content into the user’s sight. He won’t notice. That’s just like getting a mail asking for your bank account details, showing you a website which _looks_ identical, but isn’t technically. So if would allow to change your passwords, they would simply have fetched and stored your passwords being able to use that against you.

On Sunday, 19.1.2014 it looked like this


In that specific region, the problem is even more specific: is offering a software download. A software which is installed on thousands of servers. By serving an identical software package, everyone will trust it and just upgrade/install it, as the documentation of many projects say. The user won’t recognize any changes. What if they decide to add a phone home functionality to their software? You’ll never know, because you are trusting the name. You are trusting a vendor, which had been replaced. That obviously means “compromising” or “hijacking” a website.

The legal part of copying a copyrighted content just came up later in the drama (being totally illegal to act like that). But still, NE at least recognized that, even if they didn’t put it into a public statement. now looks a bit different than the original content now served by Even though, their actions are still illegal. But if no-one stops them, they will continue like before.

Either way, there was an announcement the day it happened on the website, and later to the mailinglists, where a discussion started. Also some insights on the previously happened Naemon fork (look through the archive, if interested).

On thursday (16.1.2014) the discussion on IRC was still ongoing. At that time, I’ve taken the liberty to step forward, and inform the package maintainers of ‘nagios-plugins’ while the now kicked and renamed team started to continue their renaming process. Jan W. is also the Debian packager, so I just informed RHEL, SUSE, FreeBSD, OpenBSD, Gentoo and ArchLinux. These distributions are well aware of Icinga, and I do have some contacts over there. Or the other way around – me being a developer changing stuff wants to inform packagers making their lives easier. I’ve been maintaining the Icinga RPMs for 4 years now, so I know what I’m talking about. Upstream fucks up, you’re screwed with mass reworking your packages.

In any attempt made, it was about users and their changed upstream source infecting their chain of trust.

Somewhere between thursday and friday (17.1.2014) the story hit reddit (it wasn’t me). There were plenty of comments in 2 threads (linux and sysadmin). While ‘scumbag’ isn’t a nice description, reddit still allows you to name it like so. And apparently, given the fact that NE did not tell anyone about their website defacement, ‘scumbag’ the is very least one could say (not that I would, just saying, eh?).

Interestingly enough, that story was to be read on Slashdot too. And then a bit more widespread on twitter as well.

In parallel to all the news, and also overwhelming feedback & appreciation of the monitoring plugins (and also Icinga which normally benefits by any Nagios drama), there was the discussion on-going on the bugs I’ve opened for the packagers. While I was and am not essentially interested in helping NE to gain their ‘nagios-core-plugins’ package, it’s still vital to know that the Redhat bug unleashed a competitive discussion how to proceed here. While NE employee Andy B. always insisted that the Monitoring Plugins project is a fork of the original (and fixed the git repo with a single commit after my remark of trust – that same passage is also quoted in that interesting blog article referencing my community concerns), the general outcome was that no-one is able to use/update the ‘nagios-plugins’ package in EPEL. Which is a good deal imho so far.

The thing with a newly formed Nagios Plugins team is – they don’t know the code very well, even if claimed otherwise. Not even Eric S. who tends to be the only part-time core/ndo/nrpe/nsca developer of NE, may handle all the stuff. From my personal observation following their git history closely he broke more stuff than Andreas ever did (getting paid for breaking Nagios? Haha, Just kidding.).

On monday, 20.1.2014 Ethan G. joins the drama for even more drama. NE has announced the new Nagios Plugins team, while attacking Holger W. using his full name in their company blog. What a shame – no more words to add here.

Content copyright by NE ofc.

In any attempt, now looks different (Wikipedia tells about the controversy too). They’ve also changed their opinion on mailinglists (recently shut down to prevent Andreas E. announcing his Nagios fork, as rumors like Andreas E. do tell) and have now auto-subscribed all users already subscribed to monitoring-plugins list to the nagios-plugins list. Copying the subscribers database and using that personal data against the unwanted project? Well, nice try. Users are confused anyways, and will happily unsubscribe from the two mailinglists looking for alternatives to follow the news or look for support.

Anyways. Too much drama here involved. FreeBSD, OpenBSD, SUSE, Debian have reacted. Some of them will just move from ‘nagios-plugins’ to ‘monitoring-plugins’ and oprhan the old package. For RHEL, Sam K. is working on a new package (I’ve only done some additional package foo in order to help resolve all the remaining showstoppers) becoming the new ‘monitoring-plugins’ maintainer in EPEL 7. For EPEL5/6 there may not be an obvious solution, but we’ll make sure to get something working for our communities.

Even if a Nagios user installs packages from being happy about fixed problems. Is that a crime? No it isn’t. But NE thinks it is. But it doesn’t matter in the end. We (the forks) are the bad guys, while NE and Ethan G. are the ones who never started anything, and are being harmed by us. Tbh I consider this a personal attack, but hey – that’s a RHEL bug to fix problems. Not to generate more drama for the mama, eh?

An now for the closing end, some more dramatics dramas. Well, first off an interesting blog entry by Matt Simmons, which showed that my English is actually bad in terms of vocabulary. Using the term “compromised” actually had put the RHEL bug up to hacker & security news feeds. Well, public attention for what price. Still NE is probably using that argument against me, but who cares anyways. That really was a language accident or incident, call it whatever you like 😉 Oh, and I’ve made sure that the internet doesn’t forget even if this story goes offline.

There are probably some more addons or plugins around which are compatible with Icinga, or mention that on their websites. They’ll probably get an email soon-ish and being ordered for censorship serving the greater power of the NE dictatorship. And then they’ll lose their motivation. Or step down, because renaming all the stuff causes that much work, moving the domain, re-establishing a trust platform. Or they are forked by NE into their enterprise stack, like happened with Nagiosql (now Core Config Manager), NagTrap (now NSTI) or Teeny Nagios (now Nagios Mobile). But you as addon developer get all the support questions, that’s for sure.

Nagios Enterprises has shown in many ways how to kill their own community being grown around Nagios. Conferences have started popping up where people meet not to only talk about Nagios (the Nagios World Conference Italy renamed itself to Open Source Systems Management Confernce while the Open Source Monitoring Conference attracts >250 open source enthusiasts every year here in Nuremberg). Community platforms which have resisted against their censorship and renamed themselves (the real community lives at Addons which are looking for alternative hosting platforms. Developers who just do not like them for their contribution agreements (and other licenses which explicitely forbid you to fork). Everyone chooses their platforms where they’re free. Like we are on #icinga on freenode. Or social media where censorship just doesn’t work. Not even fork developers will send them patches anymore being either banned or ignored.

In any way, the ordinary user will not create any plugin or addon in the future. There are too many ways to fail, and if you’re open-minded you’ll get a letter from a lawyer forbidding you to talk about competitors with your Nagios-exclusive tool. A zombie community – and then it will be interesting to see how the open core business model of Nagios will be able to survive, if the community stops contributing. Or if the community starts over in Icinga/Shinken/Naemon, or any other cool devop tool such as Sensu.

But, that’s another drama for the future. In order to be prepared, get this t-shirt. Then meet with all Nagios forkers, create cool code & laugh with community members, and enjoy your beers.

And leave the drama to those having abandoned their old friends.

Icinga 2 release steps for 0.x.y

$ cd icinga2-release/
$ git checkout next
$ git checkout master
$ git fetch
$ git merge origin/next
$ git push
$ cd ..
$ rm -rf release/ ; mkdir release
$ cd release/
$ cmake ../icinga2-release -DCPACK_SOURCE_GENERATOR=TGZ -DCPACK_SOURCE_PACKAGE_FILE_NAME=icinga2-0.0.6
$ make package_source
$ tar ztf icinga2-0.0.6.tar.gz | less
$ tar zfx icinga2-0.0.6.tar.gz -C ../
$ cd ../icinga2-0.0.6/
$ icinga2_normal
$ sudo /usr/sbin/icinga2 --help
$ sudo /usr/sbin/icinga2 --version

the inode problem

Remember the problem where a daemon writes files and rotates them waiting for another daemon/cron to process and remove those files? Well, yeah, icinga core did write perfdata, but npcd did not run in order to populate the pnp rrds. Actually you will recognize that by simply using check_procs on that process, or, check_disk for free space available. The problem is here that the disk space is not an issue – it’s the huge amount of files causing the inode number to be filled up (used 3000000 inodes on my system).

$ df -i
Filesystem                                             Type     Inodes IUsed IFree IUse% Mounted on
rootfs                                                 rootfs     3.0M  3.0M   20K  100% /

While it does make sense to create a check within icinga itsself…

$ /usr/lib/nagios/plugins/check_disk -w 20% -c 10% -p /

… what if something else runs amok and your system becomes unavailable? There’s a nifty script here to be run manually or via cron, reporting the used inode numbers sorted by the most huge number/directory.

find . -xdev -type f | cut -d "/" -f 2 | sort | uniq -c | sort -n

mkfifo, group write perms and umask

Ever wondered why mkfifo accepts group write permissions, but file gets created with 0644? umask is the root of all evil. Once reset, don’t forget to save the old mask and restore it later on. Details on the issue in Icinga2 #4444

         * process would override group write permissions
         * so reset them. man 3 mkfifo: (mode & ~umask)
        mode_t oldMask = umask(S_IWOTH);

        if (!fifo_ok && mkfifo(commandPath.CStr(), S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP) < 0) {
                    << boost::errinfo_api_function("mkfifo")
                    << boost::errinfo_errno(errno)
                    << boost::errinfo_file_name(commandPath));

        /* restore old umask */