2016 … what’s next?

Lots of things happened in 2016. Probably way too much as I cannot really recall what I did in which month.

It hasn’t been easy all the time, and being under pressure I also made certain mistakes or acted differently than I demand it from myself.

I’m not really a fan of “1.1.2017 – leave all things behind and start over”. Reality is different and so I am taking a moment to breathe and reflect what really happened in 2016. Things I want to improve, things I loved but also things which went not so good.


Things which I would do different

At some point in late 2016 I realised that I am doing a) core development fixing bugs b) first level support c) community support. This isn’t essentially bad, but b) should be done by co-workers and c) requires a lot of effort to train community members and share knowledge.

Together with a), the actual majority of my daily work, this literally exploded at some point. I’m not proud of the way how it went (story for real life conversation) but I couldn’t help myself in this situation. In the end changes and workflows have been adopted and I am looking forward to see progress here.

Dealing with important bugs, not knowing which one comes first isn’t easy. If the support channels rely on your answer to problems, this has a huge impact on your work load. You can’t focus on the important problems anymore but need to take the chance to analyse the problems by yourself. You loose track of the important bug you’ve been looking into before getting disturbed. Finally you’ll end up with lots of ping-pong and accusations who did it wrong. Many things not even so-called project managers could solve, we’re not big enough for that even.

When your issue dashboard says 400, like 30 urgent and a lot of possible duplicates .. well. I helped myself with getting pen and paper, writing the problems down and pinning them on the wall.

There are still some issues, even many, but one also needs a better voting mechanism than Redmine provides. I am eagerly waiting for the tags and +1 capabilities we’ll be using on GitHub.

After all I consider some situations where my mind said “fix it now” being plain wrong. I should’ve waited and not rushed for solutions. There’s no guarantee for a fix anyhow. Still a problem with freeing your mind in the evening. That’ll be a deal breaker when you’re part of an open source project for many years. You sometimes just cannot stop, or the burnout stops you.

One thing I also learned the hard way – if you help others by accidentally doing their job, they’ll always rely on you. Help them in a way that they learn from you, but are able to take notes and probably never ask again.


The good stuff

Dashing turned from a fun project into an official Icinga 2 integration. I learned a lot about Ruby and Coffeescript and received overwhelming community feedback. I even wrote some development guides to help others.

The Icinga Vagrant Boxes are still on the run, and make my life even more easy. Lately I kept working on an Icinga 1x release but didn’t want to locally install the dev environment. Or by having a FreeBSD test box.

A while back I have forked the Manubulon SNMP plugins because I needed them for my own projects. It gained more attraction and I also added community contributed patches. OpenBSD is already using it, and Debian is probably going to replace the old project as well.Debian is using them already . In order to help everyone, I decided to tag and release v2.0.0.

One of my interests after learning about Graphite, InfluxDB, Grafana as well as Puppet was to gain knowledge about the Elastic Stack. I know Graylog already but getting things done with Elastic being the primary tool at work fascinated me. And of course the Elastic Beats concept. So I went for an Elastic Vagrant box and also looked into the new 5.x release.

I’ve also finally attended the Puppet training courses. I don’t use it on a daily basis, but now can better imagine how monitoring fits into config management. That made PuppetConf 2016 even more joy for me. Since the Icinga 2 Puppet module is currently rewritten I’m waiting for its final release to integrate it inside the Vagrant boxes. I’ve also contributed a small patch for the InfluxDB Puppet module this year and am hoping to contribute even more.

I did write the Git training courses from scratch this year, and have provided 3 trainings already. In mid 2017 we’ll be adding GitLab to it and focus on even more integration possibilities. It is a “new” playground for me and before writing the training material I thought I would know everything about Git. I was wrong – I only know some of the magic practical examples, but there are so many others. And the feedback and questions during the training were also helpful to just try things out and update the training material.

I have a lot of fun doing social media during events. That’s mainly twitter with introductions and topics which interest me, be it with Icinga or NETWAYS. One thing I really love – sharing that knowledge and motivation to colleagues. I am also retweeting interesting tweets, like Icinga regularly got some Grafana stuff, or NETWAYS posts about Elastic, Puppet and so on. Twitter is what >50% of my co-workers are using for also interacting with work stuff, and it has become even more important being part of the “twitter family” – just to know what’s funny or cool these days 😉

NETWAYS also now has its own event location namely “Kesselhaus“. Primarily used for training sessions this has been opened up for employees to host their own parties. And so I did with the generous help of Georg, giving a party for my 33th birthday. Including some firemen, a blonde transformation and lots of G&T (the Saffron Gin is really tasty).

Birthday presents were lots of Lego too 😉 I did not build so many Lego this year, but then decided to go for Big Ben instead of the rather expensive Lego Technic Porsche model. Building a model piece by piece really is more enjoyable than using pre-defined bricks for that. Turns out that 2017 might bring a new addition to that – the Lego Creator Expert series is celebrating their 10th birthday with a new exclusive model, 1,5 of the size of the normal buildings. Hmmmm 😉

Speaking of Lego – we do have a yearly meeting in March where Bernd and Julian are recapping the last year. It is our own pleasure to find enjoyable presents for all their hard work. This year I came up with the idea of creating a NETWAYS logo with Lego. Everyone was excited 🙂

monitoring-portal.org has a new home, generously hosted by NETWAYS. Even HTTPS was possible. The migration from the old WBB3 platform was pretty rough in early 2016, but we made it happen with an upgrade to WBB4. There still is work to do (more categories, better theme) but it is not yet clear if we stick with Woltlab or turn into Discourse. What I really want to improve is that users who help others will get a reward. Similar to Stackoverflow.

I have improved my spoken English and the way I keep presenting Icinga in my talks. Berlin, Amsterdam, Belgrade, San Diego – in the end I already knew what to talk about next. Well, the workshop at Devopsdays wasn’t good but I learned what else to prepare (just use the Vagrant-Boxes and skip the install for example). I also brought a lot of Dragee Keksi with me. They even landed in Australia 😉

I had a lot of fun finding and creating demos for the Icinga 2 API. Depending on the audience I then decided which demos to show. For instance the AWS EC2 demo isn’t that spectacular when you are in Europe. But the Clippy demo did not need any words to make people laugh and smile.


2016 also told that I should focus on more community activities. I do have some ideas like hangouts or webinars. One I already realised was to create the “monthly snap” blog post series. This one collects positive karma and gives insights into current developments. I do see that in a 2 way benefit – users see what’s fresh and new, and developers get positive feedback and motivation.

OSMC was lovely and top notch. Really enjoyed meeting with “Gin tasting crew” on Tuesday after my Git workshop. I was really tired after the 2 days conference … but that’s also the reason when you meet a former colleague from Austria and just keep talking. I also finally met Stéphane Bortzmeyer, one of those people knowing literally everything about DNS (my former workplace). The hackathon was really good, we made a lot of improvements and I finally met Steffen from Elastic, being the Go and Beats guru.

Last but not least I have to admit that road trip #2 in California was really a pleasure and freed my mind from such a hard year. Dirk and I went for Grand Canyon, driving 2000 miles in 7 days. There was a plan to have at least Las Vegas and Death Valley on the track but we changed that a bit. Going from South Rim to North Rim wasn’t planned but hey – a thunderstorm over Grand Canyon and just you in that little car. In the end we’ve also visited spots in Death Valley I haven’t been before, and also decided to visit Sequoia National Park. Just because there was time and we were free. Last but not least – Venice Beach jumping into Tom and a revival at Venice Whaler with Julian 🙂

I took those memories with me and ended up in getting myself a CX-5 with the help of Markus and Nicole. Now I have a similar feeling when driving – free and independent. A mind changer after all.


Wishes for 2017

Don’t rush things. Let others do their job and wait for them to finish.

Turn email conversations into meetings or hangouts. If something pisses you off, do it face to face. Take notes what’s good and bad, and leave things for a day. Re-iterate problems then with a different mindset.

Deal with problems in a relaxed fashion, propose a workaround or hotfix and then focus on realising a bugfix and a software release.

Try to understand what the other one is really thinking. Imagine a real life discussion if it’s online. Don’t push your own opinion as the most important one. Try to rephrase things from “Your attempt isn’t good” to “Think of this in a different way – how would this look like …”.

Share knowledge. At some point you’re the only one knowing about cool things. Write them down and encourage others to improve and add their experience. This also includes enhancing the docs and howtos.

Look for events and talks you’d like to give. Combine it with some days off and explore the cities and surroundings.

Look into a good mix of things you like to work on. It is not only C++ with Icinga 2 or documentation. That could be …

  • RT and extensions (Perl)
  • Icinga Web 2 Modules (PHP)
  • Things with Go (icingabeat, etc.)
  • Development trainings (Git, etc.)
  • Community demos (recordings, hangouts)
  • Enhance monitoring-portal.org and move to a new platform (evaluate Discourse)
  • Contribute documentation and howtos (I still owe Michael Medin one for NSCP)
  • Discover the possibilities of GitHub and GitLab
  • Embedded hardware programming (things I used to learn during my studies)

40 hours a week are enough. Social media such as Icinga twitter might work in the evening, but prefer spare time over it. If there’s no time at work to answer questions in the community channels, do it the next day but not in the evening.

Enjoy life.

Read Perry Rhodan. Build Lego models. Play board games. Do workout and free your mind. Be offline.

Go for a ride with the new CX-5, explore the world.

Stuff that matters most

icinga_camp_san_francisco_smallIt’s been a while since I decided to go on my journey to Nuremberg working for NETWAYS, doing more stuff with Icinga and get to know what I like most – work on a team with spirit and dedication, learning new things all the way and get the chance to join conferences & meetups. That was 2012, and it’s nearly 2 years already. 2 years where we finally released Icinga 2 after 20 months of development, sleepless nights, and still not stopping there.

What really matters to me, is not the code, or the things we do. It’s the way we do it – professional, but with a love of fun, spontaneity and team work. Getting a drink together on Friday at 4pm, just because we like it. Going skiing together (where I come from), BBQ, XMas party, … there are countless ways you can join this lovely little family (did I mention we loved the spirit of #atemlos and used that for our #b2run slogan?). It doesn’t matter which dialect you’re using – even my Austrian slang sounds familiar these days.

My work and dedication is now being honored by getting invited to San Francisco, joining my fellow colleagues and Icinga team members on the upcoming Icinga Camp. I am grateful for that, that’s something I never expected to happen after the huge success we already gained through our activities online & at various conferences. After my very first Icinga 2 Training last week with kind & positive feedback, it literally pushes my motivation to the next level. Still, I am an Icinga team member for 5+ years now, and it’s even more fun when you know that your work as a team (Icinga and Netways, that is) is still going strong.

It feels like a dream you keep dreaming as a child – and I haven’t been to USA – and soon, in some hours, it will certainly become true. Cannot wait to visit “The City”, and meeting new Icinga users. But also meeting each other where travelling isn’t always possible – I feel glad to finally meet Matthew and Sam being part of the Icinga team 🙂 We (Tom, Markus & me) are leaving SFO after Icinga Camp for a road trip going south (or east even) – Monteray, Sequoia, Death Valley, Las Vegas and finally Los Angeles. Stuff you’ll do only once 🙂

And finally, the most important impressions you’ll never forget – things you cannot plan, surprises which will come, and the “friends & family” feeling 🙂

See you in San Francisco!


Nagios – More drama for the mama

It all starts with a bang

Well, obviously long time before I actually knew about anything like Linux or Nagios. Shortcut – First appearance 1999, trademark issues with the name ‘netsaint’ in 2002, renamed to Nagios which is some acronym for whatever (google it). Versions 1.x and 2.x, simplicity with a plugin api capable for everyone, fetched attraction by many users. We can read that on Google, history, done.

Update 2016-01-15: It is still going on. Now with the Nagios::Plugin Perl CPAN module and legal threats. More to read over here.

Update 2014-01-24 09:29 CET:It seems that the drama continues before it ended, so if you came here reading about “Michael Friedrich”, “Netways” and trademarks, as Ethan G. outlined in his comment on a rhel bug, here’s some additional vita (the rest can be found on XING and LinkedIn). 30, Austrian, 2009-2012 Admin/Developer at the University of Vienna, 2012-2013 Senior Consultant at Netways, 2013-present Application Developer at Netways. Lead Icinga Core Developer since May 2009.
If you’re wondering why last names are abbreviated – I prefer not to expose the possibly offended characters too much. It’s become a psycho drama.

Once you’ve read all through, you can happily start at the very beginning here to read again. Look for the 5th ace and let me know. I’ll happily grant you a beer or two, if you find it.

It’s a drama loop now.

Chapter I: How to (not) deal with problems & fork you

I figure that at some point, when companies have had put Nagios into their monitoring stack, selling consulting, workshops, whilst the original author just wrote code, it got problematic. Especially when people are demanding too much, and provide nothing in return. I do have that feeling sometimes with Icinga users as well, but that makes us human people to tell everyone about it, figure out the input both sides may provide, and reject a feature request, if it does not work out. Still, to remark – listen to the community, and answer.

Start your own business, company, etc is completely legit. The problem arising – community requires love & feedback. If there is none, you really should consider giving up your one-man-show and open up for more developers. While Ethan G. claims that Icinga is “my fork”, it apparently is not. We generally do not own projects. We instead appreciate being part of it. I joined the team one month after they’ve forked from Nagios, looking for a way to integrate Oracle into the *DOUtils backend. The dead nagios-devel mailinglists had made it clear already – a fresh fork with motivated people would allow me to participate and bring my patches upstream.

Basically that fork just happened when everyone’s local fork (Nagios with patches) was growing too big and unmaintainable. It’s still reasonable to do so, also because Ethan G. also suggested forking to users in the past being annoyed by the not-very-reponsive leadership ten years ago.

Chapter II: How to (not) deal with problems & be a drama queen

In Europe, especially Germany, it’s quite common that trademark trolls register trademarks for existing open source projects, and forcing them to rename their projects, and lose their corporate identity and name within the community but also the search engine’s index. I do believe it when Julian says (and told me in person) that he decided to register (and buy) the Nagios trademark in Germany to prevent that from happening. And it all worked out quite well.

  • NagVis was born in 2004, designed by German developers.
  • In 2006, PNP4Nagios was created. The most famous Perfdata Graphing Addon based on RRDTool. Hosted in Germany.
  • The Monitoring Portal was created in late 2003, operated by independent Germans. This is were the community meets. Statistic from 23.1.2014: “12,226 Members – 29,644 Threads – 197,306 Posts (average 51.04 Posts per day)”
  • Monitoringexchange (former Nagiosexchange) operated by Netways offers 2000+ plugins and addons

Trademarks are not free. They need to be paid. So Ethan should’ve been grateful not to pay those fees in Germany afterall, and getting supported by companies caring about the community, letting them work on their spare time projects, and not meaning any harm.

Instead, the bitching about trademarks started. First off, it was about nagiosexchange.org (now monitoringexchange.org), then related to the famous Nagios conference (now Open Source Monitoring Conference) and other brands which were demanded to be handed over to Ethan himself. It ended all with misleading information, a smear campain on twitter.

Announcements by Nagios Enterprises: I II

Statement by Julian

At that time Icinga was already forked, and gained attraction by the existing Nagios community. Furthermore, Icinga is supported by Netways with server hosting and development manpower. Which obviously played a role in this fight, but it’s not that clear.

The outcome for the Nagios community in the end was

  • People started to know Icinga
  • Nagios Business Process addon lost its domain, and it took a while to let search engines adopt bpaddon.monitoringexchange.org
  • the Nagios Conference was called Open Source Monitoring Conference and attracts more participants looking at Nagios & forks, OpenNSM, Cacti, Zabbix, etc
  • Confusion between exchange.nagios.org and monitoringexchange.org
  • Bookmark update fun

In the end, we also had the Nagios drama mailinglists announced.

Chapter III: How to (not) deal with problems & act like a child

At some point, Jean Gabes proposed his rewrite of Nagios in Python to the Nagios Developer’s list. While it was to be expected that Nagios developers wouldn’t just switch from C to Python, the reactions after making Shinken a “Nagios compatible rewrite” were a bit strange and childish. Especially in terms of approaching other people. Or attacking even.


Only quoting a passage here:

blah, blah, blah,
You heard about the drama on the mailing list but yet you are still busy creating more. If you truly believed that flaming is nonsense then why do you initiate it? It is very humble of you to request the flaming to stop after you have had your inaccurate dramatic statements posted. I suppose you thought your statements would not be challenged because we usually do not respond to ridiculous rantings of a seemingly angry person who jumps to inaccurate conclusions and makes rash assumptions.
Your reason for forking lacks substance. Nagios is an engine. A very powerful engine that has gained in popularity because Ethan Galstad has been a very successful programmer and gatekeeper of the project. If you are not happy with the progress of the project, so what! You are just one person . There are millions of Nagios users that are very happy with the Nagios engine. We do not feel the need to cater to one or a few angry people that are jealous of Ethans success. Your lack of respect for Ethan is appalling, knowing that Ethan spent 10 years of his life developing Nagios and cultivating the community.
You “forkers” try to justify your actions by saying it is for the good of the community. The rest of us know the real reason is to satisfy your own egos (really twisted). You must feel unethical about what you are doing or you would not have tried to blame us for your actions. IMO, you forked because you and Gerhard lack the skills and creativity to start your own project. You would rather “steal” someone else’s work. Further evidence of your lack of creativity is posted on your own website where even your tag line is stolen from Nagios.
It is time to stop the negative, inaccurate, flaming drama directed at Ethan and Nagios. Do your own hard work and stop attacking Ethan and the work that he has done for the last 10 years. Please share my response with your partner in crime, Gerhard Loser.

That doesn’t sound very constructive, and contains plently of personal attacks. Gerhard’s last name actually is spelled “Laußer”.

Chapter IV: How to (not) deal with problems & compare yourself

At some point, your customers ask you about your competitors. Or community members even. How do you compare to XY? We had that with Icinga a lot, and therefore created our version of it. While it may make Icinga look good, we’ve learnt one thing: The previous version compared Icinga with Nagios Core and Nagios XI (the commercial entity). While Icinga is free in every single line of code, Nagios XI isn’t. We decided to skip that part, and removed that column from the Icinga comparison chart.

After a while, we came across this interesting OpenNMS article talking about Nagios FUD. And we figured, that Nagios Enterprises had created a comparison chart for Nagios vs Icinga themselves. But actually that one lacks of any substance, comparing Icinga Core with Nagios XI (the commercial one), but only flagging it Nagios. At that time it was unclear that “Nagios” means “Nagios Commercial Version”, while “Nagios Core” follows the free open-core model.

Either way, the pdf also contains false information about IP violation done by Icinga and Netways. Both never have violated any intellectual property, but probably that’s the only arguments to tell managers and buyers that Icinga is a risk, while apparently it’s just pure hate against the fork, spreading FUD themselves all over the internet. How_Nagios_Compares_To_Icinga of that pdf, copyright both pdf and the quote below owned by NE.

About Icinga
Icinga is a young project designed as incompatible Nagios fork by German company Netways GmbH. Icinga suffers from high developer turnover and poses legal risks to organizations that deploy it.

Use Caution
Both Icinga and Netways GmbH have a history of intentionally violating international intellectual property laws. You may be putting yourself and your organization at risk if you deploy or implement Icinga in your organization.
Knowingly using or working with products or companies that violate intellectual property laws and treaties can have significant legal and financial repercussions.

After reading this bullshit, and not liking the icinga.org Matrix either, the developer in me decided to invest several days to create a comparison from a developer’s point of view. Since many developers don’t necessarily mean bad project quality. And quite frankly, being the release manager since 0.8.3 4,5 years ago, it’s still annoying to see that they advertise their lies publicly.

Btw – there are other comparisons with Centreon, Groundwork, Opsview and Zabbix.

Chapter V: How to (not) deal with problems & censor the internet

Wikipedia is an open-minded community, enforcing free speech in their articles. The english article about Nagios contains references to its forks. Some people obviously influenced or working for Nagios Enterprises (whois said so) have had a different opinion on that, and tried to censor that Nagios article by removing all unwanted strings and urls. Or, they’ve added subjective meanings into it, like “Nagios XI is the best monitoring tool in the world”. That did not end well for them after several attempts of fraud on that wikipedia article. It got even worse – Wikipedia banned Nagios Enterprises being sockpuppets.

And – trademarks join the drama queens the first time. The French Nagios community was formed by Olivier J. at former nagios-fr.org also talking about Icinga being a Nagios Fork. It seems that Ethan G. and Mary S. (wife) didn’t like that much, and attacked Olivier in a way, well, no words for that. But go figure, transferring the domain and changing the community portal to monitoring-fr.org was the only wise choice. And it’s still a good community portal for everyone, not only Nagios. And you may not censor their articles, but read them – Chrome offers auto-translate anyways.

Chapter VI: How to (not) deal with problems & treat community members like shit

In early 2012 there was an interview done by freesoftwaremagazine.com with Ethan G. and Michael L. – both sides added their arguments, comments, etc. While the article was great, I felt that it required some more additional content (some of it is found in this article as well now).


A short time after that article, Michael L., the former owner of nagios-portal.org got a letter from ICANN that Nagios Enterprises has claimed the trademark on the domain, and that the “Uniform Domain Name Dispute Resolution Policy” now applies. .org TLD disputes are directly handled by ICANN, while cTLD disputes for example for .de are handled by the local domain registry “DENIC” (and not even that, but this is handled under civil law in Germany). That ICANN procedure is pretty annoying and for non-native English speaking people a mess without lawyers.

At that time, it was already known thanks to the “Julian incident” that trademarks are now handled differently by Nagios. So the domain “monitoring-portal.org” had been prepared long time ago, the primary record was changed, and Google did get a hit. The final announcement left hope that the community would go one like before. And we weren’t mistaken – it’s even more international after all those years.

Sending a patch to Nagios to tracker.nagios.org in the name of the Icinga Development Team resulted in “Changelog clarifications” by Ethan G., and after claiming the copyright on that patch, on a ban on their tracker. Meanwhile on the nagios-* mailinglists the string “*@univie.ac.at” was banned (my previous employer) – i know that because I couldn’t write to nagios-plugdevel either. Holger unblocked me when we were debugging the root cause.

Chapter VII: How to (not) deal with problems & work with a zombie community

Have a Nagios core developer maintain the patch queue & redesign the Nagios Core 4 – awesome. Kicking that developer out of the dev team just because he didn’t test the cgis properly – priceless. So yeah, 2013 was when it all went shits with Nagios. Fun fact: Andreas chose the Open Source Monitoring Conference (the one which had to be renamed due to trademark claims by NE) to literally hijack the “Nagios future” presentation, telling the audience that he got removed, and forked Nagios 4 into Naemon. Well, he actually wrote 99% of it, so no big deal.

So it really feels like that the (former) Nagios community now meets at conferences, presenting new features and having lots of beers with community members. And it doesn’t matter if it’s called Nagios, Icinga, Shinken, Naemon or anything else. At OSMC 2013, we had a great time with Zabbix & OpenNMS developers too, it felt like one big happy family. Not celebrating that Nagios got forked again – only having a great time together, and leaving each other with visions & ideas.

So it’s not about Nagios anymore, and the community changes as well. nagios.org is bloated with do-it-yourself, student, pro and commercial editions so people won’t easily find the download urls anymore. Everything is about business and selling a product. And the difference between Nagios and Nagios XI is not clear – rather close the webpage and ask for alternatives these days.

Content in that picture is copyright by NE.

The final chapter

Many people still use Nagios. They are annoyed by bugs, or broken releases. Missing features like real distributed monitoring, or stable apis are bad, but there are other tools on the market solving those. You may still make Nagios write to graphite somehow. But at some point, #monitoringsucks will apply.

In any way, installing a Nagios/Icinga/Shinken/Naemon/Centreon/Opsview Core will require plugins. Small nifty executables, be it compiled source binaries, or just perl/python/etc scripts returning some output and an exit code. If not, your first look onto the webinterface won’t be green. Green means “Everything is OK, don’t panic”. As a matter of fact, every single admin gets nervous if there’s something not green. Especially after the first install.

That being said – without the enormous effort of the Monitoring Plugins Development Team (former Nagios Plugins Development Team) all those core check engines would be nothing. Not even a single customer would buy your support without plugins making this work out of the box. We (in terms of the Icinga team) include the Fedora EPEL repositories in our Vagrant demo boxes for Icinga 2 pulling the nagios-plugins package.

Last week, wednesday 15.1.2014 at 22:38 to be exact, the following dialog happened on IRC (#nagios-devel irc.freenode.net)

22:24:40 < dnsmichi> is that the iframe forwarder thing?
22:27:37 < emias> dnsmichi:
22:28:11 < emias> dnsmichi: And if you query that web server with "Host: www.nagios-plugins.org", you'll get a modified version of our home page.
22:28:25 < dnsmichi> modified?
22:28:34 < emias> dnsmichi: I'm pretty sure yhey're taking over the domain.
22:28:42 < emias> "they're"
22:29:11 < dnsmichi> i guess. they are trying everything to hide projects talking about nagios forks.
22:29:57 < emias> dnsmichi: Yes. Ethan had sent an email a while back, seems ge wasn't happy with my response.
22:30:00 < emias> s/ge/he/
22:31:59 < emias> Soooo!  Are we totally sure?  Then I guess we should put up monitoring-plugins.org and post to the list as soon as possible.
22:32:35 < dnsmichi> No. But 2014 is a good year to change things
22:32:41 < emias>
22:32:54  * emias wanted to go to bed early today ...
22:33:02 < dnsmichi> Like leaving sourceforge
22:33:47 < emias> Well that's done.
22:34:25 < emias> As we moved everything to our own server (+ GitHub), switching to monitoring-plugins.org would be very easy now technically.
22:35:46 < emias> http://xxxxxx/archive/np-new.html
22:36:00 < emias> That's the content I currently get from their server.
22:36:33 < emias> I think I'm sure enough.
22:38:10 < dnsmichi> ehm
22:38:24 < dnsmichi> i just figured
22:38:35 < dnsmichi> that webserver actually serves nagios-plugins.org
22:38:49 < dnsmichi> i miss icinga, neamon, shinken on the main website
22:39:11 < emias> dnsmichi: That's what I'm talking about.
22:39:40 < dnsmichi> yeah, but not only via telnet Host trickery, but live in my browser
22:40:10 < dnsmichi> can you edit the main page and look if your changes are applied?
22:40:37 < emias> heh
22:41:36 < dnsmichi> i think you have been hijacked.

So basically, at that time, the websites were identical, after the Nagios Plugins Development Team was required to rename their project. I may be biased and have reacted emotional in that situation, but my german speaking brain told me, that copying a website, and changing the dns records would mean: Put your own (censored) content into the user’s sight. He won’t notice. That’s just like getting a mail asking for your bank account details, showing you a website which _looks_ identical, but isn’t technically. So if nagios-plugins.org would allow to change your passwords, they would simply have fetched and stored your passwords being able to use that against you.

On Sunday, 19.1.2014 it looked like this


In that specific region, the problem is even more specific: nagios-plugins.org is offering a software download. A software which is installed on thousands of servers. By serving an identical software package, everyone will trust it and just upgrade/install it, as the documentation of many projects say. The user won’t recognize any changes. What if they decide to add a phone home functionality to their software? You’ll never know, because you are trusting the name. You are trusting a vendor, which had been replaced. That obviously means “compromising” or “hijacking” a website.

The legal part of copying a copyrighted content just came up later in the drama (being totally illegal to act like that). But still, NE at least recognized that, even if they didn’t put it into a public statement. nagios-plugins.org now looks a bit different than the original content now served by monitoring-plugins.org. Even though, their actions are still illegal. But if no-one stops them, they will continue like before.

Either way, there was an announcement the day it happened on the website, and later to the mailinglists, where a discussion started. Also some insights on the previously happened Naemon fork (look through the archive, if interested).

On thursday (16.1.2014) the discussion on IRC was still ongoing. At that time, I’ve taken the liberty to step forward, and inform the package maintainers of ‘nagios-plugins’ while the now kicked and renamed team started to continue their renaming process. Jan W. is also the Debian packager, so I just informed RHEL, SUSE, FreeBSD, OpenBSD, Gentoo and ArchLinux. These distributions are well aware of Icinga, and I do have some contacts over there. Or the other way around – me being a developer changing stuff wants to inform packagers making their lives easier. I’ve been maintaining the Icinga RPMs for 4 years now, so I know what I’m talking about. Upstream fucks up, you’re screwed with mass reworking your packages.

In any attempt made, it was about users and their changed upstream source infecting their chain of trust.

Somewhere between thursday and friday (17.1.2014) the story hit reddit (it wasn’t me). There were plenty of comments in 2 threads (linux and sysadmin). While ‘scumbag’ isn’t a nice description, reddit still allows you to name it like so. And apparently, given the fact that NE did not tell anyone about their website defacement, ‘scumbag’ the is very least one could say (not that I would, just saying, eh?).

Interestingly enough, that story was to be read on Slashdot too. And then a bit more widespread on twitter as well.

In parallel to all the news, and also overwhelming feedback & appreciation of the monitoring plugins (and also Icinga which normally benefits by any Nagios drama), there was the discussion on-going on the bugs I’ve opened for the packagers. While I was and am not essentially interested in helping NE to gain their ‘nagios-core-plugins’ package, it’s still vital to know that the Redhat bug unleashed a competitive discussion how to proceed here. While NE employee Andy B. always insisted that the Monitoring Plugins project is a fork of the original (and fixed the git repo with a single commit after my remark of trust – that same passage is also quoted in that interesting blog article referencing my community concerns), the general outcome was that no-one is able to use/update the ‘nagios-plugins’ package in EPEL. Which is a good deal imho so far.

The thing with a newly formed Nagios Plugins team is – they don’t know the code very well, even if claimed otherwise. Not even Eric S. who tends to be the only part-time core/ndo/nrpe/nsca developer of NE, may handle all the stuff. From my personal observation following their git history closely he broke more stuff than Andreas ever did (getting paid for breaking Nagios? Haha, Just kidding.).

On monday, 20.1.2014 Ethan G. joins the drama for even more drama. NE has announced the new Nagios Plugins team, while attacking Holger W. using his full name in their company blog. What a shame – no more words to add here.

Content copyright by NE ofc.

In any attempt, nagios-plugins.org now looks different (Wikipedia tells about the controversy too). They’ve also changed their opinion on mailinglists (recently shut down to prevent Andreas E. announcing his Nagios fork, as rumors like Andreas E. do tell) and have now auto-subscribed all users already subscribed to monitoring-plugins list to the nagios-plugins list. Copying the subscribers database and using that personal data against the unwanted project? Well, nice try. Users are confused anyways, and will happily unsubscribe from the two mailinglists looking for alternatives to follow the news or look for support.

Anyways. Too much drama here involved. FreeBSD, OpenBSD, SUSE, Debian have reacted. Some of them will just move from ‘nagios-plugins’ to ‘monitoring-plugins’ and oprhan the old package. For RHEL, Sam K. is working on a new package (I’ve only done some additional package foo in order to help resolve all the remaining showstoppers) becoming the new ‘monitoring-plugins’ maintainer in EPEL 7. For EPEL5/6 there may not be an obvious solution, but we’ll make sure to get something working for our communities.

Even if a Nagios user installs packages from packages.icinga.org being happy about fixed problems. Is that a crime? No it isn’t. But NE thinks it is. But it doesn’t matter in the end. We (the forks) are the bad guys, while NE and Ethan G. are the ones who never started anything, and are being harmed by us. Tbh I consider this a personal attack, but hey – that’s a RHEL bug to fix problems. Not to generate more drama for the mama, eh?

An now for the closing end, some more dramatics dramas. Well, first off an interesting blog entry by Matt Simmons, which showed that my English is actually bad in terms of vocabulary. Using the term “compromised” actually had put the RHEL bug up to hacker & security news feeds. Well, public attention for what price. Still NE is probably using that argument against me, but who cares anyways. That really was a language accident or incident, call it whatever you like 😉 Oh, and I’ve made sure that the internet doesn’t forget even if this story goes offline.

There are probably some more addons or plugins around which are compatible with Icinga, or mention that on their websites. They’ll probably get an email soon-ish and being ordered for censorship serving the greater power of the NE dictatorship. And then they’ll lose their motivation. Or step down, because renaming all the stuff causes that much work, moving the domain, re-establishing a trust platform. Or they are forked by NE into their enterprise stack, like happened with Nagiosql (now Core Config Manager), NagTrap (now NSTI) or Teeny Nagios (now Nagios Mobile). But you as addon developer get all the support questions, that’s for sure.

Nagios Enterprises has shown in many ways how to kill their own community being grown around Nagios. Conferences have started popping up where people meet not to only talk about Nagios (the Nagios World Conference Italy renamed itself to Open Source Systems Management Confernce while the Open Source Monitoring Conference attracts >250 open source enthusiasts every year here in Nuremberg). Community platforms which have resisted against their censorship and renamed themselves (the real community lives at monitoring-portal.org). Addons which are looking for alternative hosting platforms. Developers who just do not like them for their contribution agreements (and other licenses which explicitely forbid you to fork). Everyone chooses their platforms where they’re free. Like we are on #icinga on freenode. Or social media where censorship just doesn’t work. Not even fork developers will send them patches anymore being either banned or ignored.

In any way, the ordinary user will not create any plugin or addon in the future. There are too many ways to fail, and if you’re open-minded you’ll get a letter from a lawyer forbidding you to talk about competitors with your Nagios-exclusive tool. A zombie community – and then it will be interesting to see how the open core business model of Nagios will be able to survive, if the community stops contributing. Or if the community starts over in Icinga/Shinken/Naemon, or any other cool devop tool such as Sensu.

But, that’s another drama for the future. In order to be prepared, get this t-shirt. Then meet with all Nagios forkers, create cool code & laugh with community members, and enjoy your beers.

And leave the drama to those having abandoned their old friends.

XMas – Time for malware removal

Spam filters may not filter xmas mails which offer something for free, click once and you’ll get your browser search toolbar & hijacker without any further notice. On Linux, most of that crap can be avoided (except the browser specific stuff which may still be hijacked). On Windows, this is truly hell.

Today I’ve learned that “Plus HD 2.x” isn’t really a helping tool to always download youtube videos in full hd, but some adware crap (“Crossrider”, Websearch using url24.info as tracking provider, someone said Redtube?) which injects your system with url tracking and redirects. Same goes for toolbars such as Ask, Babylon, etc mostly shipping with so-called “free software downloads”. Most prominent examples – Java, Avira Antivir or Daemon Tools.

While toolbars aren’t essentially that bad, tracking the user’s surfing behaviour imho is. My swiss army knives in this direction have been for now

  • Spybot Search & Destroy (cleans nearly everything, but is a bit slow)
  • Malwarebytes Anti-Malware (cleans what Spybot doesn’t detect)
  • CCleaner (for the generic cleaning of tracking cookies, temporary files, etc)
  • HiJackThis (for advanced users – there is no indication of good/evil, removal of registry keys for browser addons, startup entries, etc)

For heavy problems with rootkits and other malware surviving such cleanup orgies & reboots, I tend to use

That entire list can be extended with the following tools which have proven themselves very useful to clean the last chrome addon entry file and entirely removing that malware crap.

Essential tip for the future – do not install everything which pretends to be free, and keep your antivirus software signatures uptodate whilst running spybot and/or anti-malware every month.

irc ignore list in irssi config

Once in a while you’ll get users on community support channels which may flood your private queries with support questions – even if told not to do so (ignorance is bliss, eh?). The simplest way is still /ignore but i prefer to have it globally configured in irssi.

$ vim .irssi/config

ignores = (
        { mask = "MosquitoCR";   level = "ALL"; },

Silence … ikillu! 😉