Ask most any sys admin what the No. 1, go-to product is for log analysis in the server world, and odds are you'll get the same answer from just about every corner: Splunk.
Splunk started strong and has only grown stronger as it's branched out to become a wide-ranging analytics platform. But the free version of Splunk is quite limited, and the enterprise version's pricing is based on the amount of data indexed, which adds up to prohibitive costs for some.
Still, some open source competition has been emerging in the background, via such projects as Logstash or Kibana. One project of particular interest is Graylog2 -- available under the GNU General Public License v3 -- available as a new point release, 0.20.1.
Graylog2 leverages three common technologies to do its magic, two of which are major open source items: Java 7, Elasticsearch (which Logstash also uses), and MongoDB. In particular, Elasticsearch grabbed the attention of InfoWorld's Andrew Oliver the other week when it reached full 1.0 release status, since it provides not only search but detailed analytics functionality and is strongly scalable.
Graylog2 comes in two basic components: the server itself and a Web interface. The pair can be deployed together or separately. A set of REST APIs lets you consume the service in your own customized ways, and a plug-in architecture lets you add more server-side functionality -- say, a module for siphoning up tweets as a data source. Client libraries exist for other environments as well, such as Node.js, by way of the NPM repository.
The various components of Graylog2 can also run in different places as needed. If you already have a MongoDB instance on a dedicated box, for example, you can use that as part of your Graylog2 setup. This stands in contrast to Splunk's more monolithic approach.
Despite -- and perhaps also because of -- its cost, Splunk has flourished in various ways. It's been made available as a full-blown hardware appliance by a third-party hardware vendor (Taiwan's SYSTEX Corporation), requires less moving parts to get running, needs less tweaking on setup, and has a broader feature set. Hunk, a spinoff of Splunk, lets you run the same analytics as Splunk on Hadoop nodes.
But there are places where Graylog2 is clearly catching up. It's now available as a free AWS AMI image or a free Vagrant box, as well as via the usual gamut of packages, to make deployment a little less cumbersome. Given that Graylog2 is based on Java, it'll be interesting to see if it, too, will eventually become a Hadoop-deployed app.
This story, "Splunk feels the heat from stronger, cheaper open source rivals" was originally published by InfoWorld.