Why streaming analytics is such a big deal

Some decisions just shouldn't wait

I see a future with relatively little batch processing, where even long processes will run a bit at a time. As traditional storage gives way to new data architectures and streaming becomes easier, “real time” data analytics will become the new normal.

With a client-server architecture dependent on a relational database management, streaming or event processing is relatively rare. You have traditional messaging products like Tibco, MQ Series, or your favorite messaging implementation. These scale well, but not massively -- and when kept in sync with your RDBMS, they scale only as well as your RDBMS.

More often than not, you end up doing your analytics on the back end. You’re in good company: When Google wrote its MapReduce paper, it was analyzing the Web in this conventional way.

Google moved on to streaming, however, and so should you. Systems based on streaming analytics require more resources initially, but make better use of those resources over time. Part of the reason why is they don’t reanalyze all of history simply to get a result.
View Comments
Join the discussion
Be the first to comment on this article. Our Commenting Policies