Recently I was asked what I thought was the most important trend in IT. And it wasn't hard to answer: The shift from proprietary, scale-up systems to commodity, open source, scale-out systems.
NoSQL databases provide a good example of scaling out versus up. To beef up Oracle or any other conventional, proprietary RDBMS, you need to scale up the database server with more cores, more caching, more flash, and so on. At a certain point, you'll need another server, which means you'll be saddled with sharding the database and maintaining it in that state (yes, you could use Oracle RAC, but that's another story).
With a NoSQL database -- and all the major ones are based on open source -- you scale out when you need more performance or capacity. Instead of powerful database servers than can never go down, you deploy a NoSQL database across commodity servers in a distributed architecture that permits failure of individual nodes. A popular analogy uses "pets versus cows" to illustrate. Powerful scale-up servers are like pets: You spend time and money to keep them happy and healthy. Scale-out commodity servers are more like cows: When they die, that's just part of the business, which does not suffer when Bossie goes belly up.
At the infrastructure level, private cloud management of hundreds of thousands of VMs running on white boxes can deliver a similar effect across a wide range of applications. Scalability through virtualization has now extended to SDS (software defined storage) and SDN (software defined networking). As I noted last week, Stu Bailey of InfoBlox recently announced the LINCX open source project, which is essentially a software-based networking switch that you can deploy in a Xen VM on almost any server. How's that for a threat to the network hardware business?
The trend toward commodity, scale-out infrastructure running open source software is still at an early phase in most enterprises. But it's already starting to turn the process of choosing, buying, and deploying enterprise technology upside-down, particularly in these ways:
- The total spend: With cloud and configuration management software, you can manage very large infrastructures with many fewer people. Enterprises have seen the fruits of server virtualization, and they want to go to the next level. Even more obvious is the cost savings of deploying commodity versus high-end servers or open source versus commercial software.
- The pace of change: The old cliché about "losing strategic advantage" if you miss the latest developments is coming true. Can you afford to ignore Docker if you want the flexibility to move applications among different clouds? Do you really want to leave your developers at a disadvantage by clinging to cumbersome old tools and processes? Open source pilot projects cost time, of course. But to the extent that you've deployed cloud infrastructure, you can cut that time dramatically with automated testing and deployment.
- Who chooses solutions: IT management just doesn't call the shots like it used to. Lines of business and marketing officers get tired of waiting for IT and procure their own solutions -- often through the public cloud, which offers the scalability needed for new initiatives that may either flop or take off like a rocket. Moreover, no one in an ivory management tower can keep up with the torrent of technology being generated today. Instead, developers often discover new and exciting open source projects and put that code to task to see what it can do. If it lives up to its promise, developers sell it to higher-ups who hold the purse strings inside the organization.
Following on that last point, the increasingly social nature of coding and the explosion of open source software is creating a technology recommendation engine the likes of which we've never seen. For a new generation of developers, the prevailing thought is: With the cost of entry so low, why code from scratch when you can simply plug in someone else's clever open source solution? Plus, when a project gets white hot, an ecosystem of satellite projects springs up overnight.
I realize that even today, change in enterprise tech takes longer than you think it will. But the carrot here is a particularly juicy one: Very large cost savings and vastly reduced time to market. Yes, it's going to take a while longer to loosen the hold traditional vendors have on IT management. But in the long run, the march to commodity, open source, scale-out systems is going to happen whether those in charge today like it or not.
This story, "One enterprise tech trend to rule them all" was originally published by InfoWorld.