The Coming Explosion Of Massive Scale Systems

I was speaking to the head of emerging technologies and research of a top analyst firm yesterday and one of the things that he highlighted as a key trend in the massive surge of investments focused on infrastructure virtualization and consolidation. His firm specializes amongst other things in tracking data and he definitely has the numbers to back him. What next is the question that both of us were trying to grapple with. The answers seem to be slowly emerging. Don’t confuse commoditization of computing with commoditization of computers – they are very different argues, Dr. Greg Papadopoulos, in what could be a defining model for the future of tech players. While recollecting the progress of 100,000 to 1M Times More MIPS/$ in 25 years, he highlights the 30X improvements in size, 24x improvements in power, 40X improvements in weight, 2x improvement in performance – all that has happened in the last ten years, achieved partly by exploiting massive parallelism in the core services, platforms, O/S instances and servers, storage & switches. Courtesy of Jonathan Schwartz saw his CTO’s analyst meet presentation, the shift is captured more succinctly here –in the past, Computer + Storage + Network + Power + Cooling+ Software= Blackbox ,as we move forward redshifting of services would mean that enterprises need to accommodate features like identity & security, SOA& Web 2.0, new clients etc. The Enterprise bridge shall encompass new players like eBay, Google, SFDC, Webex etc.. The power of leveraging these elements would demand more computing powers than the advances extended by Moore’s law, characterized as Redshift. Lot more computing cycles are needed and the resultant expectation of system administrators shift from virtualization & consolidation towards achieving efficiency at massive computing scale. Obviously there will be some early embracers of this and many who would wait and watch and this opens up different strata of markets for tech players, consulting powerhouses and system integrators.

In other words, you're going to need a lot more computers (or at least a lot more computing cycles). Whereas the focus of those organizations running traditional business-computing operations is cost cutting through consolidation, the focus of those operating the new mega computing operations (like Google, say) is achieving efficiency & predictability at massive computing scale (through, for instance, reducing electricity consumption). The huge advances that are being made and stratification of customer classes that demand about scale and efficiency in new ways would call for different approaches towards serviceability. There's a fundamental split opening up in the market, in other words: two very different sets of customers (one with stagnant demand and one with burgeoning demand) with very different needs. This is a precursor to a potentially change in business models of tech players and its not clear how consumers would benefit cost wise on this – normally such shifts come with tremendous increase in unit prices for early adopters, while providing the tech edge. Clearly, interesting times ahead.