The War for the 4 Pillars:
Big Data, Cognitive, Cloud & IoT

I recently attended IBM'S World of Watson conference and before that HPE Big Data, and the NY Data Summit. I also virtually attended the Teradata Partner Conference and selected sessions of Oracle OpenWorld, Microsoft WPC and others.

One thing is abundantly clear - there is a war on the new turf of Business Intelligence/ Analytics/AI (IBM calls this “Cognitive”), Big Data, the Cloud and the Internet of Things (IoT). As seen at World of Watson, these are IBM’s 4 pillars and Big Blue is gearing up for an arms race and a big war - the war for mind share and for money. According to analyst firm IDC, big data and analytics spending will hit $203 billion by 2020.

The availability of data, a new generation of technology, and a cultural shift toward data-driven decision making continue to drive demand for big data and analytics technology and services. This market is forecast to grow at a compound annual growth rate (CAGR) of 11.7% through 2020. - Dan Vesset, IDC

Companies are dealing with an explosion of data and are trying to make sense of all of this. And software super vendors like IBM are jumping into the fight by providing everything needed to implement these 4 pillars. They provide whatever you need to host (in the cloud), capture, manage and analyze your data that will hopefully ease decision-making, facilitate efficiencies and provide a competitive advantage in this global marketplace. Amazon, Microsoft, IBM, Oracle and SAP are some of the other super vendors that are connecting these dots. I believe there will be lots of consolidation of software vendors. Acquisitions of smaller vendors with complementary technologies gives each super vendor a temporary competitive advantage and is the only way for the smaller vendors to avoid becoming road kill over the long haul. Smaller vendors with competitive software are in for a rough ride.

The global managed service providers are adjusting strategies to prepare for this war too. Led by companies like IBM Global Services, Wipro, Accenture, TCS, Capgemini, Infosys and others, they will be assisting their clients with implementing solutions to fit these needs. Whether it be moving data from mainframes and flat files to big data lakes and/or data warehouses, or migrating from one big data vendor to another, there will be from terabytes to petabytes of data involved.

All of this assumes the underlying data is in reasonable shape. But is it really? What of data governance and data quality? Why is that lagging behind?

It reminds me of the early days of automated software testing (see article here) where interesting development processes of the time, like RUP and XP, were moving developers away from the waterfall process. Yet the support for automated testing lagged way behind. And without test automation speeding up the process, it caused critical applications to be partially tested, leaving them extremely buggy - resulting in huge delays in delivery and being way over budget. That seems to be where everyone is now with data quality and governance, but help is on the way. There is an Apache project named Atlas that is focusing on data governance. And there are solutions that automate the validation and testing processes - solutions like QuerySurge and others. But right now, the quality of corporate data is sketchy at best and big data lakes are in danger of becoming swamps.

Forward thinking software vendors that provide customers with the cognitive, big data & cloud solutions and opportunistic managed services providers that offer their customers a way to harness, validate, measure and analyze their data best will win the war, as will their customers, who will have a huge competitive advantage.

The war for corporate mind share and money around the 4 Pillars is raging now. It will be fun to see how this all plays out.

About the Author

Photo of Bill Hayduk

Bill Hayduk, CEO, RTTS

Bill founded software and services firm RTTS in 1996 (the parent company of QuerySurge). Under Bill's guidance, RTTS has have had successful engagements on hundreds of projects at over 600 corporations, from Fortune 1,000 to midsize firms. Bill was also the CEO of TOMOS Software, the award-winning ALM software that was incubated at RTTS, and QuerySurge, RTTS’ leading big data testing software solution. Bill lives in Westchester,NY with his wife and 2 sons.