Don’t get sloppy around the bit bucket.
Big data is well just that – Tera, Peta, Zeta.. tons of data. Lately, it has also become an excuse for inaction. Every operator living in a challenging work environment has declared data the new Nirvana. When their product offerings are failing in the market, no one thinks to change the product, or change the approach or change the leader who concocted the plan. It is just a lack of understanding the data. And if we string together enough data, the answers will magically appear and all will be right with the world.
And because it rolls downhill, the thirst for data usually becomes the CIO’s issue. Business and financial Analysts are tasked with pulling data from every imaginable system in the enterprise in an attempt to create reports that can put a positive spin on the latest business results written in spin, I wish I could make this stuff up:
Our rate of decline has improved”
“Although we missed some of our targets for the quarter, we have much more significant goals for the future and we are pleased with this quarter’s internal and external trends.”
The introduction of HADOOP in the enterprise has raised bosses expectations of the availability of data and compounded the misery of engineers, analysts, data scientists, report writers and of course the CIO. Since the data arrives at the door of the CIO in horrible shape, it must be massaged to a usable form. Today this means Web Analytics combined with data pulled from enterprise sources such as Salesforce, SAP, Seibel as corporate systems running large Oracle data bases. Simply pushing massive quantities of data into HADOOP will not solve any problems. Turning a rush of data into useable information requires business leadership, analytical expertise and data management competence.
Business leadership starts clearly defining metrics and goals. The CFO organization must be able to translate goals and metrics into an operating plan. The analytical expertise is then needed so the business and financial analysts can create a detailed plan with metrics and reports to determine if everyone is on plan. THEN will your data team be able to get a clear understanding of the types of data required, and structure it for consistent reporting results. I know I know, Big Data is synomous with data exploration but the enterprise wants resutls right nwooow! Expectations are high in the C suite.
And then an endless amount of requests follow for reporting because knowledge is power and if so and so has access to the data, then I want access to the data. And to top it off, get me all that stuff daily before 9AM so I can digest the data while I down my expensed double Latte from Starbucks.
“Don’t be late either”. Because if you are late, I am going to run to your boss and have a serious talk with them. I will introduce doubt about whether you are the person for the job or do we need a Googler to come in here and kick some A–. Your boss is somewhat supportive but knows the average life expectancy of a CIO is less than two years and most quit themselves. Why bother making the CIO’s brief stay any more miserable? They usually just let the complaints pile up knowing the fool who took the job will be gone soon anyway.
The one thing the boss will get ugly about though is a call from their boss or the CFO. The call is usually an FYI to let them know that the Bonus Santa will not be visiting this year because you and your crew are so horribly over budget. And to top it off, most of your internal customers are so unhappy, they are gathering down the hall with pitch forks and torches and thinking seriously about running you off. Now, if you had managed the money right, I could have helped – but well it is all about money now isn’t it?
All this is enough to convert the can do CIO into a Zombie CIO. This is when things get crazy and chances are taken. The “Big Data” CIO will consume just about anything that will help them lasso more data and do it more cheaply. All kinds of wacky looking vendors show up to save the day. And the CIO short cuts the process by trying too many too quickly. Instead of a long test period, the vendor is usually “live” within weeks, even days of coming by for the sales call.
This is when the CIO needs to be really careful. The established vendors often sell you a fix but it comes at a price. The established Vendors can smell desperation and they know just how get you hooked enough so you actually need what they sell. You then see the dark side of established team. For as much flexibility as they had on trials, they have absolutely none on pricing.
Data Appliances fit into this category. They are easy to understand and quantify and for vendors to price and tend to have low installation and operating costs.. but this comes with a hefty upfront cost and maintenance fees. Plus you are locked into a single vendor solution that has limited flexibility and major upgrade costs. Your vendor sales team will make a case that you should pay more. It is cutting edge stuff it will lower you TCO especially when they cut you and your crew and ship the whole shop to a low cost location.
Yes, you have to watch for the established guys but then you have to also be leery of the wannabes. The wannabe vendors usually show up in teams of two really tired guys. The wannabes like to dial in the other members of the company for a big conference call to explain what they can do for you. You are never quite sure if the guys on the phone actually work for the company or whether they are literally being paid to take the call.
Trials are less easy with the wannabes because they are writing their own code to with a whole pile open source code. Ah that open source, it is like a big schizo dog. Sometimes it all goes well and open source works as intended and other times, it puts you on the run, trying desperately to get away from its snapping jaws. So the wannebes usually blow a few things up, get a few people pissed off, make the CIO fear the absence of the bonus Santa before things start working as intended.
The payoff with using open source based solutions is harnessing the brain power of the many. Vendors can create highly scalable open source solutions or have read the Google Big Table white paper and rolled their own.
But Open source carries litigation disease so the CIO must make sure to be properly inoculated. It is a good idea to get proper indemnification from the patent trolls before engaging a wannabe. If someone is getting edgy about you asking for protection then that should be the first sign that they could be hot and should most likely be avoided. It is also a good idea for the CIO to consult legal. Calling legal may be the corporate equivalent of saying no to a deal but with the wannabes, you may not to fall in love so fast.
But look there is hope out there. Some of these wannabes have created data technology that is capable of processing massive quantities of data and scale far beyond a traditional RDBMS data base. The first round of wannabes including Vertica, Greenplum and Aster Data have been purchased by large companies. Exisitng independents like Cloudera and Datastax are gaining customers and marketshare. The next round of start ups are solving specific problems like simplifying the quering of big data systems or managing security. Interesting companies include Hadapt and Squrrl. Just make sure you perform your due diligence before giving the go ahead.
Besides, you will be looking for a job in the next 24 months. People know CIOs come with no references but people still follow the news. CIOS do not want bad press about patent litigation to their only form of reference. Good luck around the data bucket.
Bill’s Bio :
Bill McGrath recently joined Time Warner Cable as Group Vice President of Finance stationed in the Herndon office. Prior to joining Time Warner Cable, Bill worked for 12 years at AOL where he held a variety of positions in Operations, Finance and General Management. Most recently, Bill was the CIO of AOL and prior to that he was the Senior Vice President of International Operations.
Before AOL, Bill worked at UUnet as the Director of Network Finance. Bill has over 17 years of experience in the internet business.
