Sunday, March 31, 2013

Seven Big Data Insights From Facebook, Netflix, And Others


It’s rare to get Big Data experts from companies like Facebook, Netflix and HortonWorks all together for a discussion. But last Wednesday evening at the Microsoft Campus in Mountain View, California, we were able to do just that. 

My guests on the Big Date Date Night panel (so named because it took place the evening before Valentine’s Day) were HortonWorks Director of Data Sciences Ofer Mendelevitch, Netflix Director of Analytics Chris Pouliot, SurveyMonkey Director of Analytics Fedor Dzegilenko, Wix Business Analyst Isaac Buahnick and Facebook Head of Analytics Ken Rudin.

Here are the takeaways:

1. A great customer/user experience is the goal. When it comes to data, it’s easy to lose sight of the end goal. Asked about measuring the ROI of data analytics, one panelist said that the real measure of the impact of data and analytics is the value it brings to the end user. If, as a result of A/B testing, site and product optimization, and an improved understanding of the customer, developers are able to deliver better products to customers, then data analytics has more than paid for its investment. Customer satisfaction may be the best measure of ROI when it comes to data analytics, said Buahnick. 

2. Experience is what you get when you don’t get what you want. Ken Rudin reiterated this classic line. This insight wasn’t Big Data specific, but it was certainly relevant for the hundreds of technologists in the room. Startups are the bread and butter of Silicon Valley and yet, as I wrote about in my book Why Startups Fail: And How Yours Can Succeed, most startups fail. In doing startups, entrepreneurs gather massive amounts of data about what works and what doesn’t. The insights that come as a result are experience.

3. Real time versus right time. Some panelists argued that real-time data analytics are a must. But, of course, there are different levels of real-time. There’s real-time in the NASA sense, that is, down to the millisecond. And then there’s right-time. Right-time is the kind of data analytics that gets you the business insight you need when you need it--and it’s probably what makes the most sense for today’s Big Data users.

4. Separate the signal from the noise. As companies are able to store larger and larger amounts of data, one of the biggest issues is figuring out how to separate the signal from the noise. There are two approaches to dealing with this problem...

5. The carbon-based approach and the computer-based approach. Both of these ways of working with Big Data have their place. There are those cases where data is moving too quickly for humans to analyze it--fraud detection for credit card transactions or targeted online advertising, for example. But the algorithms and the analysis that makes such computer-based systems possible are still very much carbon-based, that is, developed by humans. People and computers both have a role to play when it comes to Big Data.

6. Use products appropriately. When it comes to Big Data, there’s lots of talk about Hadoop and MapReduce. But it’s important to choose the right technology for the right business use case. Choose relational databases for problems that lend themselves well to structured data storage and Hadoop and related technologies for unstructured data and for bringing massive amounts of data together.

7. Be curious while being business oriented. At the end of our discussion, I asked the panelists what they were looking for in potential hires for their teams. Almost universally, the panelists focused on the importance of not just being good with data but of being curious while being business oriented. 

No comments:

Post a Comment