Don't Get Lost in Big Data
August 1, 2013
The value of data comes only when one can derive actionable insights from it. Accordingly, the primary aim of a big data project should be to convert big data into useful data.
However, it seems like a lot of big data projects lose sight of the this goal. I think the reason for this is that these projects quickly run into engineering challenges.
It’s easy to lose the forest for the trees when the focus shifts from the end goal (delivering value to stakeholders) to acheiving difficult intermediate goals (how are we going to efficiently process this data?)
I had this realization after I recently re-read DJ Patil’s essay Data Jujitsu. The essay is so-named because it talks about how to work around the big engineering challenges that tend to crop up when working with large datasets, without meeting them head-on. I found that it really resonated with some of the findings I’ve had while working on Kwelia.
These types of “Data Jujitsu” strategies are critical for getting a big data project off the ground–not only because they help you avoid doing extra work, but also because they make it easier to stay focused on the real goal of delivering as much value as possible to the stakeholders of the project.
Tagged as: big data, data, startups.
blog comments powered by Disqus