I cringe when I see most discussions of “big data.” Color me skeptical.
Recommended reading: Moneyball by Michael Lewis. It’s not really about baseball. It’s about the difference between data and insight.
The problem with Big Data is you can’t see the trees for the forest. In my experience, Big Data focuses on answers; insight focuses on the questions. Big Data complicates; insight simplifies and clarifies.
The best insight comes from little data. Watch individuals (not the whole analytics stream. Talk to your customer-facing staff, ask open-ended questions. Bit by bit, you see the smart questions and develop intelligent hypotheses. Then you can dig into the data to learn the answers.