Retention and Compliance: The other side of the big data problem

There has been quite a bit written about Big Data including a whole issue with more mainstream articles in The Economist last month.

Interestingly, most of the talk has been focused on how to process and analyze “Big Data for Fast Insights”, which happens to be Aster Data‘s catchy tagline. Also to that end, interest in Hadoop is gaining steam, driven by Mike Olson and his all star team at Cloudera. It’s one of the reasons why I included them in my Cloud Fantasy Investing list together with Appirio, who are fast become the foremost experts in Cloud Computing, with thought leadership blog posts and innovative connectors; Zuora who’s SaaS billing will lead eventually to very Big Data as online subscriptions grow in lock step with the new wave of cloud apps; Voltage Security who’s product hits the sweet spot of privacy issues for Cloud and Big Data as highlighted by this supplemental piece in The Economist on new regulations and privacy.

Then there is quite the chatter on the definition and merits of NoSQL, if you monitor the Twitter hashtag for #noSQL like I do, you’ll see a plethora of tweets around this debate (more on this another time).

In my capacity at RainStor as VP of Product Management, we have been approaching the notion from as my friend Merv Adrian puts it “The Other Side of the Big Data Problem” (Thanks Merv, for the catchy title!)

Our efforts center around the retention and preservation of massive quantities of structured data because, as it turns out, people and more and more government regulations require that this data be retained. Not only retained, but made queriable online which places a tremendous burden of cost and effort on applications and vendors working with Big Data. While the Economist article on regulations focus more on privacy, it also points out the integrity or immutability of stored data is just as critical.

To that end, I’ve authored a post on the RainStor blog called “The Other Side of The Big Data Problem”. Please click below to take a look and weigh in on your thoughts.

http://www.rainstor.com/news-blog/blog/the-other-side-of-the-big-data-problem

P.S. I’m going to be presenting on the topic of “Ideas for Cloud Computing Performance” tonight 3/23 at the Cloud Computing SIG. I hope to see you there.