Our Blog



De-mystifying Big Data

I have heard the term ‘big data’ mentioned at least 5 million times in the last couple of years and to be really honest, at times I have struggled to define this concept perfectly. This article is an attempt to de-mystify it a bit for those who are lost in the sea of data about big data!

So here goes: What is big data about? Well it is quite simple really. It’s about these 3 things:

  • It's about large volumes of data - when I say large volumes, it means things that traditional data processing solutions can't handle 
  • It's about diversity of data - data gathered from social media, sensors, other devices, web applications, consumer behaviours etc. 
  • It's about data streaming - data delivered at different speeds and frequencies

Why, then you would ask, should we embrace big data for your company? Instead of me trying to come up with a sales pitch, let me reference some information from the Bain and Company study. Better still, let me share a cool diagram:

Big Data Diagram

(here is the reference: From Bain & Company report (2014), ‘Big Data” The organizational challenge’, Microsoft Enterprise website http://www.microsoft.com/enterprise/it-trends/big-data/articles/big-data-the-organizational-challenge.aspx)

Then is it really different from BI? Yes and No. Here is my attempt at explaining why:

  • "Big Data analytics" is a BI practice 
  • Big data analytics explores granular details of business operations and customer interactions that seldom find their way into a data warehouse or standard report 
  • Types of implementations
  1. Companies that manage big data in their Enterprise Data Warehouse (EDW) and execute most analytical processing there
  2. Companies that design their EDW for the well understood, auditable and ‘clean data’ that average business report demands and distribute their effort onto secondary analytic platforms
  3. Hybrid approach
  4. Just to put things into perspective, if you are still not convinced about big data, let me share a case study (done by Microsoft with you). Here are my notes on that study: From study conducted for manufacturing sector in US, here are some key facts –
  • $371 Billion potential net value of data divided over 4 years if companies become data smart 
  • Recommendations to perform 4 actions to realise a 60% greater return on their data assets: 
  1. Bring together even as few as 3-4 discrete data sources
  2. Use modern analytics tools to glean insights from data
  3. Surface those insights in a consumable fashion to the right decision makers across the company, and
  4. Ensure that insights from data are shared in a timely manner
  • Key Benefits (for manufacturing)
  1. Employee Productivity – running highly automated manufacturing plants where robots, devises, sensors, software are connected increases productivity
  2. Operational Improvement – Improving operational efficiencies and lowering costs using predictive analytics (condition based maintenance, better energy management, improved asset management)
  3. Product Innovation – using data to drive business growth with smarter product planning and innovation
  4. Customer Facing – Building better customer engagement models and uncovering new service offerings

And here is a pie chart to display these statistics (because I love pie charts): 

Key Benefits


(here is the reference: From <http://blogs.technet.com/b/vertical_industries/archive/2014/05/22/the-371-billion-opportunity-for-data-smart-manufacturers.aspx>)

Posted by: Kartic Kapur , Senior Consultant, Business Productivity Solutions  | 13 July 2015


Top Rated Posts

Blog archive

Stay up to date with all insights from the Empired blog