The True Cost of Big Data

By Oscar Perez and Arthur Mansourian

Between 2012 and 2020, the digital universe will grow by two times every two years.

The recent availability and generation of data has created multiple business opportunities. Decision support has been taken to a new level. The explosion in data has created new platforms to make better business decisions. Our world is a world of Business Intelligence that enable organizations to be proactive and not just react to market changes. It allows organizations to increase the pace of innovation and improve business processes. Big Data creates data driven organizations.

Companies can use current and historical sales data to adjust manufacturing and fulfillment levels. It facilitates “just in time” stock, the right product mix, reduces inventory levels and prevents out of stock situations for consumers, all while margins are improved and the company is able to approach customers in a personalized way and with the trade promotions that the market wants. Insurance companies can use large amounts of transactions to measure risks, conduct what if scenarios, stress tests, and external factors to generate exposure and risk analysis.

The Big Data Opportunity

The opportunity is to leverage all the data assets to be proactive and enhance the business opportunities, capture new markets, customers, and increase margins. The key is to leverage the value of the large amounts of data, identify the ideal business areas of an organization and maximize the use of technology to get the best business insight and edge. Most organizations do not use Business Intelligence and Analytics effectively. The vast majority are not leveraging the data and turning it into “Business Intelligence”. Companies have “Data Warehouses” or a collection of data in one place. Less than 20% of companies have mature Business Intelligence that are used in the execution of the company business strategy. Executives get out of date or incorrect information. However, companies that leverage Business Intelligence and Big Data are more competitive and profitable.

What are the true costs of Big Data? There are multiple studies concluding that an Open Source data warehouse with 30TB of data cost approximately $1,000,000 per year.

While Open Source Big Data offers low cost software, the hardware costs are high. Big Data offers business benefits in tackling Risk Management, Fraud Management, and Customer Insights. All costs of Big Data need to be evaluated including hardware, power/energy, and talent.

The latest big data statistics show some remarkable figures such as that from 2012- 2020, the digital data domain will grow by two times every two years (Source: IDC).  Also, in a recent survey by MicroStrategy, Between 2012 and 2020, the digital universe will grow by two times every two years. (Source: IDC). This shows just how important it is to fully capture the power of the information that is beneath all the data we see around us.  Another interesting fact is that data applications and analytics are estimated to grow from $5.3 billion in 2018 to an astounding $19.4 billion in 2026 (Source: Statista).