Mining for Diamonds in a Coal Mine of Data
Why NVMe is not enough.
In a world where data has been compared to oil or gold, one might expect that the more you have of it, the better. That is where the analogy is faulty, because if you think about it, we already have plenty of data and it is not more data we are looking for — instead we are searching for the precious data, much like mining for diamonds in the coal. Therefore, one might even compare data to diamonds — as today, organizations are struggling to extract the “diamonds” of data within the vast data centers that house petabytes of erroneous rocks of invaluable material.
Data lives among us in abundance and by 2025 IDC predicts we will churn out 175 zettabytes of data — a number much like our national debt ($22 trillion) that is hard to even fathom. As we connect more IoT devices like video surveillance cameras and autonomous cars or “anything” that can churn out data 24/7, they can cause bottlenecks and delays for organizations. These huge data sets are where we are trying to extract only diamonds (10%) out of the coal (90%) of the data created each day, and is impossible to mine with current technology.