The Dark Side Of Data-Anshuman Rai

Organisations today accumulate huge amounts of data from varied sources. The data deluge could come from financial statements, log files (systems, servers, architecture etc), email correspondences, customer call records, notes, presentations or old documents, surveillance video footage, previous employee data, raw survey data, and geolocation data.

However, most of the data-driven initiatives are limited only to structured data, leaving out the unstructured or the ‘dark data.’ This dark data remains in the shadows without ever being utilised. Compounding the situation is the fact that the size of dark data is continuously increasing.

According to IDC, a mind-boggling 90 per cent of all digital data is unstructured, and is locked away in different formats, in varied locations, and in diverse data stores. Besides, as per estimated by Datamation, this unstructured data is witnessing an annual growth rate of 55-65 per cent.

While data can be harnessed for driving business revenue, leaving it untapped is leading many enterprises to miss out on maximising this ‘new oil.’ It’s also not just about leveraging data, dark or unstructured data without control and visibility can pose a major bottleneck for business growth.

Let’s explore what challenges does dark data throw up, and how can enterprises mitigate them by adopting effective and intelligent data management strategies.

Impact on Cost, Compliance, Cybersecurity

Business leaders today have little or no visibility into what data is being created and how it is being stored. When it comes to information lifecycle governance, IT leaders make use of cold storage tape vaults because of the fear of losing out on something of value.

Assuming storage makes up 40 per cent of the IT hardware budget and 69 per cent of the stored data has absolutely no value to an enterprise, more than a quarter of the annual technology hardware budget goes to waste. Long-term retention of dark data, therefore, incurs cost. In the present era of ‘pay-as-you-go’ storage, monthly bills gradually rise unnoticed before going out of control.

Dark data could also contain personal information, thereby subjecting it to applicable privacy laws. Ignoring regulatory compliance of dark data can lead to privacy violations and reputational damage.

The other challenge of harboring dark data is its fallout on cybersecurity. Collecting and storing data without acting on it could pose a serious problem for an enterprise. Not looking at data would amount to not knowing if it is being attacked by hackers. There is, therefore, a high-risk factor to dark data that lies dormant.

Tackling Dark Data

To find and manage dark data, enterprises need unparalleled indexing of all workloads to gain visibility into their data, understand the associated risks and take corrective action.

Not only should the data be indexed, but it also must be examined and placed in indices with keywords created. This would ensure the file metadata gets a rich set of metadata, which can be leveraged for effective decision-making. For instance, keywords can reveal that a five-year-old email that has not been retrieved since created should be retained for e-discovery purposes and not be deleted.

Technology decision-makers need to deploy a solution that enables enterprise-wide search and efficient discovery of information from a single console and virtual repository, irrespective of where it lies in the enterprise. This would ensure access to eDiscovery and compliance information in a fast and simple manner.

Such an approach would also reduce cybersecurity risk because only what has compliance, evidentiary, or business value is retained while the rest is deleted.

To get on top of dark data, organisations should also eliminate manual processes for search and retention of corporate information. Besides, they should keep archival data in cloud storage with self-service access for competitive advantage.  

Transitioning from Dark to Light

For enterprises looking to be data ready, one of the tenets is to get ahead of dark data. Left alone, dark data can cost too much money without generating any value. However, if it is managed effectively, dark data that has ongoing value can be utilised for new uses, while that which has no ongoing value can be defensively disposed of.

Quality risk management hinges on future proofing and forward looking, which in turn calls for an organisation’s ability to update and address its risk profiles on the fly. This can be achieved only through intelligent and effective data management strategies.

The attributes of a cutting-edge solution for tackling dark data include speed, accessibility, and scale. Utilising state-of-the-art solution with data insights capabilities, file storage optimisation, data protection, and sensitive data governance, can go a long way towards solving the dark data problem with minimal cost and complexity.

Managing dark data is a journey where enterprises will keep adapting but it’s time, they looked at it as a business opportunity.

(About the author: Anshuman Rai is Area Vice President, India & South Asia at Commvault)

---