AI Will Bring Light to Dark Data

In today’s digital age, we’re amassing data at an unprecedented rate. Remarkably, 90 percent of all the world’s data has been generated in just the past two years. This surge in data acquisition has redefined exponential growth for both geospatial professionals and businesses. Every second, quintillions (equivalent to a billion billions) of data points are being generated and captured by countless people and machines.

Dark Data

Data that organizations are collecting but not using is called ‘dark data’ – and it constitutes a staggering 90 percent of the data enterprises gather. This might include data gathered from routine operations such as infrastructure monitoring, asset management surveys, and environmental assessments. Yet, this dark data often goes unnoticed. Many experts liken it to the submerged portion of an iceberg, with organizations typically only engaging with and drawing insights from the tip that’s visible above the water.

Given the sheer volume of data, it’s becoming overwhelming for organizations to manage – let alone find meaningful insights from it. That could all change with the integration of generative AI. 

AI Will Bring Light to Dark Data

Turning on the Lights

With the introduction of ChatGPT earlier this year, organizations are moving quickly to integrate new technologies into their software. With AI and a modern data integration approach, companies hold the potential to find powerful insights within their dark data icebergs. 

It’s estimated that around 75 percent of AI use cases are spread across four key areas: customer service, marketing and sales, software engineering and research and development. We’ve only just begun to see the benefits of AI in data integration and the economy as a whole. McKinsey’s latest research suggests that it will add more than 2.6 trillion dollars in productivity to the economy, having the greatest impact on software development. AI offers an opportunity for SaaS companies to think about the impact of the data they are collecting to better serve their users. 

At Safe Software, we’ve started integrating AI into a wide range of activities, starting with our localization work. We are also encouraging our engineers to experiment with AI tools like ChatGPT. In the future, we could see AI filtering and aggregating huge volumes of data to provide value through more actionable and analyzable datasets, particularly the mountain of usage statistics we’ve gathered over the past decade, which overwhelmingly has been our largest “dark data” repository.

The aspect that excites me most about dark data and AI is the potential to find patterns in datasets that would typically go ignored. For enterprises, this could include identifying outliers that foretell important risks or opportunities, assessing equipment failure potential, targeting potential customers and markets, and preparing training data for machine learning and artificial intelligence use. In our own usage statistics, this could highlight combinations of operations that we might want to optimize, which are too difficult to otherwise discern.

Modern integration approaches can further extend the utility of otherwise dark data by joining it to other datasets, with a result that the whole is far more valuable than the sum of the parts.

The Future of Data 

It’s an exciting time in the data economy as new technologies like AI could put a spotlight on previously unobservable dark data. With Generative AI, SaaS companies are positioned to break down those quintillions of data points to draw better conclusions than previously possible. 

Ultimately, dark data has the potential to dramatically change the way we look at information, and shine a light on the underside of the data iceberg, resulting in better products and outcomes for all involved.