Four Data Management Challenges and Opportunities for 2019
By Ron Agresta
2018 was a landmark year for data management – the EU’s General Data Protection Regulation (GDPR) went into effect, California proposed the most comprehensive piece of US privacy legislation with the California Consumer Privacy Act, and consumers were historically vocal about the need for organizations to better protect and use their data. It’s safe to say that data management – particularly as it relates to data privacy – has become more than just a top priority for enterprises. It has become a business-critical initiative.
Despite how much the data management landscape has evolved in 2018, I’m predicting that 2019 will be even more transformative for four reasons:
Data protection and data transparency will go hand-in-hand
Driven by strong demands from consumers, enterprises will have to accommodate the need for increased and improved data protection across platforms, social media channels, cloud applications and more, while also meeting the increased calls for transparency regarding how data is collected, aggregated and shared. To accomplish these goals in tandem, enterprises will need to adopt enhanced technology solutions that can better protect data, and deliver detailed reports to organizations and their customers about data usage.
Automation, machine learning and AI will abound
More organizations will attempt to use AI and machine learning (ML) techniques to improve data quality and data management processes. Similarly, more companies will look to automation (often supplemented by AI and ML technologies) to help scale data management processes without adding substantially more staff. By using advanced analytics paired with automation, organizations will start to ease the burden on overworked data engineers and data stewards. However, all the issues of automation, ML and AI won’t be solved in 2019. I predict that more organizations will evolve their application of these technologies beyond their current infancy, including deciding whether and where technologies like natural language processing fit into broader enterprise data management and analytics projects, but many will struggle to see meaningful results.
Organizations will have to play data defense and offense
Governance is a growing challenge for organizations as more data moves from on-premises to cloud locations, and as regulations – particularly regarding the use of personal data – become more pervasive. This extra scrutiny on data collection and usage has put businesses on defense. Many companies rely almost exclusively on monetizing data relinquished by users, but regulatory attention is increasing in this area. Organizations should be prepared for more laws centered on consumer data protection, with the associated changes to technology needed to cope not far behind. This means it will be critical for organizations to find the correct balance of “offensive” (being agile and exploratory with data and technology) and “defensive” (governance and control of data) approaches to solving data-centric problems.
Data management + advanced analytics = unlocking big data insight
It has been said that all data is now big data. Companies are increasingly looking at variations of data lake concepts that combine Hadoop Distributed File System infrastructure, event stream processing, relational and non-relational data stores, and other technologies. But making all these work in a performant and auditable fashion can be challenging.
Further, it’s not necessarily data volume in and of itself that poses the biggest challenges. Inexpensive technology to process billions of transactions is commonplace but extracting value and insights from that data (good or bad) can be difficult. Advanced analytics paired with good data management technology can help detect threats and uncover untapped opportunities. In 2019, I predict we will see the use of even more advanced analytic capabilities to solve complex problems that in years past might have taken large teams and years of research.
Over the past year, one common theme has become clear in data management – that data is the new energy source that makes modern businesses run. In order to provide the most value, and really help organizations run as efficiently as possible, 2019 will be the year that enterprises will start to iron out the kinks in how they protect data, meet customer demands, and reap meaningful benefits from emerging technologies.
As the Director of Product Management for all data management offerings at SAS, Ron Agresta works closely with customers, partners and industry analysts to help research and developments teams at SAS develop data quality, data governance, data integration, data virtualization, and big data software and solutions. Ron holds a master’s degree from North Carolina State University and a bachelor’s degree from The Ohio State University.