Expert: Are Cloud Data Warehousing Costs Out of Control?

This is part of Solutions Review’s Premium Content Series, a collection of contributed columns written by industry experts in maturing software categories. In this submission, Yellowbrick Data CTO Mark Cusack asks the question: “Are cloud data warehousing costs out of control?” and offers a guide for organizations.

Cloud data warehousing is fast becoming the largest line item on CFO budgets. For many enterprises, the move to the cloud hasn’t simplified their data and analytics strategy, it has added complexity – particularly for those companies who still retain data and workloads on-premises. As data management becomes more complex, organizations are struggling to keep pace with ballooning expenses and swelling data volumes.

The cloud has enabled the capture and analysis of nearly incomprehensible amounts of data and generated untold numbers of important insights. It’s one of the most important technological innovations the business world has ever seen but it has also created numerous challenges. IT teams, especially those in highly regulated industries, struggle to keep up with the demand to use cloud data management services due to security and regulatory concerns, and this prevents business users from comprehending and utilizing the full value of their data.

Business leaders and IT pros can take steps to avoid getting overwhelmed by the cost and complexity of taking data warehousing to the cloud. Let’s explore some ways to overcome the challenges.

Pain Points Are Everywhere

When it comes to moving data to the cloud, businesses are forced to confront some costly pain points. In 2021, cloud spending grew to $383.6 billion up from $307.7 billion in 2020, according to an IDC report. While cloud use has skyrocketed in recent years, businesses must understand the financials of their cloud computing resources and budget properly. This is especially critical during a period of rising inflation and growing fears about a recession. Free-spending attitudes that might have gone unchecked over the past two years are quickly coming to an end.

A second pain point is that legacy solutions and the limited flexibility offered by current data warehousing solutions are preventing businesses in highly regulated industries from consolidating and migrating workloads into the cloud. Consequently, siloed data persists in these enterprises. In recent years, managers have recognized the inefficiency and opportunity cost that results from isolating information, and a growing number seek to supercharge data mobility and flexibility by integrating data across the organization.

By providing wide access to information, a company increases the likelihood that it will be available wherever and whenever it is needed most. For years, businesses have been working to break down the so-called “80/20 data dilemma” – the observation that data scientists typically spend 80 percent of their time finding, cleaning, and organizing data, leaving only 20 percent of the time to analyze data.

Most siloed data is typically on premises, stored within legacy data warehouses. The cost, security, and complexity of rearchitecting to eliminate these silos has hindered cloud adoption among highly regulated industries, such as the financial services and banking industries. Today, U.S. banks’ overall use of the cloud is just 12%, according to Accenture, with adoption expected to reach only 25% by 2026.

The Changing Tides of Data Warehousing

As IT spending shifts, business leaders are looking to make the best use of their existing investments and adopt practices, and technologies, that ensure data across the enterprise remains accessible to business decision makers.

To accomplish that, organizations must carefully choose their battles and managers must determine which data warehouse solution is best for their companies. Many organizations, if they’re in a highly regulated industry, opt for a hybrid-cloud approach, one that ensures the data can be accessed by the right people — anytime and anywhere. This is especially important as legacy technologies, such as SAP IQ and IBM Netezza, have begun to phase out, and most of today’s solutions don’t scale well enough to replace these, or offer multi-cloud and on-prem capabilities.

Second, leaders must establish a healthy data governance culture within their organizations. Everyone must buy into the strategy of data deployment and commit to preventing the creation of new data silos.

Third, always test out competing data warehouse solutions on your workloads at scale through a proof-of-concept exercise. Build out a multi-year TCO projection to avoid any unpleasant financial surprises. Once you’ve selected a vendor, start small by moving a few use cases and workloads to the cloud, and prove the positive ROI.

In short, do your homework! If you haven’t already started to manage your data at least partially in the cloud, then it is most definitely time to get started.