How to Refocus the Purpose, Vision, and Mission of Your Data Warehouse

This is part of Solutions Review’s Premium Content Series, a collection of contributed columns written by industry experts in maturing software categories. In this submission, Actian Director Teresa Wingfield offers her take on how to refocus the purpose, vision, and mission of your data warehouse.

Traditionally, data warehouse users have made up only a small percentage of an organization’s total employees, primarily consisting of analysts, data scientists, and engineers, and potentially others interested in complex analytics. This subset of employees has naturally always been the group driving the purpose, vision, and mission of a data warehouse. However, data is becoming increasingly needed by employees across the business, including non-technical users. Enhanced flexibility and functionality are attracting a broader base of business users looking to run queries and perform analytics to inform various operational decisions across the business.

Organizations are starting to reconsider the vision and mission of their data warehouse as they now need to satisfy an ever-expanding user base with different requirements and uses of data. It is time to start asking questions on what purpose the data platform should serve: What should it deliver? What is its mission and how will it achieve the vision? Ideally, your warehouse will deliver data that is clean, consistent, and structured so that the business has the best data available for decision-making. Many aspects of the data warehouse’s purpose and vision will continue to apply to the data platform, while at the same time they will expand to encompass more tactical, strategic, and operational opportunities. Moving forward, the mission must encompass a focus on data democratization, which will require a much different approach than was required of legacy data warehouse architectures.

Finding a New Purpose

Historically, the data warehouse has served as a central repository of historical data, helping users analyze trends over different time periods. Data was consolidated from many sources in order to avoid impacting the performance of operations systems, optimize query performance, improve data quality, and provide a business representation of data that made it easier for users to access information.

These historical analytics capabilities still have immense value, but capturing and understanding critical events in real-time is growing in importance as it offers improved operational decision-making and faster response times. While a complementary operational data store (ODS), with its snapshot of current transactional data, has offered additional support for operational decisions, it still does not provide the real-time access needed for decisions that must be made within minutes, maybe even seconds. For example, businesses need fast analytics to inform credit and loan approvals, personalized e-commerce, investment portfolio decisions, and supply chain optimization. Data fabrics and data meshes are emerging data architecture designs that can be used through built-in data warehouse integrations to make data more accessible, available, discoverable, and interoperable for real-time data ingestion than what a singularly focused semantic layer can offer.

Creating a New Vision

Although data warehouse vision statements may already touch on making information accessible and easy to use, they are typically still framed with the rarified audience of data scientists, data engineers, and sophisticated business analysts in mind. Recently, there has been an emergence of new data platform technologies that aim to help make data easier to access and use, often referred to as a data platform or modern data warehouse. These technologies leverage the cloud for real-time speed, but also aim to satisfy the needs of less technical users. The vision of a modern data platform must embrace

data democratization and emphasize universal access to data for everyone within an organization, while still maintaining a single source of enterprise data. This will apply to more than simply data in a data warehouse – it will also include disparate and diverse data from various sources throughout a company, such as SaaS applications. Data democratization will broaden the role of the data platform from historic and tactical to strategic and operational, which means the vision must consider greater opportunities to drive operational efficiencies and generate new revenue for the whole of the organization.

Achieving a New Mission

To achieve your new purpose and vision, your new mission must be specific. As you grow to support an ever-increasing number of users and their various requirements—support for machine learning, AI, streaming analytics, along with other resource-intensive activities—the only way to move forward will be with a cloud-native data platform that leverages Kubernetes and containers. Decision intelligence workloads can quickly strain legacy architectures, so containers are needed to enable elasticity to meet demand. Kubernetes orchestration is also key to automate the deployment, provisioning, networking, availability, scaling, and lifecycle management of the containers. Discovering your company’s specific needs will allow you to determine your strategy to better meet the requirements of your growing user base and ultimately give you the ability to achieve the new mission, purpose, and vision for your data warehouse.