According to a new study commissioned by data management provider Panoply, 81 percent of users say their on-prem data warehouse solutions are too complex, making it a key big data pain point. However, with little evidence to suggest that widespread progress is being made toward those wishing to transition their data management practices to the cloud, one has to wonder what the immediate future looks like for. Panoply polled more than 800 Amazon re:Invent attendees for its third annual Data Warehouse Trend Report, with respondents coming from a variety of roles and ways in which they use data warehousing.
Users aren’t just frustrated with their on-prem environments, as more than half of those polled characterize their data warehouse solutions as either difficult or very difficult to use. This includes cloud products as well, and Panoply notes that responses included nearly every major data warehouse brand in the marketplace. They picked on Redshift a bit more obviously than the rest, with 62 percent of responses taking issue with the complexity of the offering. The report correctly notes that Redshift remains the most adopted data warehouse out there – with a nearly 40 percent market share.
The biggest performance issue plaguing data warehouses right now is hanging or slow query speeds, with more than half of the 800 respondents citing varying degrees of frustration. It’s likely that the majority of these users are utilizing outdated technologies, and as the need for more complex queries, growing data volumes and deeper analysis continues to grow, the problem will likely fester. The report also outlines other factors likely to play a role, including integration of multiple data sources, lackluster database normalization and shrinking human and technical bandwidth.
As organizations continue relying on big data to bring them the deep insight they seek, these “feed the beast” processes (as Panoply so eloquently puts it) are only going to increase. This will, in turn, create a need for streamlining and automating many common data management chores in order for organizations to keep pace.
We encourage you to read the report in its entirety.
- Data Management Officer Responsibilities Introduction - February 3, 2023
- A Data Governance Job Description by Solutions Review - February 3, 2023
- What’s Changed: 2022 Gartner Magic Quadrant for Cloud Database Management Systems - February 3, 2023