How a Semantic Layer Saves Time and Money

Solutions Review’s Contributed Content Series is a collection of contributed articles written by thought leaders in enterprise technology. In this feature, AtScale CEO David P. Mariani offers commentary on how a semantic layer can save time and money for enterprise organizations.

After a brutal pandemic and with a recession looming, just about every organization is trying to figure out how to use data to improve its business and the bottom line. There’s no doubt that organizations that leverage data do better than their peers. However, investing in data and analytics can be costly and time-consuming. In this article, we will discuss how introducing a semantic layer into your modern data stack can deliver results faster and cost less.

Recent Research Indicates Cost and Speed Improvements

DBP Institute, a data and analytics research firm, conducted a survey of data leaders to study how their investment in a semantic layer improved results and lowered costs. In the report Business Impact of Using a Semantic Layer for BI & AI, the DBP Institute found that a semantic layer reduced costs by 3.7 times while reducing time to insights by 4.4 times.

The study found that a semantic layer primarily had a positive effect in 3 areas:

1. A reduction in required manual labor
2. A reduction in cloud computing costs
3. An improvement in overall ROI

Let’s take a closer look at each of these impacts.

Benefits of a Semantic Layer

Reducing Required Manual Labor

Delivering a new data asset to the business typically requires several manual steps to be taken by data engineers, including moving and transforming the data (ETL/ELT); and optimizing the data for acceptable query performance.

Writing, testing, and optimizing code is a costly and time-consuming task that’s never quite “done.” According to a Wakefield Research survey, data engineers spent 44 percent of their time maintaining ETL (Extract, Transform, Load) pipelines at an average cost of $520,000 a year.

A semantic layer eliminates most manual ETL steps by employing data virtualization to logically map and transform data using a semantic data model. A virtual data model, authored by a business-oriented data steward, removes most dependencies on data engineers and empowers the business to create their own data products.

Besides removing the data engineering middleman, a semantic layer also automates query tuning by learning query patterns and optimizing queries without the need for data engineering intervention.

By replacing time-consuming ETL/ELT (Extract, Load, Transform) tasks with data virtualization and automating performance tuning, DBP’s research respondents reported that a “typical” project not using a semantic layer would require approximately 903 hours. Performing the same “typical” project using a semantic layer would take only 484 hours, a reduction of 419 hours, or 46 percent.

Reducing Cloud Computing Costs

Most organizations have already migrated or are migrating their analytics infrastructure to the cloud. In doing so, enterprises are implicitly trading fixed capital expenditures (CAPEX) for variable operational expenditures (OPEX), making budgeting challenging and cloud computing costs unpredictable. Simultaneously, the cloud’s infinite elasticity enables more users and more queries, driving more analytics demand and even more variable costs.

A semantic layer can reduce cloud computing costs substantially and make those costs more predictable with automated query optimization. By tracking end-user queries and using AI to cache data and optimize queries autonomously, a semantic layer reduces or eliminates redundant queries and eliminates unnecessary I/O, the primary driver for most cloud data platform computing costs.

Since the semantic layer “understands” the semantics of each query, it can rewrite queries to find the lowest cost approach that answers end-users questions, thereby reducing costs by 3.1x.

Improving Overall Return on Investment

Organizations, on average, spend between 2-6 percent of their total budget for data and analytics, and for good reason. We already know that organizations that use data effectively perform better than their peers.

There’s a nuance to these statistics, though. For data to make a difference in the business, it’s imperative that everyone uses data to make decisions, not just those with advanced data skills. By presenting data in a business-friendly way, a semantic layer makes more data accessible to more users, enabling the organization to make more informed decisions.

By distributing the costs or producing analytics over a larger number of users, organizations can expect to see a 3.9x improvement in their return on investment.

A Key Part of the Modern Data and Analytics Stack

As you can see above, a semantic layer can improve an organization’s ability to deliver new data to the business faster and at scale by reducing manual labor. In addition, by eliminating workload repetition, a semantic layer can significantly reduce data platform computing costs while improving query performance by an order of magnitude for users.

It’s not surprising that a semantic layer is becoming an increasingly popular addition to the modern data and analytics stack.