{"id":5392,"date":"2023-06-30T13:36:51","date_gmt":"2023-06-30T17:36:51","guid":{"rendered":"https:\/\/solutionsreview.com\/data-integration\/?p=5392"},"modified":"2023-08-11T11:31:29","modified_gmt":"2023-08-11T15:31:29","slug":"the-top-data-mesh-challenges-how-to-solve-them","status":"publish","type":"post","link":"https:\/\/solutionsreview.com\/data-integration\/the-top-data-mesh-challenges-how-to-solve-them\/","title":{"rendered":"The Top 3 Data Mesh Challenges & How to Solve Them"},"content":{"rendered":"

\"Data<\/p>\n

Solutions Review\u2019s Contributed Content Series<\/a><\/span> is a collection of contributed articles written by thought leaders in enterprise business software. In this feature, Ascend.io<\/a>‘s Jon Osborn offers a brief on the top data mesh challenges and how to solve them.<\/strong><\/i><\/p>\n

If you work with data, you\u2019ll have come across the term data mesh by now. This decentralized but interconnected approach to structuring data has become increasingly popular since the term was coined by Zhamak Dehghani<\/a> 4 years ago.<\/p>\n

However, while data meshes have significant advantages for scaling up your data operations, the approach comes with its fair share of challenges: Michele Goetz, VP and Principal Analyst at Forrester, calls it the \u201cdata mesh blind side<\/a>.\u201d<\/p>\n

In this article, we\u2019ll break down the main challenges of moving to a data mesh architecture and briefly explore how to tackle them.<\/p>\n

\t\t\t

\"Download<\/a><\/p>\n<\/div>\n\t\t<\/div><\/div>\n

Data Mesh Challenges<\/strong><\/h2>\n

Data Mesh: 101\u00a0<\/strong><\/h3>\n

First, a quick primer on the data mesh concept. Data mesh is a way of thinking about and organizing your data to move data ownership and accountability closer to the end users.<\/p>\n

The core tenet of the data mesh<\/a> is to distribute responsibility and governance of your data across different business \u201cdomains\u201d. This is the opposite of having a single, monolithic data architecture managed by a centralized data team.<\/p>\n

These domains are loosely coupled teams that own and manage data and pipelines in their business unit. For example, human resources data might be owned and managed by the human resources domain on a particular data platform, while the sales data domain is managed by another team on a different platform.<\/p>\n

The term \u201cmesh\u201d comes from the fact that all these data domains remain interconnected despite potentially running on different technology stacks. There\u2019s no return to the old days of siloed data warehouses. Rather, the goal is to move ownership and responsibility of the data closer to the subject matter experts who understand it, while maintaining a unified data catalog of data products that can be referenced by any other builders in the organization.<\/p>\n

The end result? A more scalable, secure, and speedy distributed architecture that helps you build and maintain interdependent datasets at scale.<\/p>\n

The 3 Biggest Data Mesh Challenges\u00a0<\/strong><\/h3>\n

There\u2019s no such thing as frictionless business transformation (no matter what the vendors tell you!). Data meshes are no exception. If you\u2019re used to operating with a centralized data platform, there will be some challenges when moving to a data mesh.<\/p>\n

Here are a few of the most common issues with data meshes:<\/p>\n

Challenge#1: Securing Stakeholder Buy-In<\/h3>\n

Or, to put it more bluntly\u2014office politics. As with any major change to the way you do business, it\u2019s only going to work if everyone buys-in. Here are a few of the hurdles you\u2019ll need to jump over:<\/p>\n

    \n
  1. Moving to a data mesh structure will require distributing ownership of the data to the business domains\u2014and that means giving more work to line of business workers that may not want it!<\/li>\n
  2. You may also find you get pushback from your central data team, especially if they feel they are losing their ability to govern and secure the data. Michael Ryan, Managing Principal Consultant and Head of Architecture at telecommunications software provider AmDocs<\/a>, warns that if your data science team \u201cdon\u2019t feel valuable, or if they feel their jobs are threatened by a data mesh, they will act against it – even though the [centralized and mesh] architectures are complimentary.\u201d<\/li>\n
  3. You need to decide which domain owns what data. That isn\u2019t always obvious\u2014competing business priorities can make it difficult to know what all the different domains should be and (even more sensitive) who should manage them.<\/li>\n
  4. Data mesh structures rely on a higher level of self-service by data users. Not everyone will be a fan of the learning curve involved\u2014especially if your self-service layer is hard for non-technical people to use. This is particularly likely to create issues if the \u201cnon-technical people\u201d hold senior roles, and feel like they\u2019re being asked to work harder to find the information they need.<\/li>\n<\/ol>\n

    The Solution\u2014A Carrot (Not Stick) Approach<\/strong><\/h3>\n

    Matteo Vasirani<\/a>, a Senior Manager of Data Science at developer platform Github<\/a>, suggests that the key for securing buy-in from the various stakeholders is to \u201cshow them the carrot\u201d\u2014the positive outcomes that will result from moving to a mesh framework.<\/p>\n

    Matteo found that incentivizing data users with \u201cwhat\u2019s in it for them\u201d is more effective and realistic than simply attempting to mandate the move from the top (the \u201cstick\u201d).<\/p>\n

    Some of the key benefits to moving to a data mesh architecture<\/a> that are likely to appeal to end users and business leaders alike:<\/p>\n