{"id":561,"date":"2024-01-01T14:52:10","date_gmt":"2024-01-01T14:52:10","guid":{"rendered":"https:\/\/solutionsreview.com\/expert\/?p=561"},"modified":"2024-02-02T14:35:02","modified_gmt":"2024-02-02T14:35:02","slug":"data-mesh-meets-universal-authorization","status":"publish","type":"post","link":"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/","title":{"rendered":"Data Mesh Meets Universal Authorization"},"content":{"rendered":"<p id=\"b69a\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data mesh has attracted intense attention because of its promise to deliver faster analytics in an agile and decentralized manner. It puts the responsibility for data quality and curation on data producers and owners within business \u201cdomains,\u201d who understand the data the best. They package the data for consumption as a \u201cproduct.\u201d The two major goals of data mesh are: 1. Remove bottlenecks arising from centralized data engineering teams, and 2. Provide authorized data consumers with read-only self-service access to curated data products.<\/p>\n<p id=\"f76e\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data mesh is a concept, and not an implementation methodology. It does not prescribe any technologies or standards to implement it. A data mesh may involve domain-level data warehouses or data lakes, but those are orthogonal to its principles.<\/p>\n<p id=\"f3fc\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">For a data mesh to deliver its promise, resolve the following issues:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"ed33\" class=\"is it gx hw b hx hy ib ic if iu ij iv in iw ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Data discovery mechanism<\/li>\n<li id=\"46ec\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Data quality and trustworthiness<\/li>\n<li id=\"6884\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Standardization of common infrastructure and reuse of assets<\/li>\n<li id=\"88b5\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Interoperability of domains<\/li>\n<li id=\"bf60\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Self service architecture<\/li>\n<li id=\"d065\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Observability, governance, and security<\/li>\n<\/ul>\n<p id=\"0618\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Due to the lack of standard reference architectures, various organizations are developing custom solutions to enable data mesh. This paper focuses on the need to ensure a cohesive and scalable data access governance and authorization mechanism. It proposes a unified governance approach to standardize and simplify data access governance based on a consistent set of policies. The domain-specific policies link the data consumer identities and their roles to make cross-domain data available to every authorized user.<\/p>\n<h1 id=\"2f75\" class=\"jg jh gx be ji jj jk jl jm jn jo jp jq jr js jt ju jv jw jx jy jz ka kb kc kd bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Key findings<\/h1>\n<p id=\"3310\" class=\"pw-post-body-paragraph hu hv gx hw b hx ke hz ia ib kf id ie if kg ih ii ij kh il im in ki ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data value chain leaders responsible for evaluating and implementing data mesh should:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"98b1\" class=\"is it gx hw b hx hy ib ic if iu ij iv in iw ir ix iy iz ja bj\" data-selectable-paragraph=\"\"><strong class=\"hw gy\">Deploy a collaborative governance platform<\/strong>. Engage all data stakeholders, such that the business, infosec, and data privacy teams work with the data and IT teams to deliver data to business without compromising data security mandates or data privacy regulations.<\/li>\n<li id=\"0674\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\"><strong class=\"hw gy\">Design a common data access governance layer.<\/strong>\u00a0Ensure data consumers have consistent access to common data products in different domains through centralized policy management. During the data mesh planning stage, perform proof of concepts of products that provide data governance capabilities. It should not be an afterthought.<\/li>\n<li id=\"0c4b\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\"><strong class=\"hw gy\">Implement the universal authorization layer.<\/strong>\u00a0Permit consumers to search and analyze domain data without performance and scale bottlenecks. The universal authorization layer is typically a best of breed product deployed in a modular and composable architecture, capable of supporting multiple data storage technologies in a hybrid multi-cloud environment.<\/li>\n<\/ul>\n<p id=\"7d8b\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Before delving into the security and governance aspects, let\u2019s take a deeper dive into the concept of a data mesh.<\/p>\n<h1 id=\"f1e8\" class=\"jg jh gx be ji jj jk jl jm jn jo jp jq jr js jt ju jv jw jx jy jz ka kb kc kd bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data mesh: a brief primer<\/h1>\n<p id=\"141b\" class=\"pw-post-body-paragraph hu hv gx hw b hx ke hz ia ib kf id ie if kg ih ii ij kh il im in ki ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Several excellent papers have been written on data mesh by its originator,\u00a0<a class=\"af kj external\" href=\"https:\/\/www.thoughtworks.com\/en-us\/profiles\/z\/zhamak-dehghani\" target=\"_blank\" rel=\"noopener ugc nofollow\">Zhamak Deghani<\/a>\u00a0and Thoughtworks. This section only provides a brief high-level overview of its concept and principles.<\/p>\n<p id=\"12d3\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data professionals have been on a quest to provide business users with a common representation of data through various storage architectures that integrate internal and external data from different sources. This has taken the shape of enterprise data warehouses, data lakes, lake houses, data virtualization, data sharing, and data fabrics.<\/p>\n<p id=\"b19f\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">These approaches rely on varying levels of data integration technologies. They also rely on centralized data engineering teams which build the data pipelines to ingest, transform, and curate data. One issue with this approach is that the data engineering team can become a bottleneck. This model leads to a disconnect between data producers and data consumers. Data producers often do not know how their data is being used, while the data consumers are unaware of the data sources. They both rely on the data engineer to be the bridge.<\/p>\n<p id=\"6c87\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data mesh has the potential to alleviate the problem of replicating data ad nauseam, and creating multiple silos. A decentralized architecture to improve agility and time to insights is not a new concept. Some of the previous approaches have included data marts and data virtualization techniques. The data mesh approach introduces a set of principles that put higher accountability and ownership of data on the domains where data is produced, and de-emphasizes the current trend of creating a centralized analytical data store.<\/p>\n<p id=\"9635\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Figure 1 shows the high-level overview of data mesh.<\/p>\n<figure class=\"kl km kn ko el kp dz ea paragraph-image\" style=\"text-align: justify;\">\n<div class=\"kq kr dj ks bg kt\" role=\"button\">\n<div class=\"dz ea kk\"><img loading=\"lazy\" decoding=\"async\" class=\"bg ku kv c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/max\/700\/0*DDzhShVMEP4ev0XI\" alt=\"\" width=\"700\" height=\"269\" \/><\/div>\n<\/div>\n<\/figure>\n<p id=\"696e\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\"><em class=\"kw\">Figure 1. The four principles of data mesh.<\/em><\/p>\n<p id=\"655b\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data mesh principles are designed to decrease the impedance mismatch between data producers and consumers. If implemented well, data mesh ensures that the data consumers don\u2019t have to guess whether they can trust data or wrangle it, as the data producers are accountable for its quality and accessibility.<\/p>\n<p id=\"71ef\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Domain driven design has been applied to software development for decades for building software applications. Data mesh applies the same principles to building data-intensive applications. Each domain is responsible for delivering data as a product. The domains ensure its quality and accessibility.<\/p>\n<p id=\"689d\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data products comprise data elements, associated technical schema metadata, business glossary terms, data pipeline code used to generate it, documentation, usage examples, and even notebooks. Data product documentation shows the level of granularity of datasets, e.g. raw data versus aggregated, schema, and mappings and transformations. Finally, data products have SQL and programmatic notebook interfaces. Data producers across domains collaborate to standardize naming conventions for common data elements.<\/p>\n<p id=\"de28\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">As with any new concept, data mesh comes with some unknowns. It is a top-down initiative that needs an envisioned data culture across the organization and stakeholder buy-in. Its challenges start with foundational questions: what is it, and why is it better from the past initiatives? All the parties involved \u2014 organizations\u2019 stakeholders and data infrastructure vendors \u2014 need to align. For example, business stakeholders need to clearly define what is the definition of a domain within their organization. This is an iterative process that goes through a phase of refinement. Similarly, IT stakeholders need to ensure that the software components making up the mesh will have required APIs.<\/p>\n<p id=\"146f\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Domain-level analytics work great until data consumers need cross-domain data to build dashboards and reports. This necessitates a well-designed data access layer.<\/p>\n<h1 id=\"492f\" class=\"jg jh gx be ji jj jk jl jm jn jo jp jq jr js jt ju jv jw jx jy jz ka kb kc kd bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data access layer<\/h1>\n<p id=\"d68c\" class=\"pw-post-body-paragraph hu hv gx hw b hx ke hz ia ib kf id ie if kg ih ii ij kh il im in ki ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Interoperability of data is one of the biggest challenges in meeting data mesh\u2019s goal of sharing business context in a self-service manner to maximize its usability. Let\u2019s take an example of a retail organization that has customers\u2019 orders data in the sales domain. A business analyst uses the data to run customer journey analytics models. However, to perform customer churn analytics, the analyst needs to tap into the customer success domain that tracks support tickets, surveys, and social media posts, etc.<\/p>\n<p id=\"8cd4\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data mesh proposes a combination of distributed stewardship with logically and physically interconnected data. This interconnected data layer should provide a lean, and cost-efficient analytical environment, with the following benefits:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"2245\" class=\"is it gx hw b hx hy ib ic if iu ij iv in iw ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Share common standards<\/li>\n<li id=\"5d5e\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Reuse common resources<\/li>\n<li id=\"3d78\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Reduced integration overhead<\/li>\n<li id=\"8fcb\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Develop deep skills in core technologies rather than every department having its own stack.<\/li>\n<\/ul>\n<p id=\"559d\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">However, the challenge is in building a watertight security architecture for data mesh. This is how Thoughtworks describes the fourth principle of \u201cfederated computational governance\u201d:<\/p>\n<p id=\"a072\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\"><em class=\"kw\">The last principle addresses the question around, \u201cHow do I still assure that these different data products are interoperable, are secure, respecting privacy, now in a decentralized fashion?\u201d<\/em><\/p>\n<p id=\"2932\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\"><em class=\"kw\">\u2026all of those policies that need to be respected by these data products, such as privacy, such as confidentiality, can we encode these policies as computational, executable units and then code them in everyday products so that we get automation, we get governance through automation?<\/em><\/p>\n<p id=\"a8bc\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The data mesh approach to help data producers quickly deliver high-quality data to the business needs to be augmented by a universal authorization layer that knocks down the data silos and automatically makes the necessary data available to the consumers.<\/p>\n<p class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\"><strong>Decentralized domain data stores need a centralized data access and governance layer to make data mesh work at scale<\/strong><\/p>\n<p id=\"56a6\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Secure and authorized data access has always been a critical requirement of every application. Data mesh is not an exception. To access data from the sources of truth, the following initiatives are a must:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"68ab\" class=\"is it gx hw b hx hy ib ic if iu ij iv in iw ir ix iy iz ja bj\" data-selectable-paragraph=\"\"><strong class=\"hw gy\">Data discovery<\/strong><\/li>\n<\/ul>\n<p id=\"55bb\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Domain data has to be discoverable and accessible. Domains may have their own individual data catalogs that link business metadata to the domain\u2019s technical metadata. In the data mesh parlance, the catalog also includes data products.<\/p>\n<p id=\"8764\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">In addition, a\u00a0<em class=\"kw\">catalog of catalogs<\/em>\u00a0is needed to provide a cross-functional semantic layer for the common and shareable data products from different domains. This uber catalog has an additional attribute \u2014 domain id. In summary, a data mesh needs a federated data catalog architecture.<\/p>\n<p id=\"dd63\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data catalog vendors, such as Alation, BigID, Collibra and Informatica provide centralized data curation and governance capabilities.<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"a5d6\" class=\"is it gx hw b hx hy ib ic if iu ij iv in iw ir ix iy iz ja bj\" data-selectable-paragraph=\"\"><strong class=\"hw gy\">Data access governance<\/strong><\/li>\n<\/ul>\n<p id=\"0d7b\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Role based access control (RBAC) with attribute based access control (ABAC) should be used to reduce the complexity of access control and to apply consistent access policies at scale. User identities are tied to roles, and roles are tied to policies. The policies are further attached to the underlying data elements, their attributes, and even user attributes, such as the data consumer\u2019s geographical location or job title. For example, if a tag for a data element says sensitive or confidential, then the corresponding policy declares which roles have access to that element. Using the combination of tags and policies, fine-grained data access governance can be performed.<\/p>\n<p id=\"353c\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">But the decentralized nature of data mesh exacerbates the data access issues. First, the catalog needs to extend policy based access control to data products. Second, it has to be aware of the domain location of the product.<\/p>\n<p id=\"5ff7\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data access governance products, such as Immuta, Okera and Privacera perform universal data authorization.<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"22db\" class=\"is it gx hw b hx hy ib ic if iu ij iv in iw ir ix iy iz ja bj\" data-selectable-paragraph=\"\"><strong class=\"hw gy\">Data observability<\/strong><\/li>\n<\/ul>\n<p id=\"509f\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The final piece in operationalizing a data mesh-based analytics architecture is providing transparency to the internal state of data as it moves from the point of origin to the point of consumption. Data observability products should provide a multi-dimensional view of data, including performance, quality, and its impact on the other components of the stack. Its overall goal is to see how well data supports business requirements and objectives.<\/p>\n<p id=\"50c2\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Unlike data catalogs, data observability is a newer space, which has attracted many new entrants, such as Acceldata, Bigeye and Monte Carlo.<\/p>\n<p id=\"13ba\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data access design should be for multi-persona and ease of use. As data mesh is an approach and not a standard, it doesn\u2019t prescribe the \u201chows.\u201d This leaves it open to interpretation.<\/p>\n<h1 id=\"923b\" class=\"jg jh gx be ji jj jk jl jm jn jo jp jq jr js jt ju jv jw jx jy jz ka kb kc kd bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Universal data authorization<\/h1>\n<p id=\"861d\" class=\"pw-post-body-paragraph hu hv gx hw b hx ke hz ia ib kf id ie if kg ih ii ij kh il im in ki ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Imagine an example where a customer\u2019s name and address are in different domains. This is common in financial services with domains, such as retail banking, wholesale, business banking, lending and leasing, and capital markets. The customer attribute may be called client, account, party, etc. in different domains.<\/p>\n<p id=\"b800\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">All occurrences of customers should be treated as PII, irrespective of the domains. But, creating separate policies for accessing customer data in each domain is prone to errors and inconsistencies and it is not a scalable option. It is not practical to expect each domain to be aware of the same customer being in other domains, and to track their associated policies.<\/p>\n<p id=\"5784\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The centralized governance layer comprises a data catalog, a data access governance product, and a data observability tool. Today, the initial deployments of data mesh architectures are building homegrown applications. However, there are a few data access governance and catalogs that can fill the void and provide state-of-art solutions.<\/p>\n<p id=\"eb59\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data access governance products should govern access control to data residing in multiple locations. It does so by centralizing access policies and applying them to the data elements, irrespective of their location. They dynamically apply policies to user identities and use privacy preserving techniques, such as masking, tokenization, and other forms of anonymization.<\/p>\n<p id=\"3308\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data catalogs discover and profile data, and infer business metadata. They also allow access policies to be defined for the data elements and their tags. A universal data authorization product is then used to enforce the policies. This centralized mechanism is needed to provide a comprehensive audit log that is consistent across all data platforms for compliance or root cause analysis reasons.<\/p>\n<p id=\"5ee6\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">This architecture ensures that different domains do not end up with inconsistent access policies for the cross-domain data products like customers. The centralized access layer can ensure that data privacy compliance regulations, such as data residency requirements, are being consistently enforced and logged.<\/p>\n<p id=\"1ad6\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">As data mesh establishes itself further, the data access governance layer\u2019s performance and scalability capabilities become a key evaluation criteria. Data architects should evaluate products that add as little as possible latency in enabling access to the data products. A successful deployment is one where the authorization platform is invisible to the data consumers browsing the data catalog for data products.<\/p>\n<p id=\"1759\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data stewards create various policies pertaining to their data products. Data stewards reside in domains and in the central team. The former curate data local to the domains, while the latter handle the cross-domain products, and they sit in a centralized team. The data stewards evaluate the universal data authorization product for its ease of use, such as the user experience of using the policy builder to develop policies in a no-code manner using a UI or programmatically through APIs. The policy library then becomes the single source of truth, allowing key policies to be mandated enterprise-wide, while allowing distributed stewardship of domain-specific policies.<\/p>\n<p id=\"3341\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">One of the most critical elements of the universal data authorization product is that policies are applied dynamically and with each query. Second, the platform deployed to do data mesh governance should provide a wide range of integration with data sources, such as data lakes and cloud data warehouses, and with analytical and data science tools. Extensibility of the platform, through the use of API, is an important consideration as we live in highly fluid analytical architectures, with novel concepts, such as metric stores, semantic layers, and feature stores.<\/p>\n<p id=\"cb44\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Finally, evaluate the platform that minimizes complexity and does not have strong vendor lock-in.<\/p>\n<h1 id=\"0a9e\" class=\"jg jh gx be ji jj jk jl jm jn jo jp jq jr js jt ju jv jw jx jy jz ka kb kc kd bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Conclusion<\/h1>\n<p id=\"2734\" class=\"pw-post-body-paragraph hu hv gx hw b hx ke hz ia ib kf id ie if kg ih ii ij kh il im in ki ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Will data mesh be the panacea for data related issues in the upcoming years? The naysayers are quick to point out that the issues this approach is addressing, and the proposed principles, are not new. While that is true, data mesh brings a fresh perspective. Data quality has been an ever-present issue which the past approaches have failed to alleviate. Data mesh\u2019s domain emphasis provides another approach. It is trite to say that data scientists spend the majority of their time wrangling data. Data mesh\u2019s attempt to treat data as a product can certainly help.<\/p>\n<p id=\"a557\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">A few large organizations, such as\u00a0<a class=\"af kj external\" href=\"https:\/\/aws.amazon.com\/blogs\/big-data\/how-jpmorgan-chase-built-a-data-mesh-architecture-to-drive-significant-value-to-enhance-their-enterprise-data-platform\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">JPMC<\/a>,\u00a0<a class=\"af kj external\" href=\"https:\/\/www.youtube.com\/watch?v=-POiudR2_R0\" target=\"_blank\" rel=\"noopener ugc nofollow\">Flexport<\/a>, and\u00a0<a class=\"af kj external\" href=\"https:\/\/www.youtube.com\/watch?v=tNcxoASumB8\" target=\"_blank\" rel=\"noopener ugc nofollow\">Intuit<\/a>\u00a0that have implemented data mesh, are reporting several benefits, such as<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"bfa0\" class=\"is it gx hw b hx hy ib ic if iu ij iv in iw ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Reducing the time between when consumers request new features and when data engineering teams deliver the functionality<\/li>\n<li id=\"ed78\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Fewer ad hoc requests for data on channels such as Slack<\/li>\n<li id=\"0ab7\" class=\"is it gx hw b hx jb ib jc if jd ij je in jf ir ix iy iz ja bj\" data-selectable-paragraph=\"\">Higher usage of data when it is made available as a product<\/li>\n<\/ul>\n<p id=\"33f3\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">But, challenges abound too. One of the most common questions is: which organizations should look into data mesh as a future state analytics architecture? The consensus is that data mesh is suitable for organizations that have very large data volumes, and especially if they are spread across various business units. If a business is not facing data engineering bottlenecks, data mesh may not be a suitable approach.<\/p>\n<p id=\"4749\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The biggest deterrent to data mesh is its lack of implementation details. No reference architecture exists, and support for tooling from software vendors is limited. As a result, various organizations are deploying their distributed architectures and calling it a data mesh. This can lead to more confusion.<\/p>\n<p id=\"74b3\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The ultimate challenge is in operationalizing analytics through a consistent data access governance mechanism to the domain and shared data. While no standards have as yet emerged, conventional approaches are not suitable. Companies pursuing data mesh need to introduce universal data authorization into their data stack.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Data mesh has attracted intense attention because of its promise to deliver faster analytics in an agile and decentralized manner. It puts the responsibility for data quality and curation on data producers and owners within business \u201cdomains,\u201d who understand the data the best. They package the data for consumption as a \u201cproduct.\u201d The two major [&hellip;]<\/p>\n","protected":false},"author":434,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[11],"tags":[],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Data Mesh Meets Universal Authorization<\/title>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data Mesh Meets Universal Authorization\" \/>\n<meta property=\"og:description\" content=\"Data mesh has attracted intense attention because of its promise to deliver faster analytics in an agile and decentralized manner. It puts the responsibility for data quality and curation on data producers and owners within business \u201cdomains,\u201d who understand the data the best. They package the data for consumption as a \u201cproduct.\u201d The two major [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/\" \/>\n<meta property=\"og:site_name\" content=\"Solutions Review Thought Leaders\" \/>\n<meta property=\"article:published_time\" content=\"2024-01-01T14:52:10+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-02-02T14:35:02+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/max\/700\/0*DDzhShVMEP4ev0XI\" \/>\n<meta name=\"author\" content=\"Sanjeev Mohan\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Sanjeev Mohan\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"12 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/\",\"url\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/\",\"name\":\"Data Mesh Meets Universal Authorization\",\"isPartOf\":{\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/miro.medium.com\/max\/700\/0*DDzhShVMEP4ev0XI\",\"datePublished\":\"2024-01-01T14:52:10+00:00\",\"dateModified\":\"2024-02-02T14:35:02+00:00\",\"author\":{\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/d3f510bbd5a4434f2da3d684f4a916ca\"},\"breadcrumb\":{\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/#primaryimage\",\"url\":\"https:\/\/miro.medium.com\/max\/700\/0*DDzhShVMEP4ev0XI\",\"contentUrl\":\"https:\/\/miro.medium.com\/max\/700\/0*DDzhShVMEP4ev0XI\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/solutionsreview.com\/thought-leaders\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Mesh Meets Universal Authorization\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/#website\",\"url\":\"https:\/\/solutionsreview.com\/thought-leaders\/\",\"name\":\"Solutions Review Thought Leaders\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/solutionsreview.com\/thought-leaders\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/d3f510bbd5a4434f2da3d684f4a916ca\",\"name\":\"Sanjeev Mohan\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/48b6d62967a183a096e11064bbd7ecdf?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/48b6d62967a183a096e11064bbd7ecdf?s=96&d=mm&r=g\",\"caption\":\"Sanjeev Mohan\"},\"description\":\"As an established thought leader in the areas of cloud, big data and analytics, Sanjeev has researched and provided advice on changing trends and technologies in the modern cloud data architectures. Sanjeev started his data and analytics journey at Oracle where he worked on emerging technologies and built cutting-edge solutions. Until recently, Sanjeev was a Gartner research vice president known for his prolific work and attention to detail. Sanjeev regularly presents on topics pertaining to end-to-end data pipelines and helps businesses discover what their data can do for them.\",\"sameAs\":[\"www.linkedin.com\/in\/sanjeev-mohan-498119\/\"],\"url\":\"https:\/\/solutionsreview.com\/thought-leaders\/author\/sanjeev-mohan\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Mesh Meets Universal Authorization","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Data Mesh Meets Universal Authorization","og_description":"Data mesh has attracted intense attention because of its promise to deliver faster analytics in an agile and decentralized manner. It puts the responsibility for data quality and curation on data producers and owners within business \u201cdomains,\u201d who understand the data the best. They package the data for consumption as a \u201cproduct.\u201d The two major [&hellip;]","og_url":"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/","og_site_name":"Solutions Review Thought Leaders","article_published_time":"2024-01-01T14:52:10+00:00","article_modified_time":"2024-02-02T14:35:02+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/max\/700\/0*DDzhShVMEP4ev0XI"}],"author":"Sanjeev Mohan","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Sanjeev Mohan","Est. reading time":"12 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/","url":"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/","name":"Data Mesh Meets Universal Authorization","isPartOf":{"@id":"https:\/\/solutionsreview.com\/thought-leaders\/#website"},"primaryImageOfPage":{"@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/#primaryimage"},"image":{"@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/max\/700\/0*DDzhShVMEP4ev0XI","datePublished":"2024-01-01T14:52:10+00:00","dateModified":"2024-02-02T14:35:02+00:00","author":{"@id":"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/d3f510bbd5a4434f2da3d684f4a916ca"},"breadcrumb":{"@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/#primaryimage","url":"https:\/\/miro.medium.com\/max\/700\/0*DDzhShVMEP4ev0XI","contentUrl":"https:\/\/miro.medium.com\/max\/700\/0*DDzhShVMEP4ev0XI"},{"@type":"BreadcrumbList","@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-mesh-meets-universal-authorization\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/solutionsreview.com\/thought-leaders\/"},{"@type":"ListItem","position":2,"name":"Data Mesh Meets Universal Authorization"}]},{"@type":"WebSite","@id":"https:\/\/solutionsreview.com\/thought-leaders\/#website","url":"https:\/\/solutionsreview.com\/thought-leaders\/","name":"Solutions Review Thought Leaders","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/solutionsreview.com\/thought-leaders\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/d3f510bbd5a4434f2da3d684f4a916ca","name":"Sanjeev Mohan","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/48b6d62967a183a096e11064bbd7ecdf?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/48b6d62967a183a096e11064bbd7ecdf?s=96&d=mm&r=g","caption":"Sanjeev Mohan"},"description":"As an established thought leader in the areas of cloud, big data and analytics, Sanjeev has researched and provided advice on changing trends and technologies in the modern cloud data architectures. Sanjeev started his data and analytics journey at Oracle where he worked on emerging technologies and built cutting-edge solutions. Until recently, Sanjeev was a Gartner research vice president known for his prolific work and attention to detail. Sanjeev regularly presents on topics pertaining to end-to-end data pipelines and helps businesses discover what their data can do for them.","sameAs":["www.linkedin.com\/in\/sanjeev-mohan-498119\/"],"url":"https:\/\/solutionsreview.com\/thought-leaders\/author\/sanjeev-mohan\/"}]}},"_links":{"self":[{"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/posts\/561"}],"collection":[{"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/users\/434"}],"replies":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/comments?post=561"}],"version-history":[{"count":0,"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/posts\/561\/revisions"}],"wp:attachment":[{"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/media?parent=561"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/categories?post=561"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/tags?post=561"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}