{"id":3550,"date":"2021-12-20T13:01:16","date_gmt":"2021-12-20T18:01:16","guid":{"rendered":"https:\/\/solutionsreview.com\/data-management\/?p=3550"},"modified":"2021-12-20T14:29:54","modified_gmt":"2021-12-20T19:29:54","slug":"data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022","status":"publish","type":"post","link":"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/","title":{"rendered":"Data Curation and Governance are the Top-Two Data Engineering Challenges for 2022"},"content":{"rendered":"

\"Data<\/p>\n

Data curation and governance are the top-two data engineering challenges for 2022, according to a report by Gradient Flow and Immuta.<\/strong><\/em><\/p>\n

Data curation and governance are the top-two data engineering challenges for 2022, according to a new report<\/a><\/strong><\/span> commissioned by Gradient Flow and Immuta. The 2022 State of Data Engineering<\/a><\/strong><\/span><\/em> survey examined the changing landscape of data engineering<\/a><\/strong><\/span> and operations challenges, tools, and opportunities. The data engineering challenges that data professionals worry about most come after data has been extracted, loaded, and transformed. Data for the report was gathered from a global audience of 372 respondents, more than half of which were data engineers or data architects, over 61 days.<\/p>\n

\t\t\t
\"Download<\/a><\/div>\n\t\t<\/div><\/div>\n

The main data engineering challenges cited by those polled include validation, data monitoring and auditing for compliance, data masking and anonymization, as well as data discovery. Nearly two-thirds of respondents (65 percent) said their company is either 100 percent cloud-based or will be in the next 12-to-24 months. In the same way, 62 percent of respondents signaled their plans to adopt one of the top-five cloud databases and platforms (Amazon Redshift, Amazon Athena, Google BigQuery, Databricks, and Snowflake) in the months ahead.<\/p>\n

While 64 percent of those polled come from organizations already collecting and storing sensitive data, the vast majority (88 percent) indicated that their firms are subject to one or more data use rules or regulations like GDPR, HIPAA, CCPA, and SOC 2. Additionally, 30 percent of respondents reported a need to comply with internal, company-specific rules around data. Somewhat concerning is that more than a quarter of all those polled were unsure of what (if any) data quality solution their organization is currently using.<\/p>\n

The data engineering landscape is changing and maturing. Whereas years ago there were few, if any, tools to solve data challenges, a plethora of technologies \u2013 both commercial and open-source \u2013 are now available. These technologies are helping organizations leverage their sensitive data for real-time access and analytics, all while protecting it in accordance with a growing body of regulatory requirements. There are also an entirely new crop of data engineering training courses<\/a><\/strong><\/span> and online certification options<\/a><\/strong><\/span> available (tto enable technical and non-technical users alike to develop on-the-fly skills.<\/p>\n

Read the 2022 State of Data Engineering report on emerging challenges with data security and quality, courtesy of Gradient Flow and Immuta.<\/a><\/span><\/strong><\/span><\/h3>\n
<\/div>\n
\t\t\t

\"Download<\/a><\/p>\n<\/div>\n\t\t<\/div><\/div>\n","protected":false},"excerpt":{"rendered":"

Data curation and governance are the top-two data engineering challenges for 2022, according to a report by Gradient Flow and Immuta. Data curation and governance are the top-two data engineering challenges for 2022, according to a new report commissioned by Gradient Flow and Immuta. The 2022 State of Data Engineering survey examined the changing landscape […]<\/p>\n","protected":false},"author":23,"featured_media":3554,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[1184,1183,737],"acf":[],"yoast_head":"\nData Curation and Governance are the Top-Two Data Engineering Challenges for 2022<\/title>\n<meta name=\"description\" content=\"Data curation and governance are the top-two data engineering challenges for 2022, according to a report by Gradient Flow and Immuta.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data Curation and Governance are the Top-Two Data Engineering Challenges for 2022\" \/>\n<meta property=\"og:description\" content=\"Data curation and governance are the top-two data engineering challenges for 2022, according to a report by Gradient Flow and Immuta.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/\" \/>\n<meta property=\"og:site_name\" content=\"Best Data Management Software, Vendors and Data Science Platforms\" \/>\n<meta property=\"article:published_time\" content=\"2021-12-20T18:01:16+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-12-20T19:29:54+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/solutionsreview.com\/data-management\/files\/2021\/12\/State-of-Data-Engineering.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"800\" \/>\n\t<meta property=\"og:image:height\" content=\"400\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Tim King\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Tim King\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/\",\"url\":\"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/\",\"name\":\"Data Curation and Governance are the Top-Two Data Engineering Challenges for 2022\",\"isPartOf\":{\"@id\":\"https:\/\/solutionsreview.com\/data-management\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/solutionsreview.com\/data-management\/files\/2021\/12\/State-of-Data-Engineering.jpg\",\"datePublished\":\"2021-12-20T18:01:16+00:00\",\"dateModified\":\"2021-12-20T19:29:54+00:00\",\"author\":{\"@id\":\"https:\/\/solutionsreview.com\/data-management\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c\"},\"description\":\"Data curation and governance are the top-two data engineering challenges for 2022, according to a report by Gradient Flow and Immuta.\",\"breadcrumb\":{\"@id\":\"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/#primaryimage\",\"url\":\"https:\/\/solutionsreview.com\/data-management\/files\/2021\/12\/State-of-Data-Engineering.jpg\",\"contentUrl\":\"https:\/\/solutionsreview.com\/data-management\/files\/2021\/12\/State-of-Data-Engineering.jpg\",\"width\":800,\"height\":400,\"caption\":\"Data Curation and Governance are the Top-Two Data Engineering Challenges for 2022\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/solutionsreview.com\/data-management\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Curation and Governance are the Top-Two Data Engineering Challenges for 2022\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/solutionsreview.com\/data-management\/#website\",\"url\":\"https:\/\/solutionsreview.com\/data-management\/\",\"name\":\"Best Data Management Software, Vendors and Data Science Platforms\",\"description\":\"Enterprise Information Management\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/solutionsreview.com\/data-management\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/solutionsreview.com\/data-management\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c\",\"name\":\"Tim King\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/data-management\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/solutionsreview.com\/data-management\/files\/2023\/12\/tk.jpg\",\"contentUrl\":\"https:\/\/solutionsreview.com\/data-management\/files\/2023\/12\/tk.jpg\",\"caption\":\"Tim King\"},\"description\":\"Tim is Solutions Review's Executive Editor and leads coverage on data management and analytics. A 2017 and 2018 Most Influential Business Journalist and 2021 \\\"Who's Who\\\" in Data Management, Tim is a recognized industry thought leader and changemaker. Story? Reach him via email at tking@solutionsreview.com.\",\"url\":\"https:\/\/solutionsreview.com\/data-management\/author\/timking\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Curation and Governance are the Top-Two Data Engineering Challenges for 2022","description":"Data curation and governance are the top-two data engineering challenges for 2022, according to a report by Gradient Flow and Immuta.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/","og_locale":"en_US","og_type":"article","og_title":"Data Curation and Governance are the Top-Two Data Engineering Challenges for 2022","og_description":"Data curation and governance are the top-two data engineering challenges for 2022, according to a report by Gradient Flow and Immuta.","og_url":"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/","og_site_name":"Best Data Management Software, Vendors and Data Science Platforms","article_published_time":"2021-12-20T18:01:16+00:00","article_modified_time":"2021-12-20T19:29:54+00:00","og_image":[{"width":800,"height":400,"url":"https:\/\/solutionsreview.com\/data-management\/files\/2021\/12\/State-of-Data-Engineering.jpg","type":"image\/jpeg"}],"author":"Tim King","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Tim King","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/","url":"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/","name":"Data Curation and Governance are the Top-Two Data Engineering Challenges for 2022","isPartOf":{"@id":"https:\/\/solutionsreview.com\/data-management\/#website"},"primaryImageOfPage":{"@id":"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/#primaryimage"},"image":{"@id":"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/#primaryimage"},"thumbnailUrl":"https:\/\/solutionsreview.com\/data-management\/files\/2021\/12\/State-of-Data-Engineering.jpg","datePublished":"2021-12-20T18:01:16+00:00","dateModified":"2021-12-20T19:29:54+00:00","author":{"@id":"https:\/\/solutionsreview.com\/data-management\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c"},"description":"Data curation and governance are the top-two data engineering challenges for 2022, according to a report by Gradient Flow and Immuta.","breadcrumb":{"@id":"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/#primaryimage","url":"https:\/\/solutionsreview.com\/data-management\/files\/2021\/12\/State-of-Data-Engineering.jpg","contentUrl":"https:\/\/solutionsreview.com\/data-management\/files\/2021\/12\/State-of-Data-Engineering.jpg","width":800,"height":400,"caption":"Data Curation and Governance are the Top-Two Data Engineering Challenges for 2022"},{"@type":"BreadcrumbList","@id":"https:\/\/solutionsreview.com\/data-management\/data-curation-and-governance-are-the-top-two-data-engineering-challenges-for-2022\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/solutionsreview.com\/data-management\/"},{"@type":"ListItem","position":2,"name":"Data Curation and Governance are the Top-Two Data Engineering Challenges for 2022"}]},{"@type":"WebSite","@id":"https:\/\/solutionsreview.com\/data-management\/#website","url":"https:\/\/solutionsreview.com\/data-management\/","name":"Best Data Management Software, Vendors and Data Science Platforms","description":"Enterprise Information Management","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/solutionsreview.com\/data-management\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/solutionsreview.com\/data-management\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c","name":"Tim King","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/data-management\/#\/schema\/person\/image\/","url":"https:\/\/solutionsreview.com\/data-management\/files\/2023\/12\/tk.jpg","contentUrl":"https:\/\/solutionsreview.com\/data-management\/files\/2023\/12\/tk.jpg","caption":"Tim King"},"description":"Tim is Solutions Review's Executive Editor and leads coverage on data management and analytics. A 2017 and 2018 Most Influential Business Journalist and 2021 \"Who's Who\" in Data Management, Tim is a recognized industry thought leader and changemaker. Story? Reach him via email at tking@solutionsreview.com.","url":"https:\/\/solutionsreview.com\/data-management\/author\/timking\/"}]}},"_links":{"self":[{"href":"https:\/\/solutionsreview.com\/data-management\/wp-json\/wp\/v2\/posts\/3550"}],"collection":[{"href":"https:\/\/solutionsreview.com\/data-management\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/solutionsreview.com\/data-management\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/data-management\/wp-json\/wp\/v2\/users\/23"}],"replies":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/data-management\/wp-json\/wp\/v2\/comments?post=3550"}],"version-history":[{"count":0,"href":"https:\/\/solutionsreview.com\/data-management\/wp-json\/wp\/v2\/posts\/3550\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/data-management\/wp-json\/wp\/v2\/media\/3554"}],"wp:attachment":[{"href":"https:\/\/solutionsreview.com\/data-management\/wp-json\/wp\/v2\/media?parent=3550"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/solutionsreview.com\/data-management\/wp-json\/wp\/v2\/categories?post=3550"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/solutionsreview.com\/data-management\/wp-json\/wp\/v2\/tags?post=3550"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}