{"id":1603,"date":"2016-07-06T12:32:00","date_gmt":"2016-07-06T16:32:00","guid":{"rendered":"https:\/\/solutionsreview.com\/data-integration\/?p=1603"},"modified":"2016-10-05T09:31:40","modified_gmt":"2016-10-05T13:31:40","slug":"data-latency-is-turning-etl-into-et-hell","status":"publish","type":"post","link":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/","title":{"rendered":"Data Latency is Turning ETL into ET-Hell"},"content":{"rendered":"

\"Database<\/p>\n

Recent fragmentation in the legacy Data Integration market made us wonder whether or not traditional integration tools were becoming obsolete<\/a> before our very eyes. With this in mind, we asked the crowd<\/a> whether or not they believed Data Integration as we’ve known it was dying. To our surprise, the answer was a resounding no, and it appears that legacy tools are still being used in many verticals as enterprises prepare for the next wave in data tools. But just because ETL Data Integration solutions are still being used in data architectures all around doesn’t mean that they’re not frustrating to use.<\/p>\n

\t\t\t

\"Download<\/a><\/p>\n<\/div>\n\t\t<\/div><\/div>\n

Forward-thinking vendors are looking to get out ahead of what is becoming a major problem for enterprises, and that is data latency. Organizations, now more than ever, need integration tools that enable their end-users to do self-service both on-premise and in the cloud. Modern integration solutions, such as those that are branded as Data Virtualization<\/a>, Integration Platform as a Service<\/a>, or more commonly, self-service, provide agility that legacy offerings simply cannot match.<\/p>\n

If you’ve been working with data for any amount of time and have been exposed to the traditional ETL process, you’re very likely aware that it can be painstaking, to say the least. And though the crowd has made it loud and clear <\/a>that this technology, though being offered by fewer and fewer solution providers, is still being utilized. Some business professionals who work in more traditional data environments explain the process only in expletives, others in screams, still others in not very attractive facial expressions. In a changing world where data is streaming from a variety of sources in real-times, collecting and integrating data via this process after the fact is time consuming.<\/p>\n

ETL tools, while formerly the market standard for migrating data from one place to another, are being replaced by software that can do all of this in closer to real-time, often as an automated process. Many data professionals believe ETL to be one of, if not the biggest IT bottleneck, forcing end-users to wait for data to be taken from a source or storage medium, transferred to the new location, and loaded into the database where it can be pulled for analysis. All of these added steps cause huge amounts of latency.<\/p>\n

Though there has been some movement in ETL from a technological perspective over the course of the last half-decade, such as change data capture feature enhancements which migrate only changed data and shaving down process time, enterprises and vendors alike have their eyes on the future. Hadoop and other more modern Data Management solutions have really taken the market by storm, providing the kinds of automation and integrated storage that integration tools have long gone without. And while it’s true that many large organizations would like to avoid replacing all of their current systems with a Hadoop because of cost and effort, the fact of the matter is that ETL is just too slow and wonky to keep up with the demands for real-time self-service integration that the market demands.<\/p>\n
Widget not in any sidebars
\n","protected":false},"excerpt":{"rendered":"

Recent fragmentation in the legacy Data Integration market made us wonder whether or not traditional integration tools were becoming obsolete before our very eyes. With this in mind, we asked the crowd whether or not they believed Data Integration as we’ve known it was dying. To our surprise, the answer was a resounding no, and […]<\/p>\n","protected":false},"author":23,"featured_media":1604,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[344,194],"acf":[],"yoast_head":"\nData Latency is Turning ETL into ET-Hell<\/title>\n<meta name=\"description\" content=\"Just because ETL Data Integration solutions are still being used in data architectures all around doesn't mean that they're not frustrating to use.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Tim King\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/\"},\"author\":{\"name\":\"Tim King\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c\"},\"headline\":\"Data Latency is Turning ETL into ET-Hell\",\"datePublished\":\"2016-07-06T16:32:00+00:00\",\"dateModified\":\"2016-10-05T13:31:40+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/\"},\"wordCount\":530,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#organization\"},\"image\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/07\/oie_o4IcMy8pUDHC.jpg\",\"keywords\":[\"Data Latency\",\"ETL\"],\"articleSection\":[\"Best Practices\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/\",\"name\":\"Data Latency is Turning ETL into ET-Hell\",\"isPartOf\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/07\/oie_o4IcMy8pUDHC.jpg\",\"datePublished\":\"2016-07-06T16:32:00+00:00\",\"dateModified\":\"2016-10-05T13:31:40+00:00\",\"description\":\"Just because ETL Data Integration solutions are still being used in data architectures all around doesn't mean that they're not frustrating to use.\",\"breadcrumb\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#primaryimage\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/07\/oie_o4IcMy8pUDHC.jpg\",\"contentUrl\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/07\/oie_o4IcMy8pUDHC.jpg\",\"width\":800,\"height\":350},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/solutionsreview.com\/data-integration\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Latency is Turning ETL into ET-Hell\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#website\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/\",\"name\":\"Best Data Integration Vendors, News & Reviews for Big Data, Applications, ETL and Hadoop\",\"description\":\"Data Integration Buyers Guide and Best Practices\",\"publisher\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/solutionsreview.com\/data-integration\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#organization\",\"name\":\"Solutions Review\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/02\/Solutions_Review_Header_Data_Integration_225.png\",\"contentUrl\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/02\/Solutions_Review_Header_Data_Integration_225.png\",\"width\":225,\"height\":90,\"caption\":\"Solutions Review\"},\"image\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c\",\"name\":\"Tim King\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2023\/12\/tk.jpg\",\"contentUrl\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2023\/12\/tk.jpg\",\"caption\":\"Tim King\"},\"description\":\"Tim is Solutions Review's Executive Editor covering the human impact of AI on the future of work and learning. He is also the Media Strategist behind Insight Jam (1M+ on YouTube) events and programming. A 2017 and 2018 Most Influential Business Journalist and 2021 \\\"Who's Who\\\" in multiple categories, Tim is a recognized thought leader in enterprise tech and AI.\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/author\/timking\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Latency is Turning ETL into ET-Hell","description":"Just because ETL Data Integration solutions are still being used in data architectures all around doesn't mean that they're not frustrating to use.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/","twitter_misc":{"Written by":"Tim King","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#article","isPartOf":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/"},"author":{"name":"Tim King","@id":"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c"},"headline":"Data Latency is Turning ETL into ET-Hell","datePublished":"2016-07-06T16:32:00+00:00","dateModified":"2016-10-05T13:31:40+00:00","mainEntityOfPage":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/"},"wordCount":530,"commentCount":0,"publisher":{"@id":"https:\/\/solutionsreview.com\/data-integration\/#organization"},"image":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#primaryimage"},"thumbnailUrl":"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/07\/oie_o4IcMy8pUDHC.jpg","keywords":["Data Latency","ETL"],"articleSection":["Best Practices"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/","url":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/","name":"Data Latency is Turning ETL into ET-Hell","isPartOf":{"@id":"https:\/\/solutionsreview.com\/data-integration\/#website"},"primaryImageOfPage":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#primaryimage"},"image":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#primaryimage"},"thumbnailUrl":"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/07\/oie_o4IcMy8pUDHC.jpg","datePublished":"2016-07-06T16:32:00+00:00","dateModified":"2016-10-05T13:31:40+00:00","description":"Just because ETL Data Integration solutions are still being used in data architectures all around doesn't mean that they're not frustrating to use.","breadcrumb":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#primaryimage","url":"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/07\/oie_o4IcMy8pUDHC.jpg","contentUrl":"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/07\/oie_o4IcMy8pUDHC.jpg","width":800,"height":350},{"@type":"BreadcrumbList","@id":"https:\/\/solutionsreview.com\/data-integration\/data-latency-is-turning-etl-into-et-hell\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/solutionsreview.com\/data-integration\/"},{"@type":"ListItem","position":2,"name":"Data Latency is Turning ETL into ET-Hell"}]},{"@type":"WebSite","@id":"https:\/\/solutionsreview.com\/data-integration\/#website","url":"https:\/\/solutionsreview.com\/data-integration\/","name":"Best Data Integration Vendors, News & Reviews for Big Data, Applications, ETL and Hadoop","description":"Data Integration Buyers Guide and Best Practices","publisher":{"@id":"https:\/\/solutionsreview.com\/data-integration\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/solutionsreview.com\/data-integration\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/solutionsreview.com\/data-integration\/#organization","name":"Solutions Review","url":"https:\/\/solutionsreview.com\/data-integration\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/logo\/image\/","url":"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/02\/Solutions_Review_Header_Data_Integration_225.png","contentUrl":"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/02\/Solutions_Review_Header_Data_Integration_225.png","width":225,"height":90,"caption":"Solutions Review"},"image":{"@id":"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c","name":"Tim King","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/image\/","url":"https:\/\/solutionsreview.com\/data-integration\/files\/2023\/12\/tk.jpg","contentUrl":"https:\/\/solutionsreview.com\/data-integration\/files\/2023\/12\/tk.jpg","caption":"Tim King"},"description":"Tim is Solutions Review's Executive Editor covering the human impact of AI on the future of work and learning. He is also the Media Strategist behind Insight Jam (1M+ on YouTube) events and programming. A 2017 and 2018 Most Influential Business Journalist and 2021 \"Who's Who\" in multiple categories, Tim is a recognized thought leader in enterprise tech and AI.","url":"https:\/\/solutionsreview.com\/data-integration\/author\/timking\/"}]}},"_links":{"self":[{"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/posts\/1603"}],"collection":[{"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/users\/23"}],"replies":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/comments?post=1603"}],"version-history":[{"count":0,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/posts\/1603\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/media\/1604"}],"wp:attachment":[{"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/media?parent=1603"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/categories?post=1603"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/tags?post=1603"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}