{"id":2233,"date":"2018-01-26T13:34:55","date_gmt":"2018-01-26T18:34:55","guid":{"rendered":"https:\/\/solutionsreview.com\/data-integration\/?p=2233"},"modified":"2018-01-29T08:54:56","modified_gmt":"2018-01-29T13:54:56","slug":"data-streaming-enables-real-time-decision-making","status":"publish","type":"post","link":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/","title":{"rendered":"Data Streaming Enables Real-Time Decision Making"},"content":{"rendered":"

\"Data<\/p>\n

By\u00a0Yair Weinberger<\/span><\/strong><\/h5>\n
Data-Driven Businesses are replacing Batch ETL Uploads with Pipelines to Transform Analytics and Business Intelligence<\/i><\/b><\/h5>\n

Most organizations are struggling to harness the real-time intelligence from their disparate data silos that\u2019s needed to improve business agility, decision-making and competitiveness. While many are acutely aware that their old, rigid batch-upload architectures are grossly inadequate, they are at a loss to find a solution.<\/span><\/p>\n

\t\t\t

\"Download<\/a><\/p>\n<\/div>\n\t\t<\/div><\/div>\n

Meanwhile, millions of organizations have placed their\u00a0data warehouses in the cloud \u2014 to benefit from on-demand storage, flexible computing power, and lower costs. Migrating to the cloud, however, does not solve their data pipeline issues.<\/p>\n

That\u2019s because traditional batch upload processes are unable to meet the continuous, real-time, data-to-data needs of warehouses and business intelligence applications. Batch processes that upload data once a day, twice a day, or even once an hour have been pushed beyond their life expectancy by today\u2019s real-time environments.<\/p>\n

Data Streaming: A Disruptive New Approach<\/strong><\/h5>\n

Data streaming is a powerful and innovative solution to organizations\u2019 data pipeline headaches. Much of today\u2019s modern data streaming capabilities are powered by open source technologies, for example, Apache Kafka as a distributed streaming platform.\u00a0<\/span>A robust and proven solution\u00a0creates secure pipelines that stream data in real-time from various sources \u2014 notably databases, applications, and APIs \u2014 to leading cloud data warehouse platforms.<\/span><\/p>\n

Leading organizations in virtually every industry are moving to a real-time data model to be more agile and achieve a competitive edge through faster and better decision making.<\/span><\/p>\n

Data Streaming in a Straightforward Two-Step Process<\/b><\/h5>\n

Step one involves importing data into the platform\u00a0from all sources, such as: transactional databases (e.g. Oracle, Postgresql) Salesforce (account information, stage, ownership etc); website tracking (all web event data); web server (customer activity such as adding inputs, and deploying new code in the code engine); backend logs (internal platform events such as data being loaded to the output, new table created); and monitoring systems (to capture system issues such as input connections and latency). This is accomplished by selecting the desired inputs, authentifying user(s) credentials and setting up how files will be imported. <\/span><\/p>\n

Step two is integration.\u00a0From here, the data can be analyzed using popular business intelligence tools.<\/span><\/p>\n

Achieving Real-Time Intelligence<\/b><\/h5>\n

Many enterprises lack a central warehouse for data available from backend relational databases, online events and metrics, support services, and other internal and external sources. Primarily because it is spread across multiple sources and different systems in different formats. Some of it\u2019s flat, some is relational, some is JSON, and writing custom scripts to integrate it all is beyond the resources of most companies. Without centralization, however, analytics are both piecemeal and siloed. This makes it difficult, if not impossible, to produce real-time intelligence. <\/span><\/p>\n

<\/div>\n

\"YairYair Weinberger\u00a0is the Co-founder and CTO at Alooma<\/a>. He’s an expert in data integration, real-time data platforms, big data and data warehousing. Previously, he led development for ConvertMedia (later acquired by Taboola). Yair began his career with the Israel Defense Forces (IDF) where he managed cyber security and real-time support systems for military operations.<\/p>\n

<\/div>\n
Widget not in any sidebars
\n","protected":false},"excerpt":{"rendered":"

By\u00a0Yair Weinberger Data-Driven Businesses are replacing Batch ETL Uploads with Pipelines to Transform Analytics and Business Intelligence Most organizations are struggling to harness the real-time intelligence from their disparate data silos that\u2019s needed to improve business agility, decision-making and competitiveness. While many are acutely aware that their old, rigid batch-upload architectures are grossly inadequate, they […]<\/p>\n","protected":false},"author":23,"featured_media":2234,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[292,478],"acf":[],"yoast_head":"\nData Streaming Enables Real-Time Decision Making<\/title>\n<meta name=\"description\" content=\"Data streaming capabilities are replacing batch ETL in enterprise settings where speed is a necessity.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Tim King\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/\"},\"author\":{\"name\":\"Tim King\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c\"},\"headline\":\"Data Streaming Enables Real-Time Decision Making\",\"datePublished\":\"2018-01-26T18:34:55+00:00\",\"dateModified\":\"2018-01-29T13:54:56+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/\"},\"wordCount\":530,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#organization\"},\"image\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2017\/07\/oie_PIu0w2NDI5LZ.jpg\",\"keywords\":[\"Alooma\",\"Yair Weinberger\"],\"articleSection\":[\"Best Practices\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/\",\"name\":\"Data Streaming Enables Real-Time Decision Making\",\"isPartOf\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2017\/07\/oie_PIu0w2NDI5LZ.jpg\",\"datePublished\":\"2018-01-26T18:34:55+00:00\",\"dateModified\":\"2018-01-29T13:54:56+00:00\",\"description\":\"Data streaming capabilities are replacing batch ETL in enterprise settings where speed is a necessity.\",\"breadcrumb\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#primaryimage\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2017\/07\/oie_PIu0w2NDI5LZ.jpg\",\"contentUrl\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2017\/07\/oie_PIu0w2NDI5LZ.jpg\",\"width\":800,\"height\":350,\"caption\":\"Data Streaming Enables Real-Time Decision Making\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/solutionsreview.com\/data-integration\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Streaming Enables Real-Time Decision Making\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#website\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/\",\"name\":\"Data Integration Tools & ETL Software | Solutions Review\",\"description\":\"Evaluating Enterprise ETL Tools, Cloud iPaaS & Real-Time Data Streaming Platforms.\",\"publisher\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/solutionsreview.com\/data-integration\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#organization\",\"name\":\"Solutions Review\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/02\/Solutions_Review_Header_Data_Integration_225.png\",\"contentUrl\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/02\/Solutions_Review_Header_Data_Integration_225.png\",\"width\":225,\"height\":90,\"caption\":\"Solutions Review\"},\"image\":{\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c\",\"name\":\"Tim King\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2023\/12\/tk.jpg\",\"contentUrl\":\"https:\/\/solutionsreview.com\/data-integration\/files\/2023\/12\/tk.jpg\",\"caption\":\"Tim King\"},\"description\":\"Tim is Solutions Review's Executive Editor covering the human impact of AI on the future of work and learning. He is also the Media Strategist behind Insight Jam (1M+ on YouTube) events and programming. A 2017 and 2018 Most Influential Business Journalist and 2021 \\\"Who's Who\\\" in multiple categories, Tim is a recognized thought leader in enterprise tech and AI.\",\"url\":\"https:\/\/solutionsreview.com\/data-integration\/author\/timking\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Streaming Enables Real-Time Decision Making","description":"Data streaming capabilities are replacing batch ETL in enterprise settings where speed is a necessity.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/","twitter_misc":{"Written by":"Tim King","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#article","isPartOf":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/"},"author":{"name":"Tim King","@id":"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c"},"headline":"Data Streaming Enables Real-Time Decision Making","datePublished":"2018-01-26T18:34:55+00:00","dateModified":"2018-01-29T13:54:56+00:00","mainEntityOfPage":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/"},"wordCount":530,"commentCount":0,"publisher":{"@id":"https:\/\/solutionsreview.com\/data-integration\/#organization"},"image":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#primaryimage"},"thumbnailUrl":"https:\/\/solutionsreview.com\/data-integration\/files\/2017\/07\/oie_PIu0w2NDI5LZ.jpg","keywords":["Alooma","Yair Weinberger"],"articleSection":["Best Practices"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/","url":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/","name":"Data Streaming Enables Real-Time Decision Making","isPartOf":{"@id":"https:\/\/solutionsreview.com\/data-integration\/#website"},"primaryImageOfPage":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#primaryimage"},"image":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#primaryimage"},"thumbnailUrl":"https:\/\/solutionsreview.com\/data-integration\/files\/2017\/07\/oie_PIu0w2NDI5LZ.jpg","datePublished":"2018-01-26T18:34:55+00:00","dateModified":"2018-01-29T13:54:56+00:00","description":"Data streaming capabilities are replacing batch ETL in enterprise settings where speed is a necessity.","breadcrumb":{"@id":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#primaryimage","url":"https:\/\/solutionsreview.com\/data-integration\/files\/2017\/07\/oie_PIu0w2NDI5LZ.jpg","contentUrl":"https:\/\/solutionsreview.com\/data-integration\/files\/2017\/07\/oie_PIu0w2NDI5LZ.jpg","width":800,"height":350,"caption":"Data Streaming Enables Real-Time Decision Making"},{"@type":"BreadcrumbList","@id":"https:\/\/solutionsreview.com\/data-integration\/data-streaming-enables-real-time-decision-making\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/solutionsreview.com\/data-integration\/"},{"@type":"ListItem","position":2,"name":"Data Streaming Enables Real-Time Decision Making"}]},{"@type":"WebSite","@id":"https:\/\/solutionsreview.com\/data-integration\/#website","url":"https:\/\/solutionsreview.com\/data-integration\/","name":"Data Integration Tools & ETL Software | Solutions Review","description":"Evaluating Enterprise ETL Tools, Cloud iPaaS & Real-Time Data Streaming Platforms.","publisher":{"@id":"https:\/\/solutionsreview.com\/data-integration\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/solutionsreview.com\/data-integration\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/solutionsreview.com\/data-integration\/#organization","name":"Solutions Review","url":"https:\/\/solutionsreview.com\/data-integration\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/logo\/image\/","url":"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/02\/Solutions_Review_Header_Data_Integration_225.png","contentUrl":"https:\/\/solutionsreview.com\/data-integration\/files\/2016\/02\/Solutions_Review_Header_Data_Integration_225.png","width":225,"height":90,"caption":"Solutions Review"},"image":{"@id":"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/154e152a275103e373e24ada7f2feb5c","name":"Tim King","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/data-integration\/#\/schema\/person\/image\/","url":"https:\/\/solutionsreview.com\/data-integration\/files\/2023\/12\/tk.jpg","contentUrl":"https:\/\/solutionsreview.com\/data-integration\/files\/2023\/12\/tk.jpg","caption":"Tim King"},"description":"Tim is Solutions Review's Executive Editor covering the human impact of AI on the future of work and learning. He is also the Media Strategist behind Insight Jam (1M+ on YouTube) events and programming. A 2017 and 2018 Most Influential Business Journalist and 2021 \"Who's Who\" in multiple categories, Tim is a recognized thought leader in enterprise tech and AI.","url":"https:\/\/solutionsreview.com\/data-integration\/author\/timking\/"}]}},"_links":{"self":[{"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/posts\/2233"}],"collection":[{"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/users\/23"}],"replies":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/comments?post=2233"}],"version-history":[{"count":0,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/posts\/2233\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/media\/2234"}],"wp:attachment":[{"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/media?parent=2233"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/categories?post=2233"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/solutionsreview.com\/data-integration\/wp-json\/wp\/v2\/tags?post=2233"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}