Qubole Pipelines Service Enables Streaming Data Pipeline Building

Qubole Pipelines Service Enables Streaming Data Pipeline Building

Source: Qubole

Qubole recently announced the launch of Qubole Pipelines Service, a new solution that enables users to build scalable streaming data pipelines. The offering lets customers build, test, deploy, monitor and manage streaming data pipelines in a single platform. Qubole Pipelines Service is designed to augment an organization’s existing data lake in a manged environment via the public cloud of their choice.

Our Buyer’s Guide for Data Management Platforms helps you evaluate the best solution for your use case and features profiles of the leading providers, as well as a category overview of the marketplace.

Named to the 2020 DBTA 100 and featured in our list of Data Management Vendors Our Editors Are Tracking, Qubole adds in its official press release: “With Qubole Pipelines Service, businesses can complement their existing data lake with advanced features that help them instantly capture streaming data from various sources, accelerate development of streaming applications and run highly reliable and observable production applications at the lowest cost.”

Qubole Pipelines Service includes a number of built-in connectors, a code generation wizard, dry run framework, and quick-start options that help hasten development. A pipeline can be developed “within a few minutes” without the user having to write any code. Qubole’s newest innovation utilizes Apache Spark Structured Streaming. Qubole has added enhancements like Rocksdb slate storage, direct writes, and memory pressure scheduling for reliability in building and deploying long-running streaming applications.

The solution touts a broad set of APIs and user interfaces for engineers to manage the streaming application lifecycle. Data management and consistency come via Qubole’s ACID framework to compact small files in the background while allowing concurrent read / write operations that don’t hinder performance.

In a statement about the news, Qubole co-founder and CTO Joydeep Sarma said: “The arrival of Pipelines Service not only equips data teams and engineers with the most comprehensive solution to quickly build streaming data pipelines and analyze massive streams of data And it also underscores Qubole’s longstanding tradition of providing businesses with an open data lake platform for batch and streaming analytics.”

Qubole Pipelines Services is now available on AWS, Google Cloud, and Microsoft Azure.

Learn more about Qubole Pipeline Services.

Follow Tim

Timothy King

Senior Editor at Solutions Review
Timothy is Solutions Review's Senior Editor. He is a recognized thought leader and influencer in enterprise BI and data analytics. Timothy has been named a top global business journalist by Richtopia. Scoop? First initial, last name at solutionsreview dot com.
Timothy King
Follow Tim