Self-service Big Data solution provider Qubole recently announced an autonomous data platform, which, according to the vendor, is the market’s first. The platform includes three new products: Qubole Data Service (QDS) Community Edition, QDS Enterprise Edition and QDS Cloud Agents. The solution intelligently automates and analyzes platform usage to increase the effectiveness of data teams in the enterprise. The architecture analyzes metadata (queries, clusters, users, data, etc.) generated by platform usage and applies machine learning and artificial intelligence to create alerts, insights, recommendations and autonomous agents which perform actions automatically.
QDS Enterprise Edition offers alerts, insights and recommendations that provide actionable information to data users. It features a machine learning infrastructure that enables AIR (alerts, insights, recommendations) that can be rule-based, workload aware and predictive to learn from user behavior over time. QDS Cloud Agents autonomously executes a range of data management tasks that are traditionally manual. It includes cloud-based software agents that offload tasks based on a set policy. The initial release of QDS Cloud Agents includes the following:
- Workload Aware Auto-Scaling: Optimizes cluster size to workload requirements. The agent is workload-aware and manages scaling both up and down based on actual processing load
- Spot Shopper (AWS): Intelligently shops across the AWS cloud to assemble the compute instances optimally to include AWS Spot Instances and Spot Blocks, mixed instances types, different availability zones and can be provisioned at run time
- Data Caching: Optimizes the locality of data for interactive access speeds. Data Transporter takes into consideration which data sets are frequently accessed and intelligently moves data into the background for performance requirements
QDS Community is a free product that provides the same functionality that the Enterprise version does. It’s free up to four nodes and five clusters.
In a statement, the company’s co-founder and CEO Ashish Thusoo explained: “Despite all the promise and technological advancements, operationalizing big data efforts is still impossible for most organizations. Data teams simply can’t scale to meet the growing demand for data across organizations. To address this hurdle, we are on a mission to remove the manual effort that comes with maintaining a big data infrastructure and empower data teams to focus on high-value, strategic work.”
The announcement was made at Data Platforms 2017, a conference focused exclusively on helping data teams build platforms.