The 8 Best Open-Source Data Lineage Tools to Consider

The editors at Solutions Review have compiled this list of the best open-source data lineage tools to consider for your next project.

Searching for data integration and data management software can be a daunting (and expensive) process, one that requires long hours of research and deep pockets. The most popular enterprise data lineage tools often provide more than what’s necessary for non-enterprise organizations, with advanced functionality relevant to only the most technically savvy users. Thankfully, there are a distinct group of the best open-source data lineage tools out there. Some of these solutions are offered by vendors looking to eventually sell you on their enterprise product, and others are maintained and operated by a community of developers looking to democratize the process.

In this article, we will examine the best open-source data lineage tools, first by providing a brief overview of what to expect and also with short blurbs about each of the currently available options in the space. This is the most complete and up-to-date directory on the web.

For an in-depth breakdown of supporting data quality processes with data lineage, our editors recommend this short guide courtesy of MANTA.

The Best Open-Source Data Lineage Tools

Apatar

Apatar is a free and open-source data integration software package designed to help business users and developers move data in and out of a variety of data sources and formats. The tool requires no programming or design to accomplish even complex integration with joins across several data sources. Apatar provides a visual interface to minimize the impact of system changes. The tool comes with a pre-built set of integration tools and enables users to re-use previously built mapping schemas as well.

CloverETL

CloverETL (now CloverDX) was one of the first open-source ETL tools. The Java-based data integration framework was designed to transform, map, and manipulate data in various formats. CloverETL can be used standalone or embedded and connects to RDBMS, JMS, SOAP, LDAP, S3, HTTP, FTP, ZIP, and TAR. Though the product is no longer offered by the provider, it can be downloaded securely using SourceForge. CloverDX still supports CloverETL in line with their standard support agreement as well.

Dremio

Dremio offers a product called Data Lake engine that provides fast query speed and a self-service semantic layer that operates directly against data lake storage. The solution connects to S3, ADLS, Hadoop or wherever enterprise data resides. Apache Arrow, Data Reflections and other Dremio technologies work together to hasten query speeds, and the semantic layer enables IT to apply security and business meaning. Users do not have to send data to Dremio or have it stored in proprietary formats to access it.

Kylo

Kylo is an open-source and enterprise-ready data lake management software platform designed for self-service data ingest and data preparation. The solution touts integrated metadata management, governance, security, and best practices inspired by Think Big’s 150+ big data implementation projects. Key features of Kylo include self-service data ingest, data wrangling and preparation via visual SQL, the ability to search and explore data and metadata, monitoring the health of feeds and services in the data lake, and batch or streaming pipeline design templates in Apache NiFi.

Talend Open Studio

Talend’s Open Studio offers a number of open-source data integration and data management solutions for a variety of use cases. Open Studio for Data Integration lets you jumpstart ETL projects and integrate data, Open Studio for Big Data helps to simplify ETL for large and diverse data sets, Data Preparation – Free Desktop enables users to freely discover, blend, and clean data, Open Studio for ESB speeds up orchestration of applications and APIs, and Open Studio for Data Quality assesses the accuracy and integrity of data. Talend also offers open-source Stitch for loading data into cloud data warehouses and data lakes.

TIBCO Jaspersoft ETL

Jaspersoft ETL is a part of TIBCO’s Community Edition open-source product portfolio that allows users to extract data from various sources, transform the data based on defined business rules, and load it into a centralized data warehouse for reporting and analytics. The tool’s data integration engine is powered by Talend. The Community Edition offers a graphical design environment, more than 500 connectors and components, and job versioning. TIBCO also offers an open-source business intelligence solution we’ve covered in a previous resource.

Tokern

Tokern is an open-source data governance framework that lets users comply with regulations and protect critical data from insider threats. The solution features a data dictionary to create and manage a single-source of truth, a data catalog for databases and filesystems, data lineage tracking across your data infrastructure via interactive graphs, and the ability to manage users and access control to data in AWS Glue using familiar SQL statements.

Truedat (Bluetab Solutions)

Truedata is an open-source data governance business solution tool developed by Bluetab Solutions. The solution enables an end-to-end vision of your data from a business and technical point of view. The environment is user friendly and features visual and easy-to-understand tools. Truedat also lets users organize and enrich information through configurable workflows. Key features include end-to-end governance, expansive customization options, simple module navigation, system connectivity, cloud or on-prem integration methods, and no licensing cost.

If you’re looking for an enterprise data management solution, consult our freshly updated Data Management Buyer’s Guide.