{"id":560,"date":"2024-01-01T14:51:51","date_gmt":"2024-01-01T14:51:51","guid":{"rendered":"https:\/\/solutionsreview.com\/expert\/?p=560"},"modified":"2024-02-02T14:34:59","modified_gmt":"2024-02-02T14:34:59","slug":"data-observability-evaluation-criteria","status":"publish","type":"post","link":"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/","title":{"rendered":"Data Observability Evaluation Criteria"},"content":{"rendered":"<p id=\"e7fa\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Two\u00a0<a class=\"af is external\" href=\"https:\/\/blog.applicationperformance.com\/a-brief-history-of-application-performance-management-apm\" target=\"_blank\" rel=\"noopener ugc nofollow\">decades<\/a>\u00a0ago, if someone had questioned the need for application performance monitoring (APM) to perform deep analysis of metrics, traces, and logs, they would be eating their words today. Today, the APM (and the log management) industry is thriving with a cadre of high-profile vendors.<\/p>\n<p id=\"c005\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">As the enterprise technology focus shifts from applications to data, we are at the same inflection point for observing data, data pipelines, and event streams. The time for data observability has arrived as its own distinct product category. There is already a groundswell of product offerings accompanied by a lot of top VC dollars in this space, which makes the rationale for data observability to be a standalone category even more imminent in the year 2022.<\/p>\n<p id=\"64e2\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">As with any new product category, there will be differences in what makes up must-have features and what are nice-to-have features. Hence, this note provides an evaluation criteria. Enterprise leaders looking to deploy a data observability subsystem can use these criteria to evaluate their shortlisted vendors.<\/p>\n<h1 id=\"a378\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Key takeaways<\/h1>\n<p id=\"b40c\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The data observability space is rapidly maturing. Key takeaways include:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"784b\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">A comprehensive data observability layer extends beyond data and into the applications and infrastructure layers. When you have all three, a complete picture emerges of your data\u2019s journey and its interactions with the infrastructure. When any one layer is missing, the observed data can lead to an incomplete representation of the state of the data.<\/li>\n<li id=\"407f\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">The data observability environment should be almost invisible by providing a minimal footprint deployment that automatically scales and does not add significant performance overhead to the already stretched pipeline.<\/li>\n<li id=\"743b\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">IT professionals need ease of use and manageability of the data observability environment to ensure its strong adoption. It is hard to realize the value of data observability with limited and partial use.<\/li>\n<\/ul>\n<h1 id=\"752d\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Recommendations<\/h1>\n<p id=\"5e9f\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data and analytics professionals responsible for ensuring a healthy end-to-end pipeline should consider data observation to:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"8e7b\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Enable application modernization and data transformation activities by optimizing the data engineering team resources.<\/li>\n<li id=\"f1ab\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Reduce the cost of rework and remediation by \u201cshifting left\u201d the detection of errors and anomalies for voluminous data.<\/li>\n<li id=\"3787\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Scale the runtime environment to meet an increase in demand in the most cost effective manner.<\/li>\n<\/ul>\n<h1 id=\"b760\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Analysis<\/h1>\n<p id=\"fd49\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Observability is not a luxury. It is the window into a complex environment that is used to ensure the success of your enterprise data initiatives. It is especially critical for data infrastructures that are growing rapidly and are struggling to drive revenue and cost savings. When the environment comprises open-source and proprietary products spread across on-premises data centers and multi-cloud, observability becomes the glue to tie together various moving parts of the architecture.<\/p>\n<p id=\"f5e5\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">To learn more about data observability, please refer to these articles \u2014\u00a0<a class=\"af is external\" href=\"https:\/\/www.sanjmo.com\/what-is-data-observability\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">What is Data Observability?<\/a>\u00a0and\u00a0<a class=\"af is external\" href=\"https:\/\/www.sanjmo.com\/datatech-vibe-data-observability-accelerates-modern-data-stack-adoption\/\" target=\"_blank\" rel=\"noopener ugc nofollow\">Data Observability Accelerates Modern Data Stack Adoption<\/a>.<\/p>\n<p id=\"5e7b\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data management has been lagging industry-leading methodologies and approaches when compared to the rest of the IT stack. It is only now that data assets are being thought of as data products. Data has also finally adopted decade-old DevOps concepts and is now actively developing its DataOps practices. In the same vein, observability practices are now becoming standard operating principles for modern data teams.<\/p>\n<p id=\"2d64\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Figure 1 shows the observability space, specifically for the IT teams.<\/p>\n<figure class=\"kl km kn ko el kp dz ea paragraph-image\" style=\"text-align: justify;\">\n<div class=\"kq kr dj ks bg kt\" role=\"button\">\n<div class=\"dz ea kk\"><img loading=\"lazy\" decoding=\"async\" class=\"bg ku kv c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/max\/700\/0*2KASr2O6GCZ-Fb5t\" alt=\"\" width=\"700\" height=\"394\" \/><\/div>\n<\/div>\n<\/figure>\n<p id=\"2575\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\"><em class=\"kw\">Figure 1. Observability data categories \u2014 IT focus.<\/em><\/p>\n<p id=\"b67c\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Established observability products, such as Splunk, Datadog, AppDynamics, New Relic, and others are focused on infrastructure and operations metrics, but they don\u2019t know, or care about, about data\u2019s context or its meaning. They are in the \u201cobservability data\u2019\u2019 space, compared to the new field of \u201cdata observability\u201d. Figure 2 shows how this space is witnessing massive growth. There are already over a dozen companies offering some aspect of observability that is being leveraged by the business teams.<\/p>\n<figure class=\"kl km kn ko el kp dz ea paragraph-image\" style=\"text-align: justify;\">\n<div class=\"kq kr dj ks bg kt\" role=\"button\">\n<div class=\"dz ea kk\"><img loading=\"lazy\" decoding=\"async\" class=\"bg ku kv c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/max\/700\/0*-hTmpUniw_g39NJR\" alt=\"\" width=\"700\" height=\"394\" \/><\/div>\n<\/div>\n<\/figure>\n<p id=\"2195\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\"><em class=\"kw\">Figure 2. Data observability categories \u2014 business focus<\/em><\/p>\n<p id=\"fda5\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">This report is focused on a few of the areas of Figure 1 that make up the \u201cdata observability\u201d space, such as data operations, data quality, and data pipeline.<\/p>\n<p id=\"02fa\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data observability is the key to delivering data-intensive applications in the most cost effective and timely manner. It provides visibility into your data\u2019s movement vertically and horizontally. The horizontal axis includes all the components involved across the entire end-to-end architecture \u2014 from data ingestion to data consumption. The vertical axis includes applications, data, and the infrastructure.<\/p>\n<h1 id=\"e6f2\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data Observability Framework<\/h1>\n<p id=\"3cb3\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data observability sits in the overall DataOps workflow. In figure 3, data observability handles the first three boxes concerning identified significant events, raising necessary alerts and analyzing the causes.In the overall DataOps value chain, remediation steps would be initiated.<\/p>\n<figure class=\"kl km kn ko el kp dz ea paragraph-image\" style=\"text-align: justify;\">\n<div class=\"kq kr dj ks bg kt\" role=\"button\">\n<div class=\"dz ea kx\"><img loading=\"lazy\" decoding=\"async\" class=\"bg ku kv c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/max\/700\/1*taIEJyQWBySptVjZE3ACyA.png\" alt=\"\" width=\"700\" height=\"159\" \/><\/div>\n<\/div>\n<\/figure>\n<p id=\"6dea\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\"><em class=\"kw\">Figure 3. Data observability flow<\/em><\/p>\n<p id=\"63c9\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Figure 4 shows the major building blocks of a data observability framework. We will use this framework to build our evaluation criteria.<\/p>\n<figure class=\"kl km kn ko el kp dz ea paragraph-image\" style=\"text-align: justify;\">\n<div class=\"kq kr dj ks bg kt\" role=\"button\">\n<div class=\"dz ea kk\"><img loading=\"lazy\" decoding=\"async\" class=\"bg ku kv c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/max\/700\/0*OemLg-i5IEMY7LXh\" alt=\"\" width=\"700\" height=\"394\" \/><\/div>\n<\/div>\n<\/figure>\n<p id=\"fc95\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\"><em class=\"kw\">Figure 4. Data observability evaluation criteria<\/em><\/p>\n<p id=\"c018\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">This report is geared for the practitioners, especially data engineering teams and their managers, who are the critical bridge between data producers and data consumers. Data engineers often are trying to keep up with the demands of ever-increasing data consumers and use cases. The last thing we want is for our data engineers to get burned out, and we don\u2019t want to add the burden of trying to identify skilled talent in this competitive labor market. Ultimately, data observability provides life support to data engineers.<\/p>\n<h1 id=\"9ba2\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data Observability Evaluation Criteria<\/h1>\n<p id=\"695b\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Every evaluation criteria must start with understanding and defining one\u2019s business requirements. These requirements should help create a baseline of must-have features. However, one needs to look past the current needs and anticipate future requirements. For example, your current requirement may be based on ingesting data from dozens of relational data sources and running heavy-duty night batch transformation jobs to create dashboards. But, in the near-future, you may be asked to migrate to a real-time streaming architecture with data science use cases. At that point, you don\u2019t want to get a new data observability product.<\/p>\n<p id=\"ed2b\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Once you have created a requirement baseline, it is time to apply it against the categories mentioned in figure 1.<\/p>\n<h1 id=\"38ed\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data Sources and Collectors<\/h1>\n<p id=\"1038\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The first area to consider pertains to the richness of supported data sources. Gone are the days when organizations had a handful of data sources and most of these were structured databases. Modern pipelines frequently connect to a few dozen sources that range from structured to SaaS applications to semi-structured or unstructured data.<\/p>\n<p id=\"6312\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">If you were to issue a request for information (RFI), some questions should include:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"6973\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What data sources and destinations are supported?<\/li>\n<li id=\"5822\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What data pipeline sources are supported?<\/li>\n<li id=\"a858\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What compute sources are supported?<\/li>\n<li id=\"df33\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">How can new connectors be developed?<\/li>\n<\/ul>\n<p id=\"08dd\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">There are three types of sources which roughly correspond to data observability product use cases. Data sources connectors observe operational and analytical data sources in order to baseline data quality and usage patterns. Data pipeline sources are used to inform the data observability product on the reliability aspects and finally, compute sources help understand the operational and, hence, cost and performance perspectives.<\/p>\n<p id=\"6f44\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The most common data sources and destinations that the product should observe include:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"4f15\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Data storage (Amazon S3, Azure Data Lake, Google Cloud Storage, HDFS)<\/li>\n<li id=\"ce11\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Operational and analytical data sources (structured RDBMS, semi-structured non-relational databases, and unstructured)<\/li>\n<li id=\"7cfa\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Legacy sources (mainframes, COBOL Copybooks etc.)<\/li>\n<li id=\"4981\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Files \/ unstructured data (using FTP \/ SFTP) and in common formats like JSON and Parquet<\/li>\n<li id=\"40c3\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">SaaS data sources using APIs \/ GraphQL (e.g. Salesforce, Marketo)<\/li>\n<\/ul>\n<p id=\"8265\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Observing data pipeline sources is more complex than data stores. Older applications use proprietary drag-and-drop data integration tools, while many of the new applications just use code \u2014 Python, Java, Scala. Common data pipelines include:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"0e23\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">ETL \/ ELT (Informatica, SSIS, Talend, Pentaho, DataStage, Databricks Notebook)<\/li>\n<li id=\"f449\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Data Transformation (dbt docs, Apache Spark, AWS Glue, Azure Data Factory)<\/li>\n<li id=\"980d\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">IPaaS metadata (Boomi, IICS, Mulesoft)<\/li>\n<li id=\"7280\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Orchestration (Apache Airflow, Prefect)<\/li>\n<li id=\"29ce\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">BI query logs (Tableau, Qlik, Power BI, Looker, Mode etc.)<\/li>\n<\/ul>\n<p id=\"0dc2\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Common compute sources are also less supported than the data sources. Some of the common ones include:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"d710\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Hadoop ecosystem (Apache Hadoop, AWS EMR, Azure HDInsight and Google Cloud Dataproc)<\/li>\n<li id=\"204c\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Streaming data (e.g. Kafka, RabbitMQ)<\/li>\n<li id=\"5019\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Enterprise architecture \/ data modeling tools (e.g. erwin)<\/li>\n<li id=\"5bb6\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Machine learning models<\/li>\n<\/ul>\n<p id=\"3b88\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">An optimized built-in connector takes advantage of the sources\u2019 architecture, such as parallelization. A generic connector may be just JDBC or ODBC-based, which may not perform very well. Source systems are generally bread and butter mission-critical systems that should not buckle under the weight of observation. Hence, it is important to understand the vendor\u2019s connector strategy. Some vendors build their own connectors, while others OEM off-the-shelf ones.<\/p>\n<p id=\"6872\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The range of sources is very large, and no product will provide built-in connectors for every source. This leads us to evaluate the product\u2019s ability to develop custom connectors. Evaluate the product for its strengths in a connector development kit (CDK). The CDK is like an SDK, with libraries for different languages that allow end users to build custom connectors. If the CDK is important to your use case, stress test it to identify the time needed and the ease of developing your connector.<\/p>\n<h1 id=\"c1cc\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Monitor and Measure Data Environments<\/h1>\n<p id=\"e4e4\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The next evaluation criteria deals with monitoring. It starts with data quality, which is the most common reason for customers seeking observability solutions. However, the overall scope should also monitor how efficient and reliable the end-to-end pipeline is. Some questions you should seek answers to are:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"6960\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What are the data quality dimensions supported by the product?<\/li>\n<li id=\"ffb3\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What is the scope of data pipeline monitoring?<\/li>\n<li id=\"d998\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What is the scope of compute infrastructure monitoring?<\/li>\n<li id=\"cd8a\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">How are the monitoring tests developed?<\/li>\n<\/ul>\n<p id=\"0fd1\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data quality is one of the most critical show stoppers of any data and analytics program. It has been an issue for decades, with a well-defined set of \u201cdimensions\u201d to monitor. A data observability product should support the dimensions mentioned in table 1.<\/p>\n<figure class=\"kl km kn ko el kp dz ea paragraph-image\" style=\"text-align: justify;\">\n<div class=\"kq kr dj ks bg kt\" role=\"button\">\n<div class=\"dz ea ky\"><img loading=\"lazy\" decoding=\"async\" class=\"bg ku kv c\" role=\"presentation\" src=\"https:\/\/miro.medium.com\/max\/700\/1*of2Q7vl4DTLDKx0ZTPouzQ.png\" alt=\"\" width=\"700\" height=\"459\" \/><\/div>\n<\/div>\n<\/figure>\n<p id=\"35d6\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\"><em class=\"kw\">Table 1. Data quality dimension attributes<\/em><\/p>\n<p id=\"53d1\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data pipeline monitoring is more involved as the product must first build a baseline and then inspect the logs to identify anomalies across various parameters, such as configuration, performance, and resource utilization.<\/p>\n<p id=\"cc8a\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The pipeline monitoring should include analytical execution environments such as Spark and Hadoop programs written in Java, Scala, Python, or others, and data transformation and orchestration engines. The monitoring process should identify source schema drift and potential impact on downstream systems.<\/p>\n<p id=\"b648\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Finally, infrastructure monitoring also first builds a baseline of resource utilization under normal load. The data observability product must alert when there are usage anomalies by translating the impact into cost increases. The product should predict infrastructure usage through the use of ML techniques.<\/p>\n<p id=\"9004\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Users should have the ability to customize monitoring through various approaches, including:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"caeb\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">User interface. A wizard driven approach to build tests with no coding required.<\/li>\n<li id=\"11bb\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Programmatically. A low-code approach to write code in a declarative language such as SQL or in other languages such as Python. Users should evaluate the sophistication of built-in functions and models. The tool should provide an ability to version control (e.g. in GitHub) the code. Finally, the entire functionality should be exposed through well-documented REST APIs.<\/li>\n<li id=\"9619\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Configuration-based. Test cases could optionally be specified in YAML.<\/li>\n<\/ul>\n<p id=\"76b0\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Finally, the monitoring should be continuous, and not just batch. Users should be able to manually trigger errors to simulate failure.<\/p>\n<h1 id=\"4903\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Analyze and Optimize Data Pipelines and Infrastructures<\/h1>\n<p id=\"4106\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Monitoring the data, pipeline, and infrastructure continuously generates valuable metadata to diagnose issues, visualize the environment\u2019s health, and provide recommendations that can optimize cost and performance. Evaluation criteria for this functionality of the data observability product should provide answers to the following questions:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"8313\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">How detailed is the root cause analysis?<\/li>\n<li id=\"dbb1\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What recommendations are provided?<\/li>\n<li id=\"74ba\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What is the level of remediation \/ resolution?<\/li>\n<\/ul>\n<p id=\"e19d\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Root cause analysis of an issue should inspect the data\u2019s state and its values, its environment, the pipeline it is a part of, and the infrastructure it runs on. This complete, multidimensional analysis is required to accurately pinpoint the cause of investigation. This complete picture is needed to provide a high level of confidence into the analysis.<\/p>\n<p id=\"53db\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Techniques used to perform the analysis should include a time-series analysis of the current data against historical data, and running machine learning algorithms, such as gradient boosting to predict behavior. Developing a fine-grained lineage can provide a deeper level of impact analysis.<\/p>\n<p id=\"e2d4\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">A data observability product is not expected to remediate the identified causes, but it exposes its analysis through various means \u2014 log files and the management console. An advanced feature is when the product generates the necessary SQL statements to manually run against the underlying data.<\/p>\n<p id=\"d08e\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Analysis of causes and issues is one side of the coin; the other side is the rich set of recommendations generated by the product. These recommendations should be made at technical and at business levels. At a technical level, the products should provide recommendations on right sizing of the infrastructure. At a business level, the recommendation should also include strategies for cost reduction.<\/p>\n<h1 id=\"d878\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Operate and Manage Data Operations<\/h1>\n<p id=\"6d86\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Incident management is responding to an unplanned event and restoring the service to its operational state. To perform effective incident management, the data observability product needs to generate alerts, collaborate with other teams, and integrate with other applications. The most successful products are the ones that reduce the number of steps and the level of complexity in managing incidents and restoring health while minimizing impact on business operations. The questions that your RFI should clarify include:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"8451\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What are the alert capabilities?<\/li>\n<li id=\"a71f\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What are incident management capabilities?<\/li>\n<li id=\"e521\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What is the level of ecosystem integration, including dataops?<\/li>\n<li id=\"b6c2\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">How is collaboration achieved?<\/li>\n<li id=\"8a59\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">How easy is it to use and manage?<\/li>\n<\/ul>\n<p id=\"8755\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Alerts in typical monitoring products are a boon and a bane. On one hand, they generate critical notifications of significant events, but so many alerts get generated the receiver is quickly inundated. The latter scenario leads to \u201calert fatigue.\u201d Hence, evaluate the data observability product\u2019s ability to group and prioritize alerts. The products should provide an ability to tune system or model sensitivity to make alerts more manageable. Finally, the product should provide a workflow for configuring notifications, such as priority, frequency, consumers.<\/p>\n<p id=\"c6d3\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Incident management capabilities include the ability to annotate lineage, provide feedback loop, and open tickets in the popular workload systems, such as ServiceNow, Jira, and other systems.<\/p>\n<p id=\"33c0\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data observability products are an integral part of a larger ecosystem. This ecosystem typically includes metadata management products, such as data catalogs and data quality products. Ideally, there should be connectors that facilitate the bi-directional exchange of metadata. For example, a data observability product may highlight data quality issues, which are remediated by the data quality product, and the catalogs on both the products are updated with the new state. Other integrations in the ecosystem could be with SIEM and dataops products.<\/p>\n<p id=\"5314\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Collaboration and crowdsourcing features complete the incident management workflow by permitting various users to participate in issue resolution. The feature should allow annotation of the lineage and integration with collaboration apps like Slack and Microsoft TEAMS. Users should be able to checkout and check in assets, endorse and certify them, or rollback changes.<\/p>\n<p id=\"ceaa\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Administration requirements are like any other enterprise application. For example, the product should provide automation, logging, audit, and backup support. Automation should span the entire observability pipeline, including inferring data quality rules and actions, detecting anomalies, and initiating remediation.<\/p>\n<p id=\"5040\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Usability is one of the most critical aspects to ensure adoption of data observability products. Evaluate the product on the ease of use for the features mentioned thus far. The product should allow the intuitive creation of workflows, besides providing built-in workflows for the most common tasks. More advanced capabilities include workflows specifically for vertical, industry-specific areas, such as financial services or healthcare.<\/p>\n<h1 id=\"9a1d\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data Architecture<\/h1>\n<p id=\"e4b8\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">A well-architected data observability product seamlessly scales and performs in tandem with the growth in demand. From an end-user\u2019s perspective, the underlying architecture details are not germane, as long as the product meets, or exceeds, its expectations. Key questions that should be answered include:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"3edb\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What is the deployment architecture?<\/li>\n<li id=\"c015\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">How does the product scale?<\/li>\n<li id=\"c408\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What are the supported certifications and security mechanisms?<\/li>\n<\/ul>\n<p id=\"f217\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Deployment architectures vary from on-premises to the cloud. Most organizations today have a significant on-premises footprint, while they migrate their data to one or more cloud providers\u2019 data centers. Hence, the data observability product should provide hybrid multi-cloud support.<\/p>\n<p id=\"fd82\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Cloud deployments also vary, from IaaS to PaaS to SaaS. The basic requirement is for the product to be installed in a self-managed (IaaS) manner in the major cloud service providers, such as AWS, Microsoft Azure, and GCP. More advanced products should be available as fully managed (PaaS), or even as serverless SaaS deployments.<\/p>\n<p id=\"6d5a\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The underlying architecture comprising the data discovery, classification, metadata store, orchestration engine, and analysis \/ search technology is important to determine the product\u2019s agility in handling ever-increasing loads. Products that use open-source standards can benefit from new extensions and developments by the community. The goal of understanding the architecture is to ascertain its reliability and cost performance. There should be no single point of failure, and the architecture should enable scale out\/in and scale up\/down, with cost transparency. Finally, evaluate whether the scaling is automatic or manual.<\/p>\n<p id=\"4b89\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Security evaluation criteria are paramount. All data in-motion and at rest should be encrypted. Products offered in the cloud as a SaaS service must have SOC certification. Other certifications may be required based on the industry verticals, such as HIPAA for healthcare, FedRAMP for government, and GDPR for the EU. Finally, access to the data observability product should use role based access control and integrate with the necessary identity and access management environment.<\/p>\n<h1 id=\"89a6\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">The Data Observability Market<\/h1>\n<p id=\"2d4c\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">So far, we have looked at the evaluation criteria from the technical features and functions angle. Now, we change tracks to examine non-technical evaluation criteria. This space concerns the overall market and individual vendor\u2019s viability. As data observability is a relatively new space, these criteria become more critical than if we were looking to procure a new database management system.<\/p>\n<p id=\"8225\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">As you evaluate the vendor, look at the pedigree of the founding team. Have they lived through the trials and tribulations unique to data engineers and, hence, particularly appreciate their pain points. Engineers who have worked on an extreme scale realize the need for better observability. The questions we look to answer in this section of the evaluation criteria include:<\/p>\n<ul class=\"\" style=\"text-align: justify;\">\n<li id=\"7a45\" class=\"jw jx gx hw b hx hy ib ic if jy ij jz in ka ir kb kc kd ke bj\" data-selectable-paragraph=\"\">Why do customers choose the product and what are their use cases?<\/li>\n<li id=\"dc74\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">How viable is the vendor?<\/li>\n<li id=\"8f74\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">What is the pricing model?<\/li>\n<li id=\"e773\" class=\"jw jx gx hw b hx kf ib kg if kh ij ki in kj ir kb kc kd ke bj\" data-selectable-paragraph=\"\">How sophisticated is the support?<\/li>\n<\/ul>\n<p id=\"7d98\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Customer references provide a view of how well the product performs in production environments. Use-case driven evaluation should be preferred over one based solely on features and functions. Understand how many customers have deployed the solution you are evaluating, the time it took and overall customer satisfaction, or the net promoter score (NPS).<\/p>\n<p id=\"b3df\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Identify the financial viability of the vendor by investigating their investment history, profits, revenue, and cash reserves. Often, the emphasis is on growth numbers, but equally important is the durability of the vendor. The goal of this evaluation criteria is to understand the maturity of the product, the vendor\u2019s management team, and the robustness of the product roadmap.<\/p>\n<p id=\"0bbb\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Pricing models can be a stumbling block, or even a showstopper. The vendor should have a simplified, well documented pricing guideline. The pricing models may be based on the number of data sources, amount of data managed, the number of concurrent users, or the number of applications deployed. Ensure there is clarity on the upfront cost and the operating costs.<\/p>\n<p id=\"0636\" class=\"pw-post-body-paragraph hu hv gx hw b hx hy hz ia ib ic id ie if ig ih ii ij ik il im in io ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Finally, the support infrastructure is important for all products, but more so if the product category is new. Ensure that there is support in your deployment regions and in the language of your preference. Vendors should provide support for rapid prototyping to reduce the time to onboard the product. Investigate the existing support metrics, such as the number of tickets created, resolved, and the SLA levels. Support should span configuration, deployment, build and training.<\/p>\n<h1 id=\"8fa3\" class=\"it iu gx be iv iw ix iy iz ja jb jc jd je jf jg jh ji jj jk jl jm jn jo jp jq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Summary<\/h1>\n<p id=\"ffb2\" class=\"pw-post-body-paragraph hu hv gx hw b hx jr hz ia ib js id ie if jt ih ii ij ju il im in jv ip iq ir gq bj\" style=\"text-align: justify;\" data-selectable-paragraph=\"\">Data observability has rapidly become a must-have for data-driven organizations struggling to deliver timely insights. These companies understand the power of data to deliver not just intelligence, but competitive advantage. However, without transparency into the validity of data and the reliability of data pipelines, the best laid plans go awry. If your monitoring software bubbled up known knowns, data observability products help unearth unknown unknowns. This document provides an evaluation criteria to select a data observability product that best meets your business and technical requirements.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Two\u00a0decades\u00a0ago, if someone had questioned the need for application performance monitoring (APM) to perform deep analysis of metrics, traces, and logs, they would be eating their words today. Today, the APM (and the log management) industry is thriving with a cadre of high-profile vendors. As the enterprise technology focus shifts from applications to data, we [&hellip;]<\/p>\n","protected":false},"author":434,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[11],"tags":[],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Data Observability Evaluation Criteria<\/title>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data Observability Evaluation Criteria\" \/>\n<meta property=\"og:description\" content=\"Two\u00a0decades\u00a0ago, if someone had questioned the need for application performance monitoring (APM) to perform deep analysis of metrics, traces, and logs, they would be eating their words today. Today, the APM (and the log management) industry is thriving with a cadre of high-profile vendors. As the enterprise technology focus shifts from applications to data, we [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/\" \/>\n<meta property=\"og:site_name\" content=\"Solutions Review Thought Leaders\" \/>\n<meta property=\"article:published_time\" content=\"2024-01-01T14:51:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-02-02T14:34:59+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/miro.medium.com\/max\/700\/0*2KASr2O6GCZ-Fb5t\" \/>\n<meta name=\"author\" content=\"Sanjeev Mohan\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Sanjeev Mohan\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"16 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/\",\"url\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/\",\"name\":\"Data Observability Evaluation Criteria\",\"isPartOf\":{\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/miro.medium.com\/max\/700\/0*2KASr2O6GCZ-Fb5t\",\"datePublished\":\"2024-01-01T14:51:51+00:00\",\"dateModified\":\"2024-02-02T14:34:59+00:00\",\"author\":{\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/d3f510bbd5a4434f2da3d684f4a916ca\"},\"breadcrumb\":{\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/#primaryimage\",\"url\":\"https:\/\/miro.medium.com\/max\/700\/0*2KASr2O6GCZ-Fb5t\",\"contentUrl\":\"https:\/\/miro.medium.com\/max\/700\/0*2KASr2O6GCZ-Fb5t\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/solutionsreview.com\/thought-leaders\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Observability Evaluation Criteria\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/#website\",\"url\":\"https:\/\/solutionsreview.com\/thought-leaders\/\",\"name\":\"Solutions Review Thought Leaders\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/solutionsreview.com\/thought-leaders\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/d3f510bbd5a4434f2da3d684f4a916ca\",\"name\":\"Sanjeev Mohan\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/48b6d62967a183a096e11064bbd7ecdf?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/48b6d62967a183a096e11064bbd7ecdf?s=96&d=mm&r=g\",\"caption\":\"Sanjeev Mohan\"},\"description\":\"As an established thought leader in the areas of cloud, big data and analytics, Sanjeev has researched and provided advice on changing trends and technologies in the modern cloud data architectures. Sanjeev started his data and analytics journey at Oracle where he worked on emerging technologies and built cutting-edge solutions. Until recently, Sanjeev was a Gartner research vice president known for his prolific work and attention to detail. Sanjeev regularly presents on topics pertaining to end-to-end data pipelines and helps businesses discover what their data can do for them.\",\"sameAs\":[\"www.linkedin.com\/in\/sanjeev-mohan-498119\/\"],\"url\":\"https:\/\/solutionsreview.com\/thought-leaders\/author\/sanjeev-mohan\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Observability Evaluation Criteria","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Data Observability Evaluation Criteria","og_description":"Two\u00a0decades\u00a0ago, if someone had questioned the need for application performance monitoring (APM) to perform deep analysis of metrics, traces, and logs, they would be eating their words today. Today, the APM (and the log management) industry is thriving with a cadre of high-profile vendors. As the enterprise technology focus shifts from applications to data, we [&hellip;]","og_url":"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/","og_site_name":"Solutions Review Thought Leaders","article_published_time":"2024-01-01T14:51:51+00:00","article_modified_time":"2024-02-02T14:34:59+00:00","og_image":[{"url":"https:\/\/miro.medium.com\/max\/700\/0*2KASr2O6GCZ-Fb5t"}],"author":"Sanjeev Mohan","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Sanjeev Mohan","Est. reading time":"16 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/","url":"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/","name":"Data Observability Evaluation Criteria","isPartOf":{"@id":"https:\/\/solutionsreview.com\/thought-leaders\/#website"},"primaryImageOfPage":{"@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/#primaryimage"},"image":{"@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/#primaryimage"},"thumbnailUrl":"https:\/\/miro.medium.com\/max\/700\/0*2KASr2O6GCZ-Fb5t","datePublished":"2024-01-01T14:51:51+00:00","dateModified":"2024-02-02T14:34:59+00:00","author":{"@id":"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/d3f510bbd5a4434f2da3d684f4a916ca"},"breadcrumb":{"@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/#primaryimage","url":"https:\/\/miro.medium.com\/max\/700\/0*2KASr2O6GCZ-Fb5t","contentUrl":"https:\/\/miro.medium.com\/max\/700\/0*2KASr2O6GCZ-Fb5t"},{"@type":"BreadcrumbList","@id":"https:\/\/solutionsreview.com\/thought-leaders\/data-observability-evaluation-criteria\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/solutionsreview.com\/thought-leaders\/"},{"@type":"ListItem","position":2,"name":"Data Observability Evaluation Criteria"}]},{"@type":"WebSite","@id":"https:\/\/solutionsreview.com\/thought-leaders\/#website","url":"https:\/\/solutionsreview.com\/thought-leaders\/","name":"Solutions Review Thought Leaders","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/solutionsreview.com\/thought-leaders\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/d3f510bbd5a4434f2da3d684f4a916ca","name":"Sanjeev Mohan","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/solutionsreview.com\/thought-leaders\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/48b6d62967a183a096e11064bbd7ecdf?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/48b6d62967a183a096e11064bbd7ecdf?s=96&d=mm&r=g","caption":"Sanjeev Mohan"},"description":"As an established thought leader in the areas of cloud, big data and analytics, Sanjeev has researched and provided advice on changing trends and technologies in the modern cloud data architectures. Sanjeev started his data and analytics journey at Oracle where he worked on emerging technologies and built cutting-edge solutions. Until recently, Sanjeev was a Gartner research vice president known for his prolific work and attention to detail. Sanjeev regularly presents on topics pertaining to end-to-end data pipelines and helps businesses discover what their data can do for them.","sameAs":["www.linkedin.com\/in\/sanjeev-mohan-498119\/"],"url":"https:\/\/solutionsreview.com\/thought-leaders\/author\/sanjeev-mohan\/"}]}},"_links":{"self":[{"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/posts\/560"}],"collection":[{"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/users\/434"}],"replies":[{"embeddable":true,"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/comments?post=560"}],"version-history":[{"count":0,"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/posts\/560\/revisions"}],"wp:attachment":[{"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/media?parent=560"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/categories?post=560"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/solutionsreview.com\/thought-leaders\/wp-json\/wp\/v2\/tags?post=560"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}