Building Trustworthy AI: the Technical View

BARC’s Kevin Petrie offers expert insights on building trustworthy AI from a technical perspective. Sponsored by SAS, you can also read Data and AI Impact: the New Economics of Trust now. Solutions Review‘s Expert Network is an enterprise tech publishing platform where thought leaders turn real-world authority into visibility and authority. 

Data and AI Impact: the New Economics of Trust.

Adopters recognize trust as the biggest hurdle to successful deployment of AI. This begs the question: what makes AI trustworthy?

A reasonable answer is that trustworthy AI is accurate, consistent, relevant, secure and compliant. But as we move beyond the traditional world of tables, these characteristics become ambiguous. Frontier models specialize in text or other unstructured data whose “accuracy” depends on fine-grained assessments of the user and situation. These models, probabilistic by design, mostly generate unstructured content that they calculate is likely to be trustworthy. But no guarantees.

That lack of structure and determinism makes the trustworthiness of AI a matter of nuance and judgment. We have no clean, empirical measure of whether an unstructured input or output is trustworthy. Given this ambiguity, data teams must use many levers to ensure that AI-driven decisions and actions satisfy their business’ trust requirements. It’s not easy!

We have no clean, empirical measure of whether an unstructured input or output is trustworthy.

Let’s explore how key stakeholders can bring these levers together in governed workflows that organize, refine, and enrich selected data and metadata – also known as context – in support of agentic AI. The good news: commercial tools can automate and streamline much of the work described below. We give a detailed example here to help AI adopters understand the many calibrations needed to achieve “trustworthy” AI.

Prepare                                                                                      

Data engineers work with business managers to discover the right unstructured data sources. This might include formal documents, text entries within databases, emails with customers or other types of unstructured files. They filter, reformat, and append metadata to these files, classifying them by owner, accuracy, sensitivity, relevance to key topics, and so on. The data engineer works with a data steward to catalog the metadata and create audit trails for humans, AI models or agents. They should also run data quality checks and obfuscate personally identifiable information (PII) so unauthorized users cannot see it.

Handled well, this preparation lays a trustworthy foundation for what comes next.

Conceptualize

Data and AI/ML engineers, with the guidance of business managers, next organize data and metadata – i.e., context – into conceptual models that give AI models a consistent framework for reasoning. This includes taxonomies that classify concepts or entities into specific buckets. It also includes ontologies that go further to define categories of things, those things’ key properties and the rules that organize their relationships to one another. An ontology accommodates new facts, concepts, and entities as they arise, giving structure to the data that flows through the business.

Conceptual models such as these integrate disparate data points into a cohesive, reliable, trustworthy view of the business.

Taxonomies and ontologies give AI models a consistent framework for reasoning.

Retrieve

Data and AI/ML engineers, perhaps with the help of a data scientist, next configure the mechanisms that retrieve data for consumption by agents. Multimodal retrieval helps derive meaning from multiple angles. A semantic search of vector embeddings finds text based on its similarity to earlier searches, while keyword matching finds exact phrases and traditional SQL queries pull up verifiable facts from tables. In each case, metadata guides the agents to the most relevant, accurate inputs. It also informs access controls to ensure only authorized parties can see or handle PII.

Reason, Decide, Act

All this prepared, conceptualized, and retrieved context enables agents to perform the difficult task of reasoning as they answer questions and execute instructions from humans or other agents. It is less likely to hallucinate, more likely to make sound decisions and more likely to take safe actions because its inputs are accurate, consistent, relevant, secure and compliant. Data scientists and AI/ML engineers should further safeguard agent actions with careful monitoring, escalation paths and kill switches for impermissible behavior.

Accurate, consistent, relevant, secure and compliant AI inputs drive sound decisions and safe actions.

AI creates astonishing possibilities for efficiency and productivity. But it makes governance less efficient in the near term because business and technical teams alike need to think carefully about how to achieve trustworthy AI. Designing a context workflow as shown here can get you started. To explore this topic from a strategic and business perspective, you can read Data and AI Impact: the New Economics of Trust.