{"id":2049,"date":"2025-03-11T20:37:58","date_gmt":"2025-03-11T20:37:58","guid":{"rendered":"https:\/\/solutionsreview.com\/thought-leaders\/?p=2049"},"modified":"2025-03-14T20:40:04","modified_gmt":"2025-03-14T20:40:04","slug":"the-usage-of-audit-analytics-ai-in-2025","status":"publish","type":"post","link":"https:\/\/solutionsreview.com\/thought-leaders\/the-usage-of-audit-analytics-ai-in-2025\/","title":{"rendered":"The Usage of Audit Analytics & AI in 2025"},"content":{"rendered":"

\"\"<\/p>\n

I recently attended the Audit Analytics & AI Spring Mini Conference hosted by Trent Russel at Greenskies Analytics<\/a>. This conference is a bit different than others as it\u2019s a place for internal audit practitioners to share new and innovative projects they have been working on which incorporate audit and AI technologies. The internal audit community is truly inspirational in the way that they are open to helping others move into the future of auditing. I wrote an article to share some of the top takeaways that I feel to be the most beneficial to the future of auditing. We\u2019ll start off with some background and statistics.<\/p>\n

Valerie Ann Zappia, MAcc, MBA, CPA<\/a>, Internal Audit Analytics Manager at Victoria\u2019s Secret, shared with us some interesting statistics about the structure of audit teams and how they are contributing to more innovative projects through the use of data analytics. <\/span>Typically, you see about 10% of an audit team dedicated to completing the analytics on an audit team. However, this depends on the industry as we are seeing that industries such as Finance are more mature than others such as Retail.<\/p>\n

Another statistic that Valerie shared, and one that I personally have been asked, was how to measure the effectiveness of audit analytics use in the audit. I believe that this is incredibly difficult to accurately measure and requires a clear definition for \u2018effective\u2019 and \u2018audit analytics use\u2019. Gartner<\/a> conducted an Audit Department Structure and Operations Benchmarking Survey in 2022 and the top two metrics they captured were that 58% of respondents are tracking the number of audits using analytics and 51% collect success stories from audit team members. You can see from the diversity of responses how difficult this is to measure:<\/p>\n

\"\"<\/p>\n

What I found particularly interesting were the challenges that Valerie shared, mostly because a lot of these challenges have been around, and nothing particularly new has been introduced with the emergence of GenAI, aside from increased security concerns. Data quality will always be a challenge and will only be solved if companies focus on fixing the issue at the source. It\u2019s challenges like these that increase the time and resources it takes to complete an audit because, typically, 80% of the time is spent on data cleaning.<\/p>\n

If you don\u2019t have complete and accurate data, you aren\u2019t going to produce useful results. Resistance to change has always been around as well, it just shifts focus depending on the flavor of the day. Additionally, I feel that the skills gap will always be a concern especially if we continue to see a lack of priority placed on proper training, development, and hiring.<\/p>\n

I very much enjoy presentations that talk about frameworks, maybe that\u2019s all auditors, but I was happy to hear from\u00a0Antonio T.U. Ramsey, CIA, CISA, CRISC, CRMA, CFE<\/a>, Director of Internal Audit at San Antonio Water System, who walked us through how his team implemented continuous auditing using the CRISP-DM Framework. CRISP-DM stands for the CRoss Industry Standard Process for Data Mining and is a great framework for data analytics projects. \u00a0Antonio points out that one of the best things about this framework is that it\u2019s tool agnostic and it was really easy to adapt to the needs of internal audit to provide more coverage and better insights to the business.<\/p>\n

One of the challenges he faced during this implementation was the general hesitancy from management around continuous assurance. \u00a0He\u2019s heard things like,<\/p>\n

\u2018Continuous assurance is like having a nosy neighbor-always watching, never sleeping, and ready to report any suspicious activity before you even finish your coffee.\u2019<\/p><\/blockquote>\n

I\u2019m sure us auditors resonate with this sentiment; I know I do.\u00a0 But Antonio was prepared for this and knew he needed to find the \u2018Perfect Storm\u2019 to gain management\u2019s support. He was able to almost immediately get this buy-in after presenting a solid framework, a well-defined process, and a focus on a high-risk business area.<\/p>\n

Antonio shared some really great lessons learned and takeaways, like understanding the limits of collaboration and looking for ways to sell. \u00a0But the best quote, IMO,<\/p>\n

\u2018Document like you\u2019re leaving tomorrow\u2019<\/p><\/blockquote>\n

As an auditor, I know how important documentation is to have but also how difficult it is to create. \u00a0Lastly, Antonio pointed out that having the courage to learn is invaluable, especially in the field of auditing.<\/p>\n

Speaking of learning, I\u2019m noticing that one of the most popular tools that auditors are learning how to use is Copilot. \u00a0I was excited to hear from\u00a0Michael Monwuba<\/a>, because who better to talk to us about Microsoft Copilot than the Senior IT Audit Manager at Microsoft!<\/p>\n

Copilot is integrated with Microsoft 365, so you basically have an assistant to help you with all your work. \u00a0I emphasize \u2018assistant\u2019 here, because Copilot can\u2019t do everything for you and its output needs to be verified. \u00a0Michael shared a wonderful analogy, describing Copilot as the staff auditor. \u00a0Think about when you hire a new staff auditor, do you let them run free and sign off on their own audit reports? \u00a0Absolutely not! \u00a0You need to give your staff auditor clear and detailed instructions so they know what to do.\u00a0 But instructions can be interpreted in different ways, and especially since staff auditors are very new, they don\u2019t have the expertise to get things perfect (no one does, really).<\/p>\n

Therefore, when they complete the assigned task, you must review it to make sure that it meets your expectations, it\u2019s accurate, and the job is complete. \u00a0If not, you need to give the staff auditor additional instructions to fix any mistakes and finish the task. This is the same with Copilot. \u00a0It might not get things right the first time, and you will always need to review and verify the output to make sure that it meets your criteria. \u00a0It\u2019s never a copy\/paste exercise.<\/p>\n

Michael also touched on the security aspect of this tool, and reminded us that you should be cautious about what data you’re sharing with your Copilot assistant. \u00a0What I appreciate about this tool is that Microsoft thought about security and created different modes. \u00a0Specifically, there is a \u2018Web\u2019 mode and a \u2018Work\u2019 mode. \u00a0The Web mode is much like ChatGPT, where the tool performs general web searches and casual inquiries. \u00a0With this mode, it\u2019s important to NOT share any work-related data. \u00a0The Work mode is the more secure mode. \u00a0This is integrated with Microsoft 365, which provides access to enterprise-focused features. \u00a0 Meaning any Microsoft product that you have at your company, you have a built-in Copilot assistant to help you with your work tasks. So you can use it for things like drafting emails in Outlook, summarizing meetings in Teams, analyzing data in Excel, and building presentations in PowerPoint.<\/p>\n

But remember, you are the pilot, and the Copilot is\u2026 the co-pilot. As the pilot, you are in control and need to monitor, evaluate, and adjust any inefficiencies or inaccuracies.<\/p>\n

I love this picture that Michael shared:<\/p>\n

\"\"<\/p>\n

Atanas Bosakov<\/a>, Internal Audit Analytics & Automation Manager at IGT, brought an interesting twist by sharing some perspectives about open-source AI as a more competitive alternative.\u00a0 He agrees that these AI tools have been \u2018life-changing\u2019 and a \u2018game-changer\u2019. \u00a0However, AI \u2018is a necessary condition to compete, but it is not a sufficient condition to differentiate yourself in the market\u2019. \u00a0We\u2019ve been seeing more and more companies market their products as having AI capabilities, so I agree that this isn\u2019t such a differentiator anymore. \u00a0Atanas says the competitive advantage now comes from these open-source LLMs used in combination with proprietary data.\u00a0 With open-source LLMs, you\u2019re able to download the model on your local machine, which is a secure alternative since you don\u2019t even need access to the internet to use it.\u00a0 He also mentioned that this might help solve the issue of hallucinations. \u00a0If you\u2019re interested in looking into this, you can download these open-source AI models from ollama<\/a>.<\/p>\n

Daniel McCarville<\/a>, Arch’s Internal Audit Manager, gave a particularly fun presentation. Daniel is known as Bendord\u2019s guru and is currently working on a book about Benford\u2019s Law. This was my very first introduction to using data in audits, so I\u2019m extremely excited for this book to come out!<\/p>\n

Daniel focused on the fact that we need to find ways to leverage our creativity better so that we can get transformational value out of using GenAI because we aren\u2019t there yet. \u00a0Creativity is the intersection of expertise, novel thinking, and motivation. \u00a0You need to be comfortable with the tools, techniques, and subject matter while also having the ability to generate new ideas. \u00a0But you aren\u2019t going to achieve this if you lack the motivation. What helps is if you\u2019re in the right environment, one that provides you with freedom in how you work, allows you to collaborate, accepts risks, and rewards those who innovate. \u00a0It also helps immensely if you\u2019re working on a diverse team because everyone has something unique to contribute.<\/p>\n

He also talked a bit about DiSC, which is a personality model that describes four main styles of human behavior. \u00a0D\u2019s are people motivated by success, I\u2019s are motivated by inspiration, S\u2019s are motivated by stability, and C\u2019s are problem solvers.<\/p>\n

Which DiSC type are you?<\/strong><\/h4>\n

Charles King<\/a>, Partner at KPMG for US AI in Internal Audit had the most unique presentation, it wasn\u2019t a presentation at all. \u00a0We went directly into project mode and built an AI agent. \u00a0AI Agents are basically software systems that use AI to complete tasks on behalf of users. \u00a0We specifically focused on building a declarative agent, which is part of Microsoft 365 Copilot. \u00a0Charles mentioned that to build an agent, you do need to have some technical skills as well as the ability to leverage your creativity. \u00a0If you\u2019re interested in building one of these AI agents, I noticed there are a lot of tutorials available on YouTube.<\/p>\n

I love the variety of tools, approaches, and perspectives shared during this conference. \u00a0I think because Microsoft has integrated Copilot into their software, most audit teams are starting to leverage these capabilities to help with their typical audit tasks so I thought it would be helpful to share a compiled list of various audit tasks that were mentioned for Copilot:<\/p>\n