A Deep Dive Into the Shortcomings of M365 Reports

CoreView’s Vasil Michev offers a deep dive into the shortcomings of M365 reports. This article originally appeared on Solutions Review’s Insight Jam, an enterprise IT community enabling the human conversation on AI.

Many organizations need insights into how Microsoft 365 services are being leveraged in their daily operations. In the most simple case, such “reporting” involves the usage data across various services as it relates to the licenses assigned to each user. After all, Microsoft 365 is a paid offering, and understandably, customers do not want to overpay for features and services they might not need or use.

Initially, these insights were gathered through custom PowerShell scripts, a significant challenge for most small and medium-sized businesses due to the required skillset. ISVs (independent software vendors) approached this gap by offering automated tools and SaaS-based reporting products. Over time, Microsoft has introduced various reporting tools, but despite these advancements, built-in reports still fall short of meeting all customer needs. Let’s review some of their shortcomings and limitations and highlight ways ISVs have provided additional functionalities on top of the built-in Microsoft 365 reports.

Accessing the Reports

Microsoft has significantly improved reporting in the 365 suite, but reports remain scattered across multiple admin endpoints, each requiring separate instructions and potentially different permissions. Even “global” roles like Reports Reader or Global Reader aren’t universally implemented. The diverse interface controls and data formats add to the complexity for users. Another commonly overlooked need is delegating report access based on business units, such as departments or countries. While administrative units allow scoping access, this feature isn’t uniformly implemented, and report support is notably lacking.

Additionally, there’s no functionality to grant access to specific reports. Admin roles provide broad access; for example, any role granting the user access to the Copilot usage report will also provide access to the Microsoft 365 active users report. This raises some privacy concerns since toggling data obfuscation affects all reports, not just specific ones.

The last use case we will mention in this section is the ability to schedule delivery of a given report, usually done using email as the medium. While Microsoft does offer this functionality for some of the compliance reports, none of the reports found in the Microsoft 365 admin center support it. In turn, customers have no choice but to manually access the report from the portal. You’d be surprised to hear how often C-suite executives or users in other roles cannot be bothered with such an approach and instead prefer a given report to end up in their mailbox (or, say, a Teams channel).

Scope and Customizability

Another major shortcoming of the built-in reports is their need for more customization. The built-in Microsoft 365 reports lack customizability, as they are predefined and do not allow modification of included objects or properties. Companies cannot add specific properties like employee ID or cost center or display properties like Department or City available in other reports. Additionally, these reports are always generated for the entire tenant. While exporting to a CSV file for filtering is possible, it could be more convenient due to the limited properties available. Creating custom reports is also not an option,
 with Microsoft suggesting PowerBI or Copilot, which require additional licenses and skills. In contrast, ISV solutions offer incredible customizability, allowing additional properties, data correlation, sorting, filtering, and custom visualizations.

Data Granularity

Built-in Microsoft 365 reports often need more granularity and provide aggregated views with detailed breakdowns. For example, usage reports per service do not specify actions and are usually limited to specific timeframes without date-specific data. ISVs can enrich such reports with data sourced from the Unified audit log datamart, additional reports, PowerShell cmdlets, or the Graph API. While it’s not always possible to go as granular as the customer might desire, the granularity level of reports provided by ISVs is usually much better than that of the built-in reports. Furthermore, built-in reports cannot consolidate data to show a single view of all data points for a user across services or provide aggregations across user sets. Conversely, many ISV products offer pivot table functionality or similar features to meet these needs, providing a more comprehensive and detailed reporting experience.

Breadth of Reporting

Built-in Microsoft 365 reports cover limited services, applications, and objects. Even with reports from the Teams admin center, Exchange admin center, Compliance portal, Defender portal, and Entra ID portal, many scenarios still need to be addressed. For instance, permission-related reports are absent, including directory-wide roles and mailbox or folder-level permissions. Reports on mobile devices, Entra ID integrated applications, service principals, and externally shared files must be included. While this data exists within Microsoft 365 services, there is no built-in report or dashboard to access it. Exporting data, even for a single user, is complex and sometimes requires convoluted PowerShell cmdlets or GraphAPI endpoints. ISVs excel here, often founded by experts with deep knowledge of Microsoft 365 services and motivated to address niche scenarios Microsoft overlooks as the financial well-being of the ISV is directly tied to the value they provide to their customers, ultimately providing more excellent value and comprehensive reporting.

Extensibility

As discussed, built-in Microsoft 365 reports need more customizability and functionality to compare and correlate data across different services, limiting the understanding of dependencies between Microsoft 365 components. Incorporating data from external systems, including other Microsoft products like Entra ID, Intune, Power Platform, or Dynamics 365, adds complexity. ISV products, however, offer a more comprehensive view, integrating data from Azure services and third-party cloud suites.

Moreover, there needs to be programmatic access to many built-in reports. For instance, no Graph API endpoints or methods exist to export Visio or Project usage data, and Copilot has limited usage reporting with no API support. Reports scattered across admin endpoints, like the Teams admin center orDefender portal, add to the difficulty. While the lack of APIs affects ISVs too, they often invest in unsupported methods to ensure their products address these scenarios effectively.

Integration with the Unified Audit Logs

ISV products often excel at integrating the Microsoft 365 Unified audit log into their reporting and automation solutions, a functionality that Microsoft provides, but third parties simply handle better. While Microsoft offers integration through Cloud App Security (MCAS), this solution is often more expensive than third-party products, which may provide sufficient features at a lower cost. Integration with the Unified audit log enhances reporting capabilities. For instance, Microsoft 365 activity reports cover only specific products and actions, limiting the ability to assess user activity comprehensively. The Unified audit log captures most user interactions, allowing for a better assessment of user activity. Additionally, sign-in events from the audit log can be correlated with activity reports to make more informed decisions, especially since Microsoft 365 usage reports can be delayed or unreliable.

Reliability

Anyone regularly accessing Microsoft 365 reports has likely encountered reliability issues, such as significant delays in report currency and missing properties. For instance, the SharePoint Site usage report has not shown the Site URL property for over nine months. While ISV products use the same data as built-in reports, they often address these issues by employing alternative methods to obtain or correlate data from other sources. Additionally, ISVs quickly inform users about data quality and integrity problems.

Cross-Tenant Reporting

Mergers, divestitures, and acquisitions often require reporting across multiple Microsoft 365 tenants, ranging from aggregated usage and adoption data to migration analytics and troubleshooting. MSPs, CSPs, and large multinational conglomerates also have these needs, which built-in Microsoft 365 reports need to meet. ISV products, however, can handle multi-tenant reporting, eliminating the need to manage multiple sets of credentials and endpoints while providing granular, RBAC-based data access. These products’ widespread “virtual tenant” concept offers customers flexibility and saves significant work hours.

Actionable Reports

Thus far, we have been focused on the “data” aspect of reporting, when in reality, nobody looks at reports just for the sake of glancing at numbers. Specific business scenarios usually need to be addressed, such as those related to the cost of service. Specific business scenarios, like service cost, require report data for subsequent processing and actions, yet built-in Microsoft 365 reports are not “actionable”. Third-party reporting products excel by allowing users to select objects from reports and perform various actions, such as removing licenses, managing permissions, assigning phone numbers, or triggering workflows. They also support bulk actions, scheduled executions, custom Power Shell-based actions, workflows, and automation, greatly enhancing productivity and efficiency based on report data.

Summary

In summary, built-in Microsoft 365 reports have several shortcomings. Still, ISVs can help address these issues by better exposing, collating, and aggregating the same data, offering more consistent and customizable reporting, including custom reports. Beyond basic metrics like mailbox sizes or WordOnline access, ISVs enhance reporting with extensibility features that support scenarios beyond license optimization, including provisioning workflows and troubleshooting access and performance issues. Notable ISV-enabled features include multi-tenant reports, granular delegation, actionable reports, and workflow integration.