Regardless of industry, enterprises generate a frankly overwhelming amount of unstructured data; the largest corporations produce 10 terabytes of plaintext data a month between traffic monitoring, server activity, and user interactions. To make sense of this information, many enterprises have utilized traditional log management solutions.
Log management is designed to collect information from disparate sources, centralize it, and make it available for examination and threat hunting. It’s a vital component of digital forensic investigations and fulfilling industry and government compliance mandates. However, these solutions often result in unforeseen challenges that make them a headache for your cybersecurity team and for the rest of your business.
Here are the 4 most common challenges in traditional log management:
Log Management Can’t Distinguish Good and Bad Activity
Traditional log management’s job to collect data, and therefore it generally can’t make the distinction between data that results from everyday business activities and data that are red flags for malicious activity.
Furthermore, it generally won’t alert you to major security events, such as a hacker infiltrating your enterprise’s network. Even if it could, some threat actors will disguise their actions as normal behavior to escape notice and blends in with normal data.
In a similar manner, when log management solutions collect data, it does not correlate that data in a sensible manner in the centralized “bucket.” Therefore the collected logs will require a specific search language to make sense of them and allow your team to find ongoing attacks or breaches. Of course, that hinges on if your cybersecurity team knows what to look for in the first place. Log management can make finding serious threats in your data logs like finding a needle in a haystack.
By the same token…
Automation Doesn’t Mean Hands-Off
Traditional log management solutions are typically automated, generating logs by machine processes for central storage. But this creates a bucket filled with an overwhelming volume of logs that require human agency to examine for digital threat hunting. Some enterprises don’t even pretend to make the effort to do so, simply allowing the logs to accumulate and potential security events to continue unchallenged on their servers.
Log management necessitates time, expertise and resources devoted to it to do it justice, and to get the true benefits out of it in a security analytics context.
The reason for this is simple…
Log Management Lacks of Analysis or Customization Capabilities
Traditional log management solutions are designed to do their simple task well, but aren’t designed to do anything else. But if you want to utilize the collected logs to extract key metrics from your system, mandating your enterprise’s cybersecurity team to dive into them head first to extract those metrics. This make compliance more challenging than anticipated, as finding the relevant compliance data may require serious manual correlation efforts; this constitutes another investment of time and resources.
Additionally, traditional log management will have trouble with custom log formats, which can make the job of your security analytics team much harder than it already is to collect the data they need.
Collecting Logs From The Cloud is a Hassle
Part of this challenge may be the assumption that cloud services will monitor activity and collect data for your enterprise. This is generally a misunderstanding of those services. Whatever log management solution you enact, you should treat all of your data as if it is still on your servers. After all it is still your data, just located elsewhere.
Because traditional log management solutions are often insufficient for enterprise needs, many professional IT teams have turned to SIEM solutions instead. While SIEM will also mandate time and resources to function properly, it has the security analytics, correlation, and customization capabilities to make threat hunting and compliance much easier. If your enterprise is still employing a traditional log management solution, it may be time for a switch.
Latest posts by Ben Canner (see all)
- Key Findings from Gartner’s 2020 Market Guide for SOAR - September 23, 2020
- Top 5 Cybersecurity Intelligence Books for Professionals - September 21, 2020
- Top Five SIEM Books for Cybersecurity Professionals - September 17, 2020