Log Management And Data Centers
What every data center IT pro needs to know about log manage-ment.
Log management is an important aspect of IT. Let’s take a closer look at the top three use cases for log management that every data center IT professional should know: troubleshooting, security, and compliance.
Since log management solutions capture nearly every action performed by network devices, workstations, applications, and servers, they are uniquely positioned to provide value when you are troubleshooting problems, identifying root causes, or seeking to improve performance.
One common example is when a user reports a web application error. To resolve these types of problems, you can leverage a log management solution to help focus on these three basic troubleshooting best practices:
Develop a consistent description of what happened and how you can reproduce it. Of course, this is much easier said than done. In the web application example above, the complexity depends on who reported the error and whether or not the person trying to collect information on the problem has access to your log management solution. If an internal engineer is the one that reported the issue, then you will likely get everything you need quickly to reproduce the error. However, if a non-technical internal user or external user, such as a customer, reports the issue to a support team, it will likely take a significant amount of back and forth effort to collect all the data needed to reproduce the error. In either case, it’s critical to give the support team access to the log management system so they can supplement the initial report with log data.
Identify the root cause by starting with the data you have the most confidence in. Based on the data collected, you should do searches for the data you have the most confidence in, such as timestamps, URLs, error strings, etc., and keep drilling down until you identify the root cause.
Fix the problem so it doesn’t happen again. Again, this is easier said than done. You should start by looking for solutions to the problem by leveraging the log data you collected as part of the first best practice above and the root cause analysis you completed as part of the second.
Data breaches are constantly in the headlines. In fact, the Identity Theft Research Center has identified 500 breaches and over 12 million records exposed in just the first six months of 2016. Security information and event management (SIEM) solutions are a specialized log management tool that automatically detects, alerts, and responds to suspicious behavior on network devices, servers, workstations, and applications.
Top-tier SIEM solutions also provide real-time, in-memory event correlation for instantaneous detection of suspicious activity, automated active responses for threat mitigation, file integrity monitoring, threat intelligence, USB monitoring to protect sensitive data, built-in correlation rules, and “audit proven” report templates.
There are a number of best practices you should follow if you plan to use log management (or an SIEM tool) to enhance your security posture, including:
Clearly define what data you need to capture and how long you need to store it. If you are collecting data for compliance, it should be clear as to exactly what you need to collect (at a minimum) and how long you need to keep it. If compliance isn’t an issue, then you should consider storing the data for as long as it may be needed.
Normalize your data. Log data comes in many different formats, shapes, and sizes. For security, normalizing log data is especially important as it helps you better analyze and report on logs and events in detail, without getting annoyed by unreadable machine data. Syslog, event logs, and flat files should be normalized to provide a detailed account of the specific event name, insertion/detection time, source machine IP, severity, destination account, etc.
Correlation is what provides the value. Event correlation analyzes a set of related events based on rules that are used to interpret the data contained in the events. Many times, a security incident may not be evident when a single log is analyzed, but can be detected only when logs from multiple devices are correlated. Event correlation is required in order to identify and stop many of today’s most common security threats.
Monitor your data and take action on it. It always amazes me how many companies have set up an SIEM only to practically forget about it. In fact, I know of a company whose IT professionals didn’t even realize their SIEM tool had stopped working until they were getting ready to generate their compliance reports. Get familiar with your SIEM tool. Monitor it regularly and take action when it alerts you to a possible security event. When the big one hits (and it eventually will), you will be glad you followed this advice.
Log management plays a crucial role in compliance with all major commercial and government regulations, such as HIPAA, PCI DSS, SOX, ISO, FISMA, FERPA, NERC CIP, GLBA, and others. When I speak with IT professionals about why they are purchasing an SIEM or log management solution, compliance is nearly always at the top of the list.
As I recently wrote, when it comes to compliance and your data center, there are three key things you should focus on that an SIEM can help you with:
Think of regulatory compliance as a starting point on your journey to becoming more secure.
Seek to constantly improve your own security posture and better collaborate across your industry.
Move towards a continuous compliance model to help reduce and limit your exposure to compliance and security risks.
Log management solutions can be incredibly valuable to you as a data center IT professional who undoubtedly deals with troubleshooting, security, and compliance on a daily basis. Following the best practices and advice for each use case above will significantly improve the value you get from your log management tool. Of course, it all starts with getting the right log management or SIEM to begin with.