Log analysis - Use your logging information
For a client I am implementing a Log analyses solution. All logging of all servers, network components and appliances are sent to one system. From this system, reports can be generated with log information and the logging can be used for analysis.
There are two reasons for using logdata:
- Reports can be generated about (security) trends;
- Logs can be used to investigate incidents. They could be used for evidence in forensic analysis, when someone broke company rules or the law.
Log analysis solutions are regularly used for compliancy reasons, like SOX, COBIT or BASEL-II demands, or to prove to auditors that the company is compliant to the ISO/IEC 17799:2005 Code of practice for Information Security Management.
While most log analysis solutions can be used for alerting in real-time, for instance when something suspicious happens, or when a threshold is reached, I think log analysis is something fundamentally different than monitoring. Both have different goals.
Monitoring systems, like IDS systems or SNMP based systems are real-time systems. As soon as something happens, an alarm goes off.
Log systems are meant for analyzing events afterwards.
Log analysis solutions are part of SIEM (Security Information and Event Management) solutions. Roughly the market is split into two parts: software running on servers and hardware appliances. Net Report is an example of a software solution, Loglogic is an appliance. Snare Server can deliver both.
Usually log systems deal with large amounts of data (having gigabytes of logging per day is not unusual for a large environment). This means the solutions have to be designed for storage, bandwidth and fast searching in large amounts of data.
Reports
Log reports can be used to give a quick overview of events. Some examples are:
- The amount of port scans a firewall experienced over a given period of time;
- Which stations performed more than 5 un-successful login attempts;
- What is the top-10 websites that are visited during work-hours;
- The amount of emails sent (internally and externally) during a typical business day;
- What is the account of which the password is changed the most often.
Based on these reports incident investigations can be started.
The reports are made from data in a database, which contains aggregated logdata. Net Report has some examples of generated reports online, check the Report and Dashboard Samples.
Investigating incidents
Because all logging of all systems are stored centrally, it is easy to get a complete view of all logging events for a certain moment in time.
Log solutions usually contains all raw logs on a large harddisk. Storing this data must be done in a way that gives enough assurance that the data is not altered and original. This way, the logdata can be used for forensic investigations and can be used in court as evidence.
Usually the data is encrypted and hashed, so it has a digital signature.
This entry was posted on Thursday 02 August 2007