The trouble with Audit and Logging

[JGomez]

I am curious to hear from people who work in organizations successful at "auditing" their system events, instead of simply logging events.   My organization uses the terms interchangeably, but I feel there is a distinct difference.   Logging is easy, capture events and store them for however long you want; but they are pointless to capture and store if no one is actually reviewing them, which to me is auditing.

We keep over 5 years of log data, yet no one reviews it.   We do use the logs to collect some key events for the purpose of providing metrics, but other than that, no one is looking at the data to determine what is or isn't normal events.      So why keep logging, unless you are an admin needing to troubleshoot?

[DAlexander]

I don't work in the area that does the auditing so I can't really say how they are doing it; just that it is being done thoroughly.  With that disclaimer out of the way, five years of log data seems a bit excessive if you are just keeping it for "just in case you ever need it."  Logs are meant to be audited so that would seem the most important reason for retaining them. 

Another great use for all that history is to have a valuable data set to conduct research, testing, and training on.  There are countless ways you could put that log data to work for you in a lab environment.  I spent months building a data set of over 6 million data points for my master's thesis a few years ago and can tell you it is still being used at the university today. 

[Shannon]

As @DAlexander said, logging can cater to a host of requirements, including: -

1. Log analytics: Logs can be assessed to provide information such as user behavior, patterns in the use of networks and systems, etc. This can be used to audit compliance to regulations or for a SIEM, which will correlate logs and alert you to anomalies.
2. Investigations: Tracing the causes of incidents may require that you check system logs for records. (Of course, if we could go back in time this wouldn't be needed)

Logging may not be enabled by default on all systems, since it can increase processing, in which case you'll have to configure the devices to do the needful.

As for retention, unless there's a specific requirement or your organisation is blessed with abundant storage, there's no need to retain the logs for such a long time.

[RobertWohllaib]

Hey Boss,

So yeah 5 years is ridiculous.  Let's get that out of the way.

So what's the advantage of JUST logging?  Well according to the last Mandiant Annual Cyber Security report dwell time for an intrusion is around 100 days, having logs lets you try to track down when you were initially compromised once you know you have been.  That's nifty.  There nice for troubleshooting network programs, and they can be used to have some sort of "baseline" for what normal is to identify anomalys..if you ever audited them.

Really though, if you're already logging but not using them for anything security related implementing that's the easy part.  Assuming you're in a Microsoft environment, Microsoft even has a list of log events they think you should inspect as a likely sign of compromise, so maybe just audit for those ones to get something going.  With Windows Event Forwarding you could even setup a "Log Collection Server" to forward likely signs of compromise to from your workstations, domain controllers, etc. for security to have a one stop shop to look at in an afternoon. 

If you don't have that kind of budget, which I know I don't, you could still work some sort of "Known bads" automated solution for any centralized logging solution pretty quickly.

[CISOScott]

So if you log without auditing, what is the purpose?

You can't audit what you don't have. So the purpose of retaining the logs is so that you have the POTENTIAL to audit it, IF NEEDED. While that may seem pointless it does provide at least the ability to audit in the case something arises. I would rather have the data and not need it than need the data and not have it.

[CISOScott]

---

@JGomez wrote:

    So why keep logging, unless you are an admin needing to troubleshoot?

---

Here is a real life example of how having the ability to audit when needed helped me out. Our agency has our traffic pass through another agency which is managed by that agency's SOC. Sometimes the SOC will pick up malicious activity (virus) and generate a SOC alert. We were averaging about 5 per year. However our local workstations were locally managed and our endpoint protection (EP) did not feed into the SOC. We had over 250 virus infections that were taken care of by the local team. The CIO would get the SOC alerts so she thought we were only getting about 5 "viruses" a year. I was able to go into the EP console and go through the logs and show that we actually had over 250+ that were being dutifully taken care of by the local staff. She was shocked that she wasn't being notified about them and I explained that the IT people just thought they were doing their job and since they weren't considered security staff they didn't tell anyone. The previous CISO didn't bother to investigate either and had trouble securing security staff due to the lack of being able to prove why more security staff was needed.

By having the log data for this and some other things, I was able to prove that there were lots of security items being taken care of and there were some more that were lacking that needed attention. Having the data helped me prove my case and I was awarded more security staff. I am glad it was there to be found.

[wimremes]

Logging is useful. Audit/Monitoring in the way that we - in security - currently understand it is not anymore. 

Audit/Monitoring was defined in a time when systems weren't as complex as they are now. It was fairly easy to develop rules that allowed us to look for specific events. In complex systems this isn't true anymore. Enter Systems theory. What we are looking for are manifestations from the system that indicate a certain expected state or a deviation thereof. We're still logging, as in collecting data from systems, but now we're talking Observability. There's a growing corpus of literature out there. The article below might be a good start:

https://medium.com/@steve.mushero/observability-vs-monitoring-is-it-about-active-vs-passive-or-dev-v...

[joseksamuel]

'Audit logs' is a legacy system configuration term that has carried on. System event logs are used to audit or investigate an incident, however 'Assurance Audits' by internal or external auditors testing control/design effectiveness generally do not perform log reviews; Operationally the SEIM performs the review and analysis role and provides alerts for the Sec Ops/Incident  team to respond to.