In my recent blog, I explored why Security Operation Centres (SOCs) are so expensive, and the net cost of the SIEM in terms of license, IT resources and analyst engagement is a fundamental cost driver even at the best of times. It’s time to address that bloat with a good diet!
Far better than the traditional maxim of hoovering up all data available into a proprietary data platform ‘just in case the SIEM needs it’, and then writing all the content you can imagine ‘in case a threat is real’, a better approach is to threat model your systems and be strict in processing only data related to the detection of specific threats on specific systems. That’s not just my view, the UK NCSC agrees.
But what do you do with all that other data you’ve now excluded – the noise that overnight becomes vital during post-incident analysis? You have a single data stream and you take it or bin it.
I’m a big advocate of having a data routing layer abstracting log sources from SIEM tools, allowing you to pick and mix your data into streams for different purpose toolsets. If you wanted one 5-10 years ago, you would have had to make your own in NiFi or Storm/Kafka, which were pretty tough to support. These days, purpose built products exist.
Battling proprietary backhaul fatigue
Let’s wind back to a statement above calling out how SIEMs are typical proprietary data platforms. Traditional SIEM architectures involve forward-deployed proprietary log collectors that backhaul data to the centre for processing. If you want two SIEMs, you forward deploy two sets of collectors and backhaul data twice. Want Analytics? Do it three times. Painful!
All of those suppliers will tell you that you have critical failures in their generously free of charge maturity assessment because you haven’t backhauled all your worldly data to them. So you end up dragging all that data back en masse for each individual platform. This typically results in a mess of traffic flows and interdependencies that now have to be managed and maintained!
Data routing layers – of which there are a variety of tools available – all create high performance many-to-many pipelines for log data that allow parsing, transformation, aggregation, enrichment, truncation and filtering before delivery to one or more downstream receivers.
This means that you can be very specific about (and agile in adjusting in real time) the data you feed to your SIEM. You can also then stream your bulk data to an archive for incident response and threat hunting – lower performance, long-term storage locations that cost peanuts.
You can also tidy up data before SIEM ingest, for example truncating windows logs to remove unnecessary content. Together with the hugely improved filtering, this will significantly trim license utilisation on per GB licensed SIEM platforms (which is the majority of them) and thus reduce costs.
Secondary benefits
By introducing a data routing layer, there are also some valuable added benefits that make the whole case for them even stronger.
- Preparing, and debugging, log streams in the routing layer prior to SIEM parsing is going to significantly reduce the pain of parser maintenance. No more tweaking those multi-line RegEx filters when a vendor tweaks its log syntax – just fix it in-flight back to the old format in the routing layer.
- No more complex management of forward deployed collectors for all your various products. Centralise them in the core of the SOC and manage them together, letting the data routing layer handle buffering, rate limiting and load balancing.
- Is your CISO back from a conference with a new toy to PoC? Just point it at a stream of data from your routing layer and that’s all that’s needed! Not ‘the right data’ for the PoC? Just fix it for the stressed presales guy by filtering in that extra data, and let him show the toy off in its full glory without any CISO-mandated interference in your IT estate.
- Want your alerts from your ‘can’t quite deny but want to’ shadow Azure Sentinel, XDR and incident response platforms (which you prefer) to all feed back to your SIEM? Instead of buying a SOAR (Security Orchestration, Automation and Response) solution, just get them all to log to the data routing layer (or better yet, pull via API) and that’s the integration done.
The great thing is that data routing layers tend to pay for themselves, or even give you money back over time. SIEMs and the resources they consume are so ludicrously expensive that the license saved by all that pre-processing will probably surpass the license and effort costs for the data routing layer.
That’s a get-fit routine worthy of Rocky Balboa. We could even do a montage of it!
We believe that strong digital defences come from security of both the Enterprise and the Nation
Understand the evolving threat landscape is a key part of maintaining robust defences. BAE Systems' Threat Intelligence team generate original insights through research and collaboration with customers and partners