Don’t go broke trying to look secure

In my experience, IT related ideas often start with good financial intentions that then escalate wildly into a money pit you can’t quite bring yourself to admit defeat on. At home, that means explaining to your other half why the Alienware laptop and seven Raspberry Pis are more important than food; at work it’s customary protocol to beg a friendly CIO/CISO for any budget that can be requisitioned from the great money tree in the sky to help the cause.

SOCs are a case in point. They always seem to turn out as extraordinarily expensive for what they do. But why? How can it be – in this age of open-source software, cloud resources sold for pennies and Google as your guide – for SOCs to always be like this?

The answer: it’s because they are far harder to operationalise than anyone realises. The normal parallel that people make is to an IT project – it’s a database, a bit of compute time, and a quick configuration change on a few firewalls. Costs for that kind of IT problem are small (and shrinking), so that’s what I’ll predict my SOC to cost. Right?

Wrong! And not because the IT problem was wrong per ce, but because the key factors that drive cost aren’t being recognised:

• Scale: That database and compute is all going to be exponentially more than you expected.
• Process: This isn’t another IT project. It doesn’t draw on the same ITIL processes and needs tailored ones.
• Time: SOCs take a long time. Somehow you can’t do it quickly however hard you try (see my past blog on this). That’s time for SOC staff, professional services, and from company internal SMEs. Time costs money.
• Complexity: SOCs mitigate threats. Plural. It’s not a single ‘build’ because threats change. And it’s not something engineers can periodically do in their spare time; it’s a full time task to maintain tools to just stay relevant, let alone improve defences.
• Skills: For reasons I have never understood, security tools are incredibly complex to use. This drives up resource costs immensely, along with training costs (something I touched on in my previous blog).
• Licenses: If you don’t go open source (and in my next blog I will delve into whether you should) then prepare for a shock when licenses are purchased for SOC tools!

Let’s drill down into three areas that can each be a cause for skyrocketing costs.

Choosing your tools wisely

The spend on tooling is the most visible cost of a SOC project. Licenses spiral (see below on that) and tool after tool is installed just to address the poor SOC detection rate. Why do we see these tools “fail” so often? Usually, I see poor architecture and engineering decisions as the root cause and thus each tool tried will continue to fail.

An absolute must is an expertly maintained SIEM, which in my experience can achieve ~99% detection of threats that actually manifest in most enterprises (noting that APT activity that is really undetectable by SIEMs is a much rarer occurrence for most organisations than a CISO probably fears), delaying your need to spend big on other tools.

It’s also wise to pick tools that aren’t absurdly complex and need 6 months of training or, if you must, invest early in simplification. That can be via simple 3rd party dashboards that abstract the complexity away, or it could be through the automation of complex tasks using a SOAR so an analyst can do the same tasks with less training. That early investment will reduce the time for new starters to be productive, lessen the impact of inevitable analyst churn, and reduce your training spend.

Finally, use open source to cut costs when it’s beneficial, without being blinkered on the flip-side of open source that necessitates a greater number of high skill engineers, which often costs more. It’s not always the miracle cure people hope.

Choosing your data wisely

Ask a product vendor of any kind of detection or data platform and the answer to your SOC problem will be to bring every last fragment of data into their platform. But then their sales commission is based on the size of the license they sell, and the pro-rata annual support contract they upsell, so why would they advocate anything else?!

There is some truth to this, in that it is vital that granular historical log data is available in the event of any kind of significant incident, but nowhere does it say that you must centralise the data in the SOC – let alone in a single platform. Store it wherever is cheapest, and don’t worry about performance. Keep the expensive licenses for the hard graft and only bring the data you need for the detection content you have identified as necessary (read more here), and you will vastly reduce your license outgoings.

There’s no substitute for experience

Everything above is written from a perspective of experience. Mistakes I made and watched others make over more than two decades. Your SOC will need to be personal to your needs, led by architects and specialists who have faced this before. It will be expensive, but there is a place for frugality in procurement and a place for heavy investment. You don’t need to have an industry partner to help you navigate that, but sometimes doing it in-house is just too hard – especially if time is tight.

And it’s not just at the coalface where you need experts; watch for internal pitfalls. History is full of SOCs shaped by procurement processes designed to select a partner or software/service supplier that ticks the scoring boxes with a good story of huge wins and low prices in its ITT response – an ITT that itself was written with well-meaning but poorly articulated requirements vendors just delight in responding to and thus compound the cost issue. So often I see these perpetual cycles of new vendors coming in with promises of vast SOC improvements and, after cosmetic enhancements fail to underpin any kind of long-term improvement, a year later a search for a replacement supplier looms once again.

Employing an independent and experienced “client’s friend” fighting in the organisation’s corner may help mitigate this by ensuring that suppliers are sought, selected and held to account by someone who knows as much or more than they do. I very rarely see this approach adopted, which is a shame.

Whichever route you take, SOCs are expensive playthings. But with experienced leadership, your spiralling costs can be tamed without sacrificing long-term quality. In-house or external, just be certain your experts can confidently explain the challenges, using experience not Google!