The Golden Hour of Incident Response

As a CSIRT consultant, I cannot overemphasize the importance of
effectively managing the first hour in a critical incident.

Finding out what to do is often a daunting task in a critical
incident. In addition, the feeling of uneasiness often prevents an
incident response analyst from making effective decisions. However,
keeping a cool head and actions planned out is crucial in
successfully handling a security incident. This blog will elaborate
on some key points to help readers facilitate better incident
response procedures.

Preparation is essential

Before taking on any incidents, security analysts would need to
know a great deal of information. To start off, incident response
analysts need to familiarize themselves with their roles and
responsibilities. IT infrastructure has evolved rapidly over the
past years. For example, we observed increasing movement to cloud
computing and data storage. The fast-changing IT environment
frequently requires analysts to update their skill sets, such as
learning about cloud security. Consequently, analysts will need to
have hands-on practice and maintain a complete picture of the
topology of all systems. In the real world, external CSIRT analysts
should quickly identify all assets under their responsibility. At
the same time, the in-house CSIRT analysts should also actively
participate in the vulnerability management and the discovery
scanning processes.

The quality of collected information determines the outcomes of
incident response. In addition, the CSIRT analysts would also need
to understand the threats they will be facing. As defensive cyber
security technologies are upgraded each day, the threat actors are
poised to evolve. For example, according to a paper in 2020, four
out of the top ten active ransomware actors are now using the
“Ransomware as a service” business model [1]. This pattern denotes
that malicious actors will more easily deploy ransomware because of
the lack of technical requirements to leverage such attacks. After
all, CSIRT teams need to identify the primary threats they are
likely to encounter.

For example, a CSIRT specialist may see common malware and
conclude that no additional threats exist. But when this situation
arises for more sensitive scenarios, such as an attack in the
energy sector, they will have to think critically and look out for
unconventional attack methods. To effectively prepare for incident
response, the analysts need to be familiar with the infrastructure
they will be working with and the cyber security threat landscape
they will be facing.

Get robust procedures in place

Knowing is only half the battle. When the alert sounds, we need
to calm ourselves quickly and plan to answer the first question,
“what should I do in the first hour?” The paper “Phases of a
Critical Incident” refers to the first hour in a critical incident
as the “crisis phase” and is “characterized by confusion, panic,
rush to the scene, and gridlock.”[2] Well-rehearsed CSIRT analysts
do well to exercise discernment in their investigation.

On the other hand, in many scenarios, they may be prone to the
obscurity of information, the inability to effectuate a solution in
a limited time frame, and lack of operational jurisdiction. In such
times, the incident response team must take matters into their own
hands, clearly express their professional knowledge, and push
through with their operations.

When performing the investigation and root-cause analysis, the
incident response team often gets stuck on finding missing pieces
of the puzzle. These difficulties lead to doubt and indecision.

In such events, the analysts often speculate the incident to be
caused by one or more possibilities of a breach without certainty.
In these circumstances, it’s advised for them to assume the most
likely cause and act accordingly. In the first hour, time is
imperative. Like taking an exam, where time is limited, skip the
questions you’re stuck on first.

Nowadays, the incident response containment process is often
simplified due to the widely adopted Endpoint Detection and
Response (EDR) technologies, which offer network containment
capabilities at the push of a button. Nonetheless, even with
traditional network containment tools, containing the network is
not always an easy one. People do not always choose the safer
option when it is available. But as the saying goes, it’s always
better to be safe than sorry!

Find out what really happened and close the
gaps

Perhaps after one hour, there are still pieces of the puzzle
left missing. Now it’s a good idea to take some time and reflect
upon all the possibilities and work down a list.

For example, I handled a security incident where the attacker
launched a reverse shell on a server. I immediately decided to
contain the server and gathered all evidence. But my teammates and
I still couldn’t figure out how the server was compromised, so we
made a list of all the accessible services and examined relevant
logs for each service.

Initial speculations put an IT operation tool as the indicator
of compromise. But eventually, we overrode this speculation by
crossing out all possibilities and concluded that there must be an
inherent security flaw in its web service.

From time to time, during the post-breach analysis, CSIRT
analysts may encounter setbacks in connecting the dots. But the
truth will always prevail with enough patience and a correct
mindset.

What you should consider

In conclusion, effectively managing the crucial one-hour time
interval after a critical incident requires more than learning on
the spot.

In addition to technical specialties, experienced CSIRT analysts
will also benefit from extensive preparation on their assets and
their adversaries, prioritization of tasks and making quick
decisions when required, as well as being able to discern
down-to-earth facts using the process of elimination.

This is just another excerpt of the stories in the Security Navigator^[1]. Other interesting stuff
like actual CSIRT- and pentesting operations, as well as tons of
facts and figures on the security landscape in general can be found
there as well. The full report is available for download on the
Orange Cyberdefense website, so have a look. It’s worth it!

[1] Midler, Marisa. “Ransomware as a Service (Raas) Threats.”
SEI Blog, 5 Oct. 2020, https://insights.sei.cmu.edu/blog/ransomware-as-a-service-raas-threats/^[2]

[2] “Phases of a Critical Incident.” Eddusaver, 5 May 2020,
https://www.eddusaver.com/phases-of-a-critical-incident/^[3]

Note — This article was written and contributed by
Tingyang Wei, Security Analyst at Orange Cyberdefense.