Sorting Through Haystacks to Find CTI Needles

Clouded vision

CTI systems are confronted with some major issues ranging from
the size of the collection networks to their diversity, which
ultimately influence the degree of confidence they can put on their
signals. Are they fresh enough and sufficiently reliable to avoid
any false positives or any poisoning? Do I risk acting on outdated
data? This difference is major since a piece of information is just
a decision helper, whereas a piece of actionable information can
directly be weaponized against an aggressor. If raw data are the
hayfields, information is the haystacks, and needles are the
actionable signal.

To illustrate the collection networks’ size & variety point,
without naming anyone in particular, let’s imagine a large CDN
provider. Your role is to deliver, on a massive scale, content over
HTTP(s). This attracts a lot of “attention” and signals, but only
on the HTTP layer. Also, any smart attacker will probably avoid
probing your IP ranges (which are public and known in your AS).
Hence, you only receive the indiscriminate “Gatling guns” scanners
or direct attacks over an HTTP layer. This is a very narrow
focus.

Now if you are a large EDR/XDR or whatever glorified antivirus,
you also can argue that you have a huge detection network spanning
million of devices… Of wealthy enterprises. Because let’s face it,
not every non-profit, public hospital or local library can afford
to pay for those tools. Hence you potentially only see threats
targeted at sophisticated actors, and mostly the ones carried by
malware on LAN machines.

On the honeypot front, there is no silver bullet either. The
“Gatling guns scanners” represent the background radioactivity of
the Internet. A sort of static noise which is constantly present in
the surroundings of any Internet-connected device. Here, the
problem is rather that no decent cyber criminal group will use any
meaningful resources to target a honeypot machine. What’s the point
of investing some DDoS resources in knocking down a straw dummy?
Would you use any meaningful exploit or tool, let alone burn your
IP, on a “potential” target? Honeypots collect “intentions”,
automated exploitation, something along the lines of “this IP
wants to know if you’re (still) vulnerable to log4j“.

It can be interesting to a certain extent but it is limited to
low-hanging fruits. Also, your diversity is limited by your
capacity to spread in many different places. If all your probes
(honeypots) are sitting over ten or worse, just 3 or 4 different
clouds, you can’t see everything, and you can be “dodged”, meaning
criminals can voluntarily skip your IP ranges to avoid detection.
You also need to organize your deployment system for every
platform, and yet you’ll only see the IP not dodging GCP, AWS, or
whatever cloud you’re working with. And since those providers are
no NGOs, your network size is also limited by…money. If a fully
automated HP running on XYZ cloud costs you $20 monthly, your
pocket must be deep to run thousands of them.

Establishing a counter-offensive

To curb the trajectory of mass cyber criminality, we need to act
on a resource that is limited in essence, otherwise, you cannot
organize a proper “shortage”. The famous Conti-Leaks cast an
interesting light upon the actual pain points of a large cybercrime
group. Obviously (crypto) money laundering, recruitment, payrolls,
the classical ones you’d expect. But interestingly enough, when you
read the exchanges on their internal chat system, you can see IP,
changing them, borrowing, renting, cleaning them, installing the
tools, migrating the ops and C2, etc. is … costly. Both time &
money-wise.

There are nearly infinite variations of hashes and SHA1 offers a
space of 2^160 possibilities. So collecting them is one thing, but
you’re almost sure any new malware variation will have a different
signature. As we speak, most of the CI/CD procedures of any decent
cyber criminal group already include the modification of one byte
before sending the payload to a target.

Aiming at domain names is fighting against an infinite space in
size as well. You can book domain1, domain2, domain3, etc. There is
technically no limit to the number of variations. There are smart
systems out there, protecting your brand and checking if any domain
names similar to yours have been booked lately. These
pre-crime-style systems are very helpful to deal with an upcoming
phishing attempt. You start to be proactive with this kind of
stance & tools.

It’s anyway useful to track & index malevolent binaries based on
their Hashes or the C2 they try to contact or even indexing IP
trying to auto-exploit known CVE, but doing so is a rather reactive
stance. You don’t strike back by knowing the position or tactic of
the enemy, you do so by crippling its offensive capabilities, and
this is where IP addresses are very interesting. The system is
decades old and will still be there after us. It’s

Now there is a resource that actually is in scarcity: IPV4. The
historic IP space is limited to around 4 billion of them. Bringing
the fight to this ground is efficient because if the resource is in
scarcity, you can actually be proactive and burn IP addresses as
fast as you are aware one is used by the enemy. Now, this landscape
is an ever-evolving one. VPN providers, Tor, and Residential proxy
apps offer a way for cybercriminals to borrow an IP address, let
alone the fact that they can leverage some from already compromised
servers on the dark web.

So if an IP address is used at é moment in time, it’s possible
that it isn’t anymore the next hour and you then generate a false
positive if you block it. The solution is to create a crowdsourcing
tool protecting all sizes of businesses, across all types of
places, geographies, clouds, homes, private corps DMZ, etc., and on
all types of protocols. If the network is big enough, this IP
rotation isn’t a problem because if the network stops reporting an
IP, you can release it, whereas the new one rising in a number of
reports needs to be integrated into a blocklist. The larger the
network, the more real-timish it becomes.

You can monitor almost any protocol except UDP-based ones, which
must be excluded since it’s easy to spoof packets over UDP. So by
considering reports on a UDP-based protocol for banning an IP, you
could easily be tricked. Other than that, every protocol is good to
monitor. As well you can definitely look for CVE but, even better,
for behavior. By doing so, you can catch business-oriented
aggressions that may not only be CVE based. A simple example,
beyond the classical L7 DDoS, scans, credential bruteforce or
stuffing is scalping. Scalping is the action of auto-buying a
product with a bot on a website and reselling it for a benefit on
eBay for example. It’s a business layer issue, not really a
security-related one. The open-source system CrowdSec was designed
exactly to enable this strategy.

Finally, for the last two decades, we were told, “IPV6 is
coming, be ready”. Well… let’s say we had time to prepare. But it’s
really here now and 5G deployment will only accelerate its usage
exponentially. IPV6 changes the stage with a new IP addressable
pool as big as 2^128. This is still limited in many ways, not the
least because all V6 IP ranges are not fully used yet but also
because everyone is getting many IPV6 addresses at once, not just
one. Still, we speak about a vast amount of them now.

Let’s couple AI & Crowdsourcing

When data start to flow massively from a large crowd-sourced
network and the resource you try to shrink is getting larger, AI
sounds like a logical alley to explore.

The network effect is already a good start on its own. An
example here could be credential stuffing. If an IP uses several
login/pass couples at your place, you’d call it a credential
bruteforce. Now at the network scale, if you have the same IP
knocking at different places using different login/pass, it’s
credential stuffing, someone trying to reuse stolen credentials in
many places to see if they are valid. The fact that you see the
same action, leveraging the same credentials from many different
angles, gives you an extra indication of the purpose of the
behavior itself.

Now, to be honest, you don’t need AI to sort out Credential
bruteforce from Credential Reuse or Credential stuffing, but there
are places where it can excel though, specifically when teamed with
a large network to get heaps of data.

Another example could be a massive internet scan, made using
1024 hosts. Each host could scan only one port and that would
likely go unnoticed. Except if you see, in many different places,
the same IP scanning the same port within a similar timeframe.
Again, barely visible at the individual scale, obvious on a large
one.

On the other hand, AI algorithms are good at identifying
patterns that wouldn’t be visible if you look only in one place at
a time but blatant at the scale of a large network.

Representing the data into appropriate structures using graphs
and embeddings can uncover complex degrees of interaction between
IP addresses, ranges, or even AS (Autonomous Systems). This lead to
identifying cohorts of machines working in unison toward the same
goal. If several IP addresses are sequencing an attack in many
steps like scanning, exploiting, installing a backdoor, and then
using the target server to join a DDoS effort, those patterns can
repeat in logs. So if the 1st IP of the cohort is visible at a
given timestamp and the 2nd 10 minutes later and so on, and this
pattern repeats with the same IPs in many places, you can safely
tell everyone to ban the 4 IP addresses at once.

The synergy between AI and crowd-sourced signals allows us to
address each other’s limitations effectively. While crowd-sourced
signals provide a wealth of real-time data on cyber threats, they
might lack precision and context, eventually leading to false
positives. AI algorithms, on the other hand, usually only become
relevant after absorbing an enormous amount of data. In return,
those models can help refine and analyze these signals, eliminating
noise and unveiling hidden patterns.

There is a powerful couple to marry here.