Hello, this is Eva, and you're listening to the In Between Tech
and Trust podcast. In this week's episode, we dive
deep into all aspects around threat intelligence.
And for that important topic, I have Asaf kidneys with me in
conversation. And Asaf comes with over 10
years of experience in that industry.
He has worked for some of the major tech companies, for
example, Meta or LinkedIn. And currently he's building his
own platform because he's convinced that data provenance
is one of the most important aspects to provide and help us
engaging with the future. And so together we look at the
intersection of technology and trust from a threat Intel
perspective and also from a defense perspective.
We talk a lot about what it actually means to avoid threats
and also counter them, of how the operations behind it really
work and what in times of becomes most important To then
also dive deep into the actual happenings of attacks and what
happens behind the scenes. And also how to, you know,
leverage the knowledge behind it.
And we do discuss of why understanding your product and
your platform is the most important aspect for you to
enable security and also to avoid threats going forward.
Welcome Asaf and happy to have you on the In Between Tech and
Trust podcast all. Right.
Thank you so much for having me. I really appreciate it.
For the listeners that don't know you, do you briefly just
want to say a bit about your background and also with what
perspective on tech and trust you come into the conversation?
Sure. So my name is Asaf Kipnis.
I've been in, let's call it the tech space for over 12 years
now. I've been in information
security and threat intelligence integrity spanning LinkedIn,
Google, Meta 11 Labs for a little while and I've been
mostly working on the investigation space and the
threat intelligence space within Infosec and Integrity, trust and
safety. What were underlying challenges
that you keep identifying over and over again?
And it doesn't need to be dependent on the tech company.
On the contrary, is there anything that was particularly
visible for you in the tech space and also in the background
that you worked out? I think the main thing that I've
noticed throughout my career, and This is why I decided to go
build something on my own, is there's a big focus in, and this
is more in this security and safety and threat Intel space.
There's a big focus on dealing with the problem that's right in
front of our faces and dealing with the manifestation of that
problem. And what that causes is that we
end up dealing with the same problems over and over and over
again. And you can see that the
industry is very much on a reactive kind of work.
And a lot of the work that it's done is, is not learning from
what we what we identified. There's a big lack of kind of
gap and learning as we continuously fight fires to find
the next shiny thing that is in front of us.
And why do you think is that so? I think there's multiple reasons
for that. I think the most kind of like
structural reason is that it's very hard to measure what you
didn't, what didn't happen. Riding in on a White Horse and
saving everybody from a fire that looks really good, that
measures really well. You can say we took down X
amount of actors or we stopped an attack.
And so if you want to keep your team working and viable in the
company, you need to show something.
The other issue there is that I think the learning piece falls
between teams. Usually you'll have the team
that's actually doing the prevention or the detection.
You'll have the teams that are doing the analysis and the
finding and it's, it's very hard to connect between those two
teams. So a thing I've been thinking
about is how threat intelligence and Intel in general is not
incorporated into those spaces into safety and, and, and
security. It's more kind of like adjacent
and feeds the information when needed.
But there isn't a clear structure that controls the kind
of inflow or the influx of information from threat
intelligence into actual detection.
Over the years, can you to describe a bit of how threat
intelligence has potentially also changed and how systems
have also evolved over time to acknowledge potential blind
spots that they have or might not have?
So I think the way threat intelligence has changed is in a
good way and a bad way. So in a good way, I think threat
intelligence have become more front and center when when we're
working on complex actors and we want to try to understand how
actors are operating. So threat intelligence is taking
more of a kind of front of the house area.
On the flip side of of that, some companies have are using
threat intelligence just to identify new upcoming fires and
kind of and threat intelligence just stays in the point where
they're not telling you how to prevent.
They are, they're, they're focused on telling you, hey,
what's coming up right now that you need to block right now.
And they're not very well empowered to actually prevent
attacks or prevent abuse. And, and we're seeing that is in
the scam space that it's just become completely rampant and
super complex. And there is no, I haven't seen
any, any good structured way of dealing with it at scale.
Your second question was how did teams evolve?
So some teams have evolved into connecting threat intelligence
with detection and really doing their best to take all findings
and tell, take new things that are found and putting them into
into detection, putting them into the understanding of the
attackers. On the other hand, a lot of
teams have evolved into kind of working with how do we make the
metrics look really good. So and it's kind of like two
sides of the spectrum. There's a how do we work this?
And those are those are the more complex teams of how do we
actually identify what is, what is actually going on.
And on the other side, you have the teams that evolved into,
well, what can I measure? Let's just do that.
And that's because that's going to give us more funding or make
us look better. So imagine you would be head of
the threat intelligence again at one of the companies that you
work for and you have you've identified either an upcoming
someone or one that might be on the horizon.
How? What happens?
What do you do? So from a threatened
intelligence perspective, we want to understand what the
attackers are doing. They have been doing other
places. So for example, you take
investigations that were done in other companies that are
similar. You take reports from the FBI,
reports from CISA, report from anything out there that can give
you information which threat intelligence do that,
Intelligence teams do that. And then you start also you want
to understand what you are vulnerable to what, what attacks
have happened on your platform in the recent past, in the
further past that you did not identify.
You can identify now hopefully, but what did you not identify in
the past? And you synthesize this
information to basically create a threat after journey.
So we want to understand whether it's on our platform or not on
our platform. So there's the end of the chain,
which is what actually we saw and what happened.
Where's the harm? But a good threat intelligence
team or a good team will go and pick up all the pieces of how
this happened along along the way.
And the Holy Grail there is not how do we stop the attack is how
do we make this exhausting and expensive for the attacker to
get to the point of their attacking.
At Facebook, one of the teams, they had this saying it was
like, get off my platform and get off my platform says exactly
that. I just want to make it so you
just think that MAG platform is the worst place for you to be
and you'll go somewhere else. And breaking down your path
towards the attack is where that becomes more difficult because,
and the really, really good teams that do that, they focus
on the threat actor journeys and how to circumvent them along the
way. Was there an incident or a
threat in your career that was quite memorable that you have or
have either lived through or prevented, that you could say of
why it was so memorable? Yeah.
So right before the 2020 elections, we were looking into
specific actor group or a set of actor groups that in the past
were taken down by the company was was very clear that this is
we, we need this not to happen again.
This happened in 2016. We need this not to happen again
in 2020. So I received the the structure
and the resources to actually deal with that at scale, meaning
I could investigate it with the team that I had, but we also had
a direct connection to the team that was deploying detection and
operations and policy and then kind of put it all together.
That was very successful because the the intelligence part of the
team was able to identify what they're doing, identify how
they're doing it, and then immediately put that into
policy, into operational flows, into prevention and detection.
And the great thing about it is that we literally saw them
leaving the platform. They were talking to each other
and saying we don't, this doesn't work for us anymore.
We're now in 2026. Threat landscape moves faster
than ever. Where do you see the biggest
cracks in Be it detection, attribution, context,
provenance? Choose the field that you want
to explore with us. So the generic answer is AIAI
makes everything bad and scary. It's true that with the advent
of AI and we're God in the last three years, attackers are armed
with more tools that can just make them faster and more agile.
But the way I look at it is, while yes, this is a kind of
somewhat of a pivot point that attackers are getting faster,
but I feel attackers have been getting ahead of us for the at
least the last six years, if not more.
So they are already far ahead because of the way we were.
Even as defenders, we don't know our product as well as as
attackers know our product and our landscape.
And I think kind of the thing I've always said is as an
attacker, I can focus on on a target and then learn everything
about that target and just keep going.
As a defender, I need to defend everything that's coming across.
So we're already hamstrung. So I think we've always been
like that in trust and safety and in security that we're kind
of like plugging holes and trying to get ahead of attacks
now. So the attackers with AII able
to do 2 things. One, they're able to be a lot
faster. And two, they are able to
enhance their knowledge and enhance their tactics by
learning faster. It's not just automating, it's
just they can automate and then they can learn from it really
quickly and then they can iterate.
And we get to a point that where we used to worry about the
complex actors and the much less complex actors, but not and then
we would say, OK, we would just make it very difficult for the
non complex actors and then we'll deal with the more complex
actors. But now a lot of the less
complex actors have the ability to become a lot more agile, a
lot more complex and challenge us a lot more, while we are kind
of in the same space of we need to defend against everything.
And now you're part of a podcast that looks at the intersection
of tech and trust. But if I listen to you, I might
assume that trust in tech is something that doesn't come
naturally in the sense that you necessarily need to 1st think
about everything that could happen before you start trusting
into the ways that things don't happen or how is it for you.
So I think the way I see trust, I see it from a practitioner
perspective, trust is really important to me from first of
like how do the customers trust you?
And then how can I trust the data I'm looking at?
And that the, if we go into the trusting the data I'm working
with, that's really complex because as you grow in scale,
it's much harder to validate what you're looking at and say,
yeah, and this is 100% what's happening.
And then I can, I can make assumptions from there.
The problems that I've seen in the industry or in my specific
work is if I can't trust my findings, then it's very hard
for me to deploy them into something that's scaled because
you can make a small mistake. And in a small company it might
be a small issue. But as you scale, you're going
to have, you're going to come across across growth, you're
going to come across false positives, you're going to come
across, you're going to break your network.
So trusting in what you're doing, or at least the data
you're dealing with, is very, very important.
And you've now said that you're working on a business idea
yourself and that you also look into the way of how data
provenance can be deployed. Can you explain to me or also
the listeners of what data provenance like really is and if
it's connected to the way that you can trust data?
So my friend and I now came up with an idea we called the our
company Catalyst Labs. The idea behind it is how do we
help companies not fight the same fires over and over and
over again? How do we actually leverage the
intelligence so it doesn't sit on a shelf somewhere?
Because what I found and what I throughout the years, this is a
systems engineering problem. Like I said, you have
information security on one is like a system.
You have threat intelligence that's tangential, but threat
intelligence is only pulled in when it's necessary.
And even then it's very difficult to take tactic
techniques, procedures and actually narrative or prose and
turn that into something you can you can deploy.
So what I created and I'll explain the Providence thing is
a way to take let's say an analyst or FBI said threat actor
X does this and then this and then that and then and then does
all these actions and this is how they X filled it, their
information. How do we turn that into code?
So my system, instead of producing it to a detection
engineer, that will take hours or days or weeks to turn that
into actual detection. We can do that within two
minutes. But the first question that I
had when I thought about it is if I do this with an LLM, if I
do this with an AI, I don't trust it, especially at scale,
because it can read whatever information is on the on the
document and then it can infer something that's not true or it
can get bias from the document. And that's where I came up with
how do I have data provenance that I can actually go to to a
team like the team I worked at at Facebook and say here is
detection logic. But I know 100% that this came
exactly from the document that you provided at the very core of
the product, the extraction doesn't happen with the LLM.
It's a lot more programmatic that that provides you the here
is the rule and and then you have full provenance of it came
from this page of the document that you provided me from this
line. It is exactly what it says.
This is why we turned it into this detection logic.
And that's really critical because as I've been looking at
other companies doing it, there's this sense in the
industry that AI is this like magic tool that can do these
things, can do AI can do a lot, but what it can't do because
it's not built to do that is be deterministic.
So while you can trust it for doing one report, maybe 2
reports, as you start scaling you're going to start getting
very weird information flowing your way that will look like
it's real detection, but it's actually going to cause a lot
more work than anything. Why do you think what you what
you are building is so foundational?
Also, how does it work? How can I?
Yeah, if I use it, what would happen?
OK, So the reason it's foundational in my eyes is that
we've tried to do this forever. We've always tried to say how do
we take our intelligence and make it so it's not shelf wear.
So the intelligence is not just sitting there or, or for
example, I would see an attack and then I would say, I worked
on this exact thing three years ago.
I remember this. It's the same issue.
It's just a completely different actor.
So this has always been a problem.
And I think the one of the things that have changed is
that, yes, you can start doing that more with the help of LLMS
of turning intelligence into actual detection.
But I think the deeper problem is different.
The deeper problem is that this issue falls between teams.
So that's the big foundational problem here.
That's it's very, very difficult to have this even at smaller
scale deployed across teams and have it.
Work. And during your design process
for the platform, what trade-offs do you face when you
need to choose between speed or trust, openness and security,
and usability versus verification?
So right now my biggest trade off is speed and currently I'm
OK with it. The speed is not that major,
major of a difference. I would like it to be under a
minute, but now it's 2 to 6 minutes depends on the size of
the document. You cannot let go of the
Providence. It must be there.
So you can always the, the let's say that the secure engine
engineer can always say I know exactly where this rule came
from. And then it actually provides
you with an audit trail that you can say that rule that that did
XYZ, maybe that rule failed, maybe that rule's really
effective. We can actually say that rule
was created at this point by breaking down this type of
report at that time. So the provenance and the trust
are critical here. Much, much more than speed.
And if we now do you see any issue with the new like AI tool
providers that could potentially influence the user trust and
that also you would advise them to change?
Yes. So LLMS are non deterministic by
design. If you tell it never do a thing,
you are telling it be deterministic, always find this
thing and never do it, and that's you're going against the
design of the actual idea or the actual system.
What I have been seeing with AI is a couple of things.
There's a mad dash to implement AI into absolutely everything,
and some places that's great. Some places don't really need
it, but that's not the issue. The issue is that there is, I'm
noticing this around leadership mostly of AI can just do
everything. We'll just need to implement it
and it will solve everything and it will automate.
And then this, there's this push of you either implement AI or
we're not going to talk to you. And then I am seeing new
companies that are acting in kind of the same way machine
learning works is this is a black box.
You put your information, this spits out the right things and
trust it. And we are putting a lot of
trust into prompt engineering into.
Specific models and saying, yeah, we'll put this here and it
will just magically create what I wanted to create.
And I think there's a big chasm opening between providers of AI
solutions and actual practitioners.
Have you ever worked in defense in some sort of defense tech and
threat intelligence on a defense and regional kind of context?
I have but not in tech. So I was in the military and I
was doing defense, border defense, and interestingly
enough, it is exactly the same concepts.
That's what I wanted to get at because there's so much strategy
in the way that threat intelligence needs to be
designed and that could be translated either from a defence
tech. And I do want to ask that
because we're currently in quite tense times.
Also from a regional perspective, I'm not getting
into US, whatever Europe kind of discussion.
I'd rather want to abstract, like have an abstract view on
the way that this is approached. Do you have insights on that, on
how this is moving forward according to your views?
This is something I've said for for years in tech that we would
see. So in kinetic defense, like
order defense, we would see actors behave in a in a specific
way. We would see actors pretending
to be just benign activity. And that's really you can really
correlate it to, for example, network defense of you're saying
are you see actor go low and slow and or actor not automating
but or making different automation that doesn't look
like it's repetitive activity. What I have seen is that actors,
bad actors, no matter what they are, their main job is to learn
your defenses. And once they figure out your
defenses, they will start chaining A vulnerability and
another vulnerability and another vulnerability.
So an example would be you have a fence with the border, you
have cameras on the fence. You will it will it will take
them, could take them a year. They will find the place where
they get to the fence and you're not seeing them with the
cameras. And then they will see how long
it takes you to actually show up when they touch the fence.
And what do they need to do if they need to not touch the
fence. And then a lot of times we see
those as specific atomic instances and we'll say, oh, so
we need to add a camera here. Oh, we need a response time to
be faster. But what they're doing is
they're testing, continuously testing until they can say I
know instead of a zero day, I can now say that I know exactly
where you can't see me, how long it is until you see me.
If I, if I do something, how long it will take you to get
here. And then I can figure out my
tactic around that. Where do I strike?
How long will it take? What exactly what are the tools
that I need to use in order for you to not see me?
I think that the military is a lot better at that because their
lifes are at stake. There aren't metrics that you
need to measure. You measure and how many people
died. You don't even measure that.
But there's a lot of in the, in the military that there was
constant learning. And I, I think this is a part of
kind of space that I'm seeing that there is time to learn.
We are constantly chasing the new thing that's happening.
We are unable to, to learn which is, which is, it's a scale
issue. It's how many attackers are
attacking you issue, which the military doesn't have in the
same way. But that's the thing that I've
been always leaning towards at any company that I've been.
I said, OK, there was a big attack, something happened.
Are we doing a postmortem? And you would start doing a
postmortem and then who has time for that?
A new, new fire came. Now we need to fix this.
So that's at least when I look at at issues like that, I look
at them from that perspective. And for someone who's now
building up a threat intelligence team, what are the
top three things that you would advise them to do?
The top thing to do first is to understand your platform as well
as you can. It's very, very hard, but you
really need to understand the platform and appreciate that
you're not going to understand the platform as well as your
attackers, but do your very, very best to understand the
attackers, the platform. Then the second thing is to
break down your adversaries into adversaries that are are
opportunistic or like low sophistication.
And then kind of what's the scale of that versus attackers
that are going to attack you? They're going to wait and do
something massive and and work with that the and can really
understand the threat actor journeys for both.
The last critical thing is relationships must be on
understanding your product. You must if you're a threat
Intel team, you need to understand how the detection
teams work. What do they need from you?
What are they gold on and then work with them to empower them
and the business versus being a team that says no, don't do this
now do that you want to understand your partners and
then see how can I empower them to move forward safely and with
adherence of the law and the word escapes me regulations.
If we now come back to the lab that you built and the platform
that you aim to deploy in the future, so how does provenance
help us rebuild the concept of evidence in that world and also
for a particular potential users?
So that's a great question. The, the most critical thing
that I'm thinking about when I'm building is how do we build a
knowledge corpus that we know an attack happened or, or we
learned something. Now we can utilize it to learn
something else or now we can, when something new comes, we can
say we actually have this in our, in our, you know, evidence
storage. And we, we put a new report and
we say actually that's another attack that happened somewhere
else or happened with us a month ago, a year ago, or this is
something that we got from the FBI couple of months ago.
We should actually incorporate that information into what we're
doing. The way I was thinking about it
is one, we create this knowledge corpus of everything that's
happening and two, we are able to remove the friction between
teams into something that flows a lot faster.
Instead of producing something that an detection engineer will
have to now figure out what to do with, you produce them with
here's here's actual detection that you just need to review and
deploy and then you can go do the other things you want to do.
This actually brings all these teams together into having the
ability to circumvent the attackers together.
And if you now look ahead in a near future, what do you see?
What will be critical to deploy and also to follow through from
a platform provider and from a user perspective?
So I think one of the critical things to deploy are well is, is
a robust understanding. We are currently talking about
it. I see people frustrated with
attacks that keep coming back. They that I see it a lot in
scams of we are expecting other other entities to save us.
For example, we're expecting the government to create rules to to
save us, which which is great. The government, the government
creates rules that makes companies do things.
That's great that that usually doesn't happen because what
happens is that the government creates rules, smart company
figures out how to do the very bare minimum.
So I think what we need to move forward is if I think about it
from the AI perspective, is one of the things that's really
critical is not to look at AI as how can we use, how can we have
less people? How can we reduce headcount?
But instead, how do we give the people that we have superpowers
so they're faster. They're, they can, they can
analyze things faster, they can deploy things faster.
And that's again where the trust comes in and the Providence
comes in that as we move faster, we also need to move with
confidence. We need to know that what we're
doing is the correct issue so we don't fall back into
firefighting. Most teams are just stuck in
firefighting mode and it's very, very hard to get out of that.
But I think if we don't get out of firefighting mode, we're
going to continue losing. We're now coming to the end of
our podcast, even though I could talk to you probably for another
hour. And before we end our
conversation, I want to get into some in between moments with
you. At the first moment that I want
to start with you is what does trust online feels like to you?
Currently, Trust Online feels like a pipe dream, like fiction.
And what is the one thing that AI must verify in the future?
AI needs to verify both the output and the input.
Have AI verify both the input that is not junk and the output
that it can actually relates to something that wasn't invented
by the LLM. What is the biggest blind spot
in information security? Bias on that, But the biggest
blind spot is adversarial complexity.
Everybody likes to talk about adversarial complexity, but then
it ends up being a discussion about 0 days, which persistent
actors don't use zero days Willy nilly.
They consistently get around you daily without 0 days.
I need a sidestep here. What are 0 days 0?
Day is an exploit that nobody has heard of before.
Is this is a new vulnerability that nobody has heard of that
they can they can now leverage and there are no defenses for it
because this is the first time it's ever been used.
All right. Understood.
What does provenance unlock? Provenance unlocks trust in the
system, trust in the fact that you can move forward without
without creating a additional fiction.
What is the future that we are not preparing for?
I think we're not preparing. I think we're talking about it a
lot, but nobody's really preparing for the future where
threat actors will be using image, voice, video generation
to completely go around any current static mitigations.
And what is one word that you're currently in between?
Currently in between. I'm currently stuck on on
accuracy and robustness. I think those are the words.
What do you mean with that? So as I'm building and working
with partners and people to work with on deploying this product,
I constantly see opportunities to make the rules more robust,
to make the rules more accurate. But that's how you get into
analysis paralysis of like, we are not, we are not there yet.
We're not there yet where we need to worry about that, that
much. This needs to work.
And then we'll see how we make it work better.
But accuracy is very important to me and also robustness that
I'm not telling you here's an attack or here's a threat.
And this is how you can mitigate it.
And you can say, oh, actually you're missing a bunch of parts.
That's that's the thing that scares me the most.
For now, thank you so much for the time, for sharing all your
insights, and for joining us on the podcast.
Thank you so much. I appreciate it.