Hi everyone, My name is Patrick Akio and joining me today is CEO
and Co founder of Deploy Marta Stock.
We discussed a few things, some of which you might think are
controversial. First one, bottom up initiatives
are meaningless without a strong vision, A dot on the horizon top
down from leadership. Another one, the AI Act that the
European Union has put into place is actually beneficial to
all parties involved. And lastly, Marta shares how
he's building this AI hub here in the Netherlands, in Amsterdam
and what the benefits are going to be.
So enjoy what made you start this AI Hub or AI community in
the first place. I didn't really, I wouldn't say
start. I think those things start to
exist. So not giving myself all the
credits or so, but if you talk especially about the physical
location, I promised them as I started together with my car
furnace to other companies and I promised to myself and also to
my car furnace. I'm not going to spend a lot of
time on on this. If there's no demand for
anything, if it is a hub or something else and there's no
proper demand. If nobody really wants this, I'm
not going to spend any time on this because I have plenty of
other things to do. And that's already hard enough.
But then we found that more and more people recognise that in
the nodes, we don't achieve a lot when it comes to AI at the
moment or data or infrastructure in general.
So for me, that was the reason to, to, to start talking with a
lot of founders, like what is the, what are the challenges
you're facing at the moment? And in January, we decided to,
to host a dinner to get up with a few other founders who were
happy to spend some time on this.
We found Techie Peer to also be part of it and that was clear
that everyone is in the end doing a bit their own thing in
Eint, over in Rotterdam, in Amsterdam, in Delft, in here in,
in so many different places and others.
Everyone is kind of doing their own thing and that doesn't
really work for AI because AI successful AI companies are
scars I think. And you really need the top
0.001% of people that are both really smart and passionate
about something related to AI and entrepreneurial enough.
And if that's Canada across a pretty small country, then you
won't really find each other. And I strongly believe that if
you, if you create one place where we are all together, where
we can help each other, where we can, where we can help each
other finding investment, where we can maybe invest in each
other a bit, where successful start-ups will see spin outs out
of the company. They create that flywheel and
with that flywheel, we'll create bigger and bigger companies.
Without that flywheel, everyone keeps on doing their own small
things. And yeah, that's not going to
work. So the, the, the, the polar
model in that sense is and, and, and and and, and, and, and
trying to do this in a fair and, and, and, and distributed way.
I think that's not going to work for AI that we keep on thinking
too small. We keep on acting too small.
So for me, that was the main reason.
And then a lot of people got quite unjust about it.
And that's good to see. So that that that that motivated
me to spend more time on this to, to, to together with those
other founders to yeah, start searching for location, to
negotiations, find out that the real estate world is a
completely different world than the AI world.
I'm not a big fan of it. Yeah.
So I'm not sure if I want to do it again, but but that was the
starting point for us. And what helped a lot for me is
that with one of the companies we founded, Engines, we're based
in both in Berlin and we are based in the AI campus.
So we actually see how it works to be with a lot of AI companies
in the same space to do meet ups together to basically help each
other out, become each other's customers.
We actually see it working there.
So for us, that was like a great blueprint what we can achieve
here and on us as well, I think. Yeah, I like that a lot.
I mean, I like to start of how your thinking works in the 1st
place. If there's no demand then why
spend any time on it at all? Like what's the point?
What's the purpose? I learned that the hard.
Way and in general, I think one of the things that I love being
in tech as a software engineer is people share a lot with
regards to knowledge more so on the implementation side more
than anything else. Look, I there was this problem.
This is the context this is how we solve it.
They do that for free articles, blogs, podcasts, videos,
conferences, meetups, everything like that and this is kind of in
a same vein, but then for AI startups with regards to
challenges, funding, what problems are we solving?
Learning from each other, being each other's customers sounds
really cool man. I think we need it.
I think we, we have seen it before that it actually works
not even on purpose. Like we didn't create per SE a
fintech ecosystem in them. So there was not, there was not
one person who said let's create an ecosystem here.
But it happens. It's whenever there are a few
successful companies, you've got spinners, you've got, you got
more money in the ecosystem as well.
I think that's what we, that's what we lack in and out now.
But if you talk about cloud infrastructure, data and now AI
as well, we, we kind of missed that.
I can't name, I can't remember a lot of companies at least that
started here that have a lot of spin outs in that, in that
vertical and we see now the the results of that I think.
Got you. And you said NL is behind with
regards to that flywheel and that ecosystem.
Are there any countries that you say, OK, those are actually
quite far ahead, those are companies or countries to look
up to with the rest of their ecosystems?
Yeah, I think if still work ecosystems is more cities or
yeah, basically cities or the the, the, the large area around
it. Of course, Paris and Berlin are
the most obvious one I think in Europe, maybe next to London.
You do see other successful companies in other smaller
cities sometimes. So it doesn't have to be a
physical ecosystem I guess. But especially Paris is
fascinating I think. And it's good to realise that it
didn't start like five years ago, it started 25 years ago.
Szechuan F was funded in the end by the founder of Scaleway and
Scaleway exist for like 25 years, I believe.
So you see that flywheel already started 2025 years ago with
investments and successful companies in cloud
infrastructure didn't start yesterday.
And also don't believe like if we, if we're going to shout
really loud that we should, should, should push now that
doesn't work. There's a lot of groundwork.
There's a lot of, it takes decades of them to really become
very good in, in a certain industry as a country that that
doesn't come overnight. And that's why I, I strongly
believe in that the ecosystem, if we, if we get things
together, if we start building more successful companies, that
will lead to more successful companies that will lead to
another wave. So you need a flywheel to create
new waves and become better in a certain industry, in this case
AII. Feel like you need a very strong
vision on a long term horizon to kind of realise what you need to
do now and plant seeds with people, talk to people and
really start creating this. Is that usually how you think?
Like more in the bigger picture, long term vision.
I'm also very good to to worry about things yesterday.
The day-to-day, yeah. Definitely.
And I do think you do need to realize that it takes long.
I don't think it needs to be one person with a vision or so.
I think especially these kind of things can be organically
multiple people working towards something fakely in like 5 or 10
years, and then new people will come and they work for another
10 years or something in the same direction.
I think in that way you create a strong ecosystem.
Yeah, yeah, you get that. What's your take on kind of how
Netherlands and even EU is handling AI?
Let's start with the A i.e. U Act.
EUAI Act. It's a difficult 1, isn't it?
Yeah, I'm probably one of the fewer engineers who is actually
quite a favour of the AI Act. I've seen so many organisations
struggle to get beyond POCMPP, innovation, PowerPoint, blah,
blah. And the reason for that often is
that someone in the organization at some point is going to say,
yeah, that looks great, but you're not going to get get this
direction. And it's often because something
is lacking, which is actually written down the AI act, it's
either oversight, monitoring, it's, it's, it's, it's either
way too risky to get things beyond POC.
And I think what we, what we define now in the AIA is quite a
good set of rules and standards which are going to lead to, to,
to proper safeguards and help us to get AI beyond POC, to get AI
to production level software and systems.
So that's why I'm actually a big fan of it.
It creates standards which help us to get AI in especially more
regulated and high risk environments, production and
often the high risk service, also the high impact stuff.
So that's the reason I'm, I'm quite a big fan of it that I've
seen that having the proper governance and the proper
controls in place helps to get things beyond POC into
production. Yeah, help me deepen this out
for the people that don't know, because the the EUAI Act
categorises different types of AI in different fields of
industry, right? If it's education is different,
if it's risk and financial management, it's different.
If it's consumer facing, spam or anything else, it's mainly the
lower risk applications and then you have different
classifications with regards to what you need to adhere to.
True. Mary, first of all, disclaimer,
I'm not a lawyer. Yeah, legal specialist.
So we actually give training together with, with, with our
partners and often they are more the legal specialist and more
the, the, the technical person in the room.
And we often joke that we always avoid doing anything legal
because we hate it. And the other way around.
They understand shit about engineering.
But to summarise it in, in a way, in a way which is probably
also more interesting for engineers, You do indeed have,
have multiple categories. It's prohibited.
Don't think about it. You're not going to do it
anyway. I've hardly seen application
which you fall in that category anyway.
Then you have high risk and that actually entails quite a few
things. And to summarise, it is
everything that has an high impact on human rights, on
basically the rights you have and, and I have, which are also
a strong part of EU regulation in general.
And there, there are quite some articles which dictate what you
have to do. Basically the standards you have
to take into account. So you have to explain for
example, where your data is coming from.
You have to, you have to explain and you have to document how you
think about bias or ethical considerations.
You have to have proper controls in place.
That's called human oversight in this case, or effective human
oversight. So there are a few things which
you need to do if there's a high risk model or high risk system
in general. And I believe that those things
you should be willing to do anyway.
But it helps now to have a standard of like, OK, these
eight points with like sub bullets are the things you have
to do. And there's often software for
that. There's a reason we started
deploy is to basically make sure that you have all the tools in
place to adhere to the things you need to do for anything that
has some risk with it. And you should do that anyway
because if you don't do it and there wouldn't be an AI act, you
would still probably be in the news if you if you basically
fuck up with with a certain system.
So I think it really helps to create a standard and I don't
really think it's going to block innovation.
Is that the main counter argument why people don't like
it? That it's kind of putting the
brakes on innovation. Because I agree with what you
say, a framework can be a good thing, right?
And now this is a legal framework, so it's kind of
mandatory and you have to comply to it, but it's still a
framework. And if those are a means to an
end to create a good system, then it could be a good
framework as well. But that's subjective.
Yeah, the the only other thing that's going to happen if you
don't have an AI act is every single member state is going to
bring up an AI act kind of version.
You see the same now in the US If you want to bring something
to the US market, you have to adhere probably to the New York
State regulation on AI, which is more about bias and ethical
considerations of fairness. You have to do, you have to act.
You have to comply to the one in Illinois.
You have to comply to the one in California, which actually looks
quite similar to the AI. So it's just a patchwork of
different regulations, often also conflicting on some points.
It's way harder. And the same is going to happen
in Europe. Like France probably going to
protect tech companies so they will be quite, quite, quite
relaxed about a lot of things. But Italy doesn't.
So Italy's going to be super strict on things.
It's going to be a mess in the end.
And then we're all going to complain like, yeah, it's such a
mess in Europe. We need to have harmonise things
and it's all over again, the same thing.
So I'm really happy we actually have some clarity.
We have standards. It's going to help actually also
because you will do those things anyway.
You know, the opposite is exactly what I explained.
It's it's, it's basically what's happening in the US.
Yeah. And I wonder why people are not
a fan of it, right, If it's otherwise, kind of, we don't
have many other options. And each country is going to do
their own thing. There will be something because
I feel like people need to be protected towards the
applications that they use, they consume, especially when they're
more on the human rights side. With regards to the impact and
yeah, mainly impact, I guess. So there will be something.
So this might be the least of the evil I guess.
Yeah. And I think, I think which, so I
do agree that we see large differences between the US and
Europe, but I think it doesn't come from regulation per SE.
I think it has to do with the way we we deal with regulation.
So we can use it in the next use.
We can always use regulation as an excuse, like I'm not sure if
that's allowed by GDPRARAX, whatever you can think of.
Or you can try to comply as good as possible in a practical way
and just go ahead with innovation.
And I have the feeling the latter happens in other parts of
the world and it first happens here.
So if you can just use legislation as an as an excuse,
yeah, we don't have to do anything.
So it's easier. We have to work less, we have to
spend less hours on the office. But that's more cultural thing
than anything related to regulation.
So if we wouldn't have the AI, we would find another reason or
those people would find another reason not to innovate.
So we have to fix the cultural path, which is the the hardest
part by the way. But I think that's the thing we
need to fix if we want to innovate quicker.
But regulation itself, I think it's not the bottleneck.
You also mentioned that this might be a means to an end to
help go from proof of concept to something that's actually
production ready that people can use.
I've been in my previous assignment at a bank where I've
seen many proof of concepts, but then they didn't comply to let's
say the standards of the environment with regards to
technology and what you can or cannot use.
It was just proof of value. All right, we see this value
now. Can we do it with the tool set
that we have? And then if it's a no, then it's
kind of yeah, there's stakeholder expectations there.
There's an investment. There's no return on investment
because the technology is not compliant.
So it's very difficult to then go from proof of concept to
something that is running an application in in production.
I don't know if many other or at least from the organizations
that that you've been in and out of are struggling with something
similar when it comes to actually we see proof of value
and then we're trying to go to production, but technology yes
or no or we're behind or there's a lot of risk and low risk
appetite. Yeah.
I, I see those things a lot of course that that that I think it
has to do with bottom up is this top down approach.
A lot of organizations are hiring a few smart people also
very young, and they kind of have to figure out what they
have to do with AI or data products.
Just build something, be innovative, experiment,
whatever. And I never saw that really
working. So if there's no pretty clear
picture where you want to go to a top down in an organization,
it doesn't work. It it just ends up in MPPS and
PO CS and proof of failure or whatever other names you want to
give it. But it's not going to prediction
because it's at the end of the day, it probably doesn't really
fit in whatever, whatever strategic direction you're
deciding to go to. So it's really also the
responsibility of board level or whatever you, whatever is
leading organization, it's their responsibility to really think
about, OK, this is where we go to in like one or two or three
or five years. These are the products we need
to get there. This is the reason we are doing
it. And that can drill down to the
rest of the organization. They can figure out how to build
it, but in the end I think you need to really have a good
design where you want to be as an organization in like 3 or
five years. Same as building houses or
building suburbs. You also don't start bottom up,
you start top down. You need to have a proper plan
of a suburb, then a proper plan of how you build a house, and
then you can actually start building bottom up.
But otherwise it doesn't work. Yeah, maybe in Belgium, but in
most countries it doesn't work. Especially in some industries,
leadership teams have always struggled with the role of tech
product, digital products. And now you have AI there as
well. And you mentioned that this
vision, this kind of dot on the horizon, where do we want to be
as an organization? How do we leverage technology
that's now out there? New is now also their
responsibility. It's quite challenging to I,
from my perspective, kind of see that take shape in some of the
organizations or industries that I've been in.
I do see people experiment bottom up.
But then in the end, like you mentioned, the effect of that or
the outcome of that might be that it doesn't reach production
at all. But I do like the
experimentation mindset, just within boundaries I guess.
Absolutely. So my advice to to more junior
people in the organization, if you feel there's no direction,
stop developing and start, stop developing and start asking
those questions because it's never going to work.
It's going to be very, very energy draining.
I think if you work on something for six months and it's going to
be thrown away and the other way around.
If you're really on top of the organization, if you're on the
board, if you're a director or whatever.
And then you have the feeling that for some reason you don't
really understand what you're working on or you don't really
understand anything about digital products, data, AI.
Either find a person and give them a board seat or Start
learning how it works and start gaining knowledge about it.
But doing neither and hope that bottom up stuff is going to
happen. I've, I haven't seen it working
for the past 1520 years. Is not going to work in the
upcoming years either if you don't have a vision.
Yeah, and I see that also if you talk about other geographies, I
see younger people actually leading companies than in most
parts of Europe. We tend to have pretty old
people in parties, especially outside of the Netherlands I
think not doing that bad, but I've seen statistics of that and
the average age is just pretty old.
Also here of people running companies.
And that's a bad thing, you think?
Yeah. Yeah, I think you need a few
younger people sometimes as well on the top of an organization.
Yeah, I haven't figured out how to get there yet, except like
this. Oh, you start something from
scratch with the years, and then you're also old by the time you
have something of scale and you get there.
There are really, really big companies in the US where the
CFO is like 35 years old. Yeah.
I think that's going to be really.
Helpful. Yeah, very good there.
What do you think of when people say culture eats strategy for
breakfast? Because I've been in a lot of
environments where people really kind of hamper on their culture
to a point where indeed they might not even have a strategy.
And then sometimes it just feels like anarchy.
I don't know. I think it goes hand in hand.
If you write something down when it's not a lift, it doesn't
work. Yeah, If you never write
something down and it's all captured, your culture, it also
doesn't doesn't scale. At least there's a lot of
unclarity. We at deploy, we have, we have a
hybrid model, so people come to the office and that helps in
culture and also work at home. And so we need to write things
down. And I have a lot of respect for
companies who are remote only because then you have to write
things down. Culture's pretty hard.
In the end, you can, you can work around if it's really hard.
So you have to write things down.
And that strategy I think is super, super important.
Vision, mission, strategy, goals, targets, etcetera.
And if you're always in the office, I think the culture is
really important. I see that here when I walk
around and obviously culture is super important and, and both
ways can work. But I think if you if you have
some kind of hybrid model, it also goes hand in hand to have
both strategy and culture pretty much aligned in that sense.
Yeah, Yeah, you mentioned that bottom up initiatives and bottom
up experimentation, it, it just doesn't go anywhere without a
strong vision, right. And then, at least from my
perspective, you can experiment within those boundaries and see
if the outcomes are a means to an end.
In the end, if they help the business, the company, the
organization and the group of people, because organisations
are still just groups of people, reach whatever their vision is,
or at least take a right step towards that.
That's super easy I think I just.
I just don't understand why it's.
It sounds complex to me. You define where you want to be
5 years from now. That should be doable for most
board members. I guess you drill down what
initiatives need to be taken and you see where AI or data or
digital products fit in. But how do you take people along
with you, right? If there's a vision where they
don't see themselves, you lose them.
That's true. So that's easier.
So it's smaller companies, of course.
I think, I think that the more you go to done, the more, the
more you can involve people. I think in any way you can
involve them. But in the end, I think clarity
needs to come from a few people. In the end, if you don't provide
a clarity, everyone is swimming a bit in the end.
Yeah, yeah, yeah. Which is super frustrating.
The thing that I've seen the most, and this is indeed coming
from board level and organisations, is we see
companies doing stuff with AI. We need to do something like
this decent of urgency that just trickles down and there's like,
we need to act something with AI on the product sense, embed Gen.
AI into whatever we have, but they don't even use Gen.
AI. It's just AI.
Everything, everything gets under this umbrella of AI.
Doesn't matter if it's predictive AI or generative AI,
That's one. And then there's the second one,
which is more for the people that already have a lot of
software engineering capabilities.
We need to be more efficient with regards to how we use AI so
we can improve our output regardless of whatever outcomes
we're trying to achieve. We need to optimize the
efficiencies and stuff like that.
Those are the two. And then especially if there's
no good framework, if there's no good story, even the people that
communicate that are lacking in communication skills or even
sometimes just charisma, they lose the buy in.
There's been a bottom up kind of challenge and push back on, OK,
we don't even have tools available.
How do you want us to work with AI or how are we going to be
enabled in this, this mismatch and expectations.
And then it just sandwiches and middle management.
It's trying to middle manage. I agree, but maybe to turn it
around and make it positive. I've seen really, really good
examples as well. That's, that's the, that's the
fortunate part of working on something like D players that
you're early adopters are the ones that actually have a strong
vision, that actually know where they want to be in two years
from now, that actually are growing really quickly.
That understand that efficiency is super important.
The ones that understand that having a few killer AI features,
AI products in the offering is going to be super important.
So it's fascinating to see them operating actually, they are
actually really growing quickly because they they do define
where what they need in like one or two years from now.
So I would just turn around and look at the ones that grow
quickly what they are doing. And often they're quite open to
talk about it. They often share quite a lot of
things. But those are actually really
interesting to watch, I think. And you don't have to do that
only in the Netherlands, but if you look in, for example, the UK
and how Fintech and also share tech now is taking over quite a
big market share from the incumbents in the market.
You can learn a lot from them. And it's super interesting, I
think to see how they are operating at the moment.
Can you elaborate on that? You said already it starts with
a strong vision of where do you want to be in one or two years,
but what are they doing that is exceptional that allows them to
excel in that way? It's not even exceptional.
I think it's it's, it's so if you think about it, it's not
even, I think it's not that they actually spend time, I think on,
on, on defining where they want to be in one or two or three
years. And they do quite some sessions,
I think together with their colleagues on discussing and
arguing whether it's going to be.
So if you talk about banking, for example, and your banks, you
see that they basically take the whole process.
Every company is a is a little factory in the end.
They take the whole process and define and every step of the
process they define here and here and here and here.
We feel that with relatively small investment, we can
actually make things much more efficient or much more effective
for basically make a business case in the end that that skills
and that's what they are doing. They basically make a blueprint
of how a bank works and then find all the places where they
are more efficient of more effective or can create new
products and new offerings which are you know, which which they
expect is going to be demand for.
So that's what they're basically doing and that's often was
lacking in all organisations. So if you're going to do the
same thing, I'm 100% sure that's also we have the right people in
the organization, but often you need less people than what you
think you can actually achieve the same thing.
And that's why I why I mentioned fintech and, and in shirt tech
in the UK, they're much further than here.
They also have already much bigger market share.
And you can, you can learn quite a lot by looking at the
presentations, meet up slides, those kind of things.
We're going to learn a lot from that, I think.
How much can you still gather from this kind of competitive
analysis? Because you mentioned, OK, look
at your own organizations, look at your whether it's process
journeys or customer journeys or workflows that you have and see
where kind of the most time is spent where you can make a solid
business case where you can rely on technologies like these to be
more effective in what you do, right?
That's very much, I feel like looking at your organization in
a nutshell and trying to optimize in that way.
What I've seen other organizations do is they see
what someone has created in certain output.
It might be chatbot is always the kind of go to example.
Someone sees a chatbot, they can be like, OK, well we can
actually also do that and it's going to plug and play.
But it might not be the best for their organization or for their
consumer base or for the branding in the 1st place, but
that's what they do. I feel like what you mentioned
in looking at your own organization, that's kind of the
better way to do it. But how much can you then still
learn from your competitors? I think how they how much time
they spent on making the blueprint, there's something you
can learn from them. The exact blueprints or the
exact use cases. That's something I think you can
better define yourself because every bank is slightly different
with different kind of customers, different kind of
offering, different market positioning.
So you have to make your own blueprint in the end.
But the fact that you have to make a blueprint that you have
to think about it, that's something you can learn from
them. I think if they have time to
spend spend hours and days on on on defining where they want to
be in a few years from now, I think also other organizations
can do that, especially prioritising innovation and the
stuff of the future of everything that happened
yesterday. It feels like there's no easy
answer and you just have to go through the motions and step by
step, kind of figure out where the value is for your own
context and for your own organization.
People are looking for a quick win though, especially.
That doesn't work. That's OK, That doesn't work.
So maybe that's that's the thing I'm trying to tell you is just
saying that we have to push something very quickly.
It's often a very stupid strategy that I hardly ever sort
of working. No.
What are some of the implementations or integrations
that you've seen that are actually greatly valuable?
Is there something you can share in that aspect, some of the AI
features that you have seen go live and actually have impact?
Yeah, I think it's good to to to distinguish different kind of AI
applications. And so you have a few AI
applications which are basically in the current process, often
predictive AI, not even generative AI, but every process
that is super repetitive, fraud detection or claim handling a
certain insurance company or you know, whatever.
So many of those repetitive tasks, you see a repetitive task
which is done by people at the moment, which is slightly more
complex than normal automation. It's often a very good candidate
for machine learning and AI. If you talk about creating stuff
which is, which is based on facts, that's also of course
very easy to, to, to, to use. Actually at Deplo we are, we are
now working with an AUS company which is doing quite a lot of
our development work, which is I think super fascinating to see
how the learning curve's going. So also with development work,
you can even go beyond Copilot, you can actually work with
companies that take over quite large chunks of your whole
development work because you have the context, you have the
code base, you have especially all the documents you can give
your own developers. You can give that also through
AI and with all the cartwheels they already have built and have
in place. You can, you can actually
automate quite a lot of development work, not as an
assistant, but actually taking over your development work.
So we see that working on our own organization as well.
But all the other more repetitive stuff, I would also
have a look at it. But all of them in the end
require all the guardrails and the governments we talked about.
If you want to go into the direction, invest in all the
groundwork and all the the the infrastructure around it to do
it in a good way. I think that's it's never a
waste of money. It's the same as investing in
cybersecurity 15 or 20 years ago.
Whenever you want to go fully digital, you do need those,
those cart fields basically to to go beyond.
Yeah, just some experimentation work.
Yeah, you mentioned there's an ex, Did I understand that
correctly? There's a company helping you
with the development work that you have on deploy itself as an
application. So of course everyone is using
Co pilots of rabbits. That's not really taking off all
your development. That's just someone piloting.
Yeah. You're, you're still operating
in the AI system we actually create.
We we are basically doing the design work, defining what we
need and then send it off to another company and they do all
the work until merch request and then we accept it or a review of
the list. Gotcha.
But isn't that your core? Isn't it scary to?
Kind of scary. Give that, yeah, like or have
that in collaboration with someone else, but it's working.
It's also interesting because we have to think even better about
the design. Yeah, we have to think even
better about the requirements. We have to review it even better
than before, but it actually creates better code.
Does it a lot of companies invest in how much their
development team understands the business, understands the
product, right? Because then you can think
along, understanding why we do things makes you challenge what
we need to do in the 1st place. And then if the requirements
don't make sense from your lens, also looking at diversity, then
you can challenge that. But I feel like with this
construction, you might miss that.
Yet you're still saying kind of quality wise we have seen
increased? For some.
So I do get your point. There's also the worry we have.
So if it is, if it requires a lot of context and expert
knowledge, it's harder, then it often doesn't really work yet.
Yeah. Although we can, the more we do
with them, the more context they have, so the better they will
understand it as well. But if it is, if it is a feature
which is pretty well scout upfront and has less of those
business context on it is actually quite doable.
And because we're not doing the development work itself for that
feature, we actually are more critical in the review, we're
actually more critical in the design.
So it actually creates better features.
Interesting. Is this like an agency or a
consultancy that does this construction?
I don't know how to classify it. We'll discuss it actually.
Is it a supplier? Is it?
Outsourcing is it. An AI tool is that freelance is
a combination of those things. Yeah, it's pretty cool zooming
in into AI features in production.
You mentioned a few things with regards to effectiveness and
efficiency. What are some of the things
that, if you're talking about a production system, needs to be
there with regards to observability?
I talked to Saud up earlier and he has a very distinct view on
AI and operations, let's say. But I'm wondering what your take
is. What are the metrics that I need
to think of or what do I need to have in observability to be in
control? That's a good question.
It depends a bit again on the on the type of application of the
type of technology you're using. If the generative AI or
predative AI. I actually listened to the
podcast you you met with sub OPS.
I'm trying to recall everything now.
I think what is so you can measure all the technical
aspects, both the the, the, the, the, the more the metrics around
traffic, but also the ones which try to capture performance.
There's a bit like you, your your critical question like 2
minutes ago. A lot of things are more require
more context and simple statistics.
So we are deeper strongly believe that one of the biggest
things you need to measure, one of the most important things you
need to measure is some kind of feedback loop, either directly
or indirectly. So indirectly could be that you
choose the second option or start arguing with an, with an
NLM that's, there's also a kind of feedback loop, but having
some kind of feedback loop and monitoring the feedback loop and
act on it. Whenever you see that things get
worse or when in a certain subcategory you see more
feedback could, could mean that the, that the system is biased
on some subgroup. I think that's the most
important thing you need to measure next to all the
statistical stuff and all the the technical stuff.
The real thing you need to measure is in the end, human
feedback, because that's, that's basically interaction between
humans and machines or human and AI systems.
And that's often pretty easy in practice to, to measure.
It's if it is a, a predictive AI model, it could be feeding back
actuals or feeding back expert opinions or overalls.
If it is an, if there's a generative AI system, it's
basically all the arguing that this happening between an LM and
a human being or over ruling decisions or trying to correct
things. Those, those things are all
forms of feedback. And if you get feedback, find an
API and start measuring that. Then you can act whenever you
see that things don't really work in certain applications or
some subgroups, or it's getting worse over time, or it's getting
better. The positive side, I think
that's the most important one because a lot of things are not
captured in statistics. Yeah, I really like that.
Like the only Gen. AI feature that I worked with in
the end, it was going to make people more effective.
So work that they were doing, spending hours on it, we had
something that would pre fill a form instead of them doing the
work, they would kind of have their own position of reviewing.
And I saw a lot of people talk about, OK, if the answer is
correct, then in the end it's correct, right?
And we can even have a model to say if what a person changed, if
the intent was exactly the same. And for me is the fact that
someone already changed it, if it's not the way they like it or
the way they want to kind of have that work be put forward
because that goes in a whole chain and other people are going
to see it. It's not good enough, right?
It's kind of this user acceptance that also needs to
take part of that, which I think is interesting.
And I think that we still call it explainability, but it's more
than just Shepherd Lyme or whatever.
But but but explain ability or support for a certain decision
or certain output is super important as well.
So if you talk about it and I'm giving the support for certain
outcomes, it's super important because the outcome can be, can
be good, can be a good recommendation for something,
but the reason behind it could be completely wrong.
So I think that support is also important to make sure you
capture the feedback about the reasoning as well, which is
harder for a Netherland because asking it for an explanation is
not an explanation of the model itself or the underlying
technology. No, it's could be just as well
hallucination. Yeah, yeah, yeah.
Reasoning with regards to why an answer is the answer it is I
think is incredibly challenging. But if you've if you have a good
way and it also makes sense from a user perspective continuously,
then I feel like you have a great outcome.
I think that's an it's going to be an an ongoing discussion for
the upcoming years. But it's basically the
discussion between closed source systems, AI systems and open
source models. I'm not sure which one is going
to be leading in a few years from now.
So you see open source getting closer to close source.
We also see that the business value of it is is pretty small
at the moment to be more open about which code you use, which
data you use, etcetera. But if we go more towards or if
we demand more transparency than it could be that we in the end
move more towards using open source models for repetitive
tasks or for certain use cases. And with open source models,
LLMS, you can to a certain extent explain why, why a
certain outcome was the way it was, like which tokens were
important, for example, or saliency of, of of of a certain
outcome. And I think that can help to
really understand why the lamps are doing some really strange
stuff, sometimes better than just asking for an explanation.
So if transparency is getting more important, if control and
source code is getting more important, if IP is getting more
important, then we might move back towards more open source
systems. If we for some reason decide
that it's not important, then we probably stay working with
closed source systems and it means that we have less
transparency. We have to rely on the
explanation we get from those providers.
Right now like those are I think big ifs and I could see them, I
have assumption on the way they are going, but are there any
incentives to use open source versus closed source currently?
Yeah, we see this quite a bit in, in in health tech.
You don't want data to be shared to those really large tech
companies. You do want to have more
controls. You do want to really go deep
into the the LLM system itself. You probably want to use smaller
language models, still pretty big of course, with smaller
language models because you have simply more control on what's
happening. So there we actually see quite
some companies working with their own open source LLMS which
they fine tune and run use case because they simply have more
control on what's happening. Interesting.
And I think the same will happen for the high risk use cases.
Yeah. Is it because I haven't
experimented as much with open source variants?
I don't know if they need to be self hosted.
For me closed source model is just an API Callaway and that's
it. And if it's compliant within the
organization then we use it. That's like and we have
experimented with other open source models, but I haven't had
any hands on experience. Do people usually self hosted or
how much expertise is needed to get that up and running do you
know? Yeah, I've seen it with some
companies that differs a bit I think, but you do, it is of
course way more investment in the end to, to make that work.
So the ones I, I mentioned are often focusing on one single use
case, which which is basically the company in the end as well.
So they provide a certain service in a certain step of
certain process. So if you talk about healthcare,
they're doing like one care pathway, 2 care pathways and
they optimise a model for that. And that on itself is already a
big enough business case to run a company on.
So if that's your whole company, then it makes sense to to
provide all the card rails. If you want to scale that to
like all the care pathways or use it in healthcare in general,
then I think it's going to be super, super complex and it's
requiring a lot of money, a lot of investment.
But the other way around, just relying on a big U.S. company to
do this for you is also another way forward, I guess.
So yes, it does require quite some investment in in multiple
different ways, but it's also the only way forward, I guess in
those in those users. So especially in healthcare, I
think you do want to make sure that you understand why system
makes certain decisions. And hence you do need to have
access to source code. So that's what what we at least
see. But they're not honesty.
If I touch card within deploy, I think our developers are going
to kill me so. For me, like government
healthcare, if you're saying like these are the typical use
cases, it's quite specialistic work.
But then the people that want to work in those environments, they
might not want to work in healthcare governments because
it's also like slow organizations, very slow moving
people that have been there for 20 years and a part of the
walls, let's say. And they don't have the same
experimental or innovation mindset that other startups
might have. So it's kind of this chicken and
egg problem. I don't know how to get out of
that yet. I feel like we do need
innovation in specifically those fields, but the way we've been
doing it so far, it's just not effective.
Yeah. And like these kind of
innovations often start with things being under pressure.
So in healthcare we will run into a point where it's simply
not affordable anymore. And I think that will, that will
make it easier on the end to innovate.
That will, that will help letting organization make
decision that has to do with risk.
So if we don't want to take risk, then we then it's hard to
innovate. At some point we need to take
the risk because the other, if we don't make that decision,
then the decision basically we can't provide healthcare for
anyone if not for everyone anymore.
So I think because both government and Healthcare is
probably under pressure of the upcoming decades in Europe, we
will make decisions which stand towards innovation and will
become easier to innovate and will become easier to both
companies and that in that industry.
But I completely realise that at the moment it's super, super,
super hard to build a health tech company or Med tech company
or company which is providing software to the government, not
consulting with actual software, you know, super hard.
Which is strange, right? Because in your scenario,
there's like, OK, there's this building pressure, which means
we don't have an option anymore and we have to.
It's like back against the wall. Whereas now I feel like we still
have time, yet it's not working as well as it should.
But right now would be the best time.
The best time for a lot of things, we wait till it's almost
too late and then we start innovating.
But it, yeah, it has to do with our personal horizon, I think.
Do you know what you're going to do in five years from now?
No, I hate that question. Whenever you've been talking
about these long timeline horizons and I'm like, Oh my
God, like I have no clue. Most people don't, I think, so
they're not optimising for the long term.
Do you like, do you have that strong vision for you as a
person or even you as a company? I am my my girlfriend at home.
She often finds it very annoying that I'm thinking I had for
quite some years. Quite some.
Because it makes it less romantic I guess.
Yeah. So both personally and
professional, it's something you do.
Yeah, I also like it. Mary's a bit of dreaming
sometimes about about some stuff as well, but I think it can also
be really important to at least have some fake idea where you
are in a few years from now. And it would indeed be helpful
to do that in, you know, in with the government, in healthcare,
with hospitals, in the financial system because they're also
going to be under pressure. We talked a bit about fintechs
initiatives which are definitely going to take market share.
So if I think it's going to be crucial for them as well to put
innovation high on the agenda and not on like on Friday
afternoon, also for them, it's going to be critical.
But probably you'll see that when it's almost too late,
people really start to innovate. Yeah.
People will move. And that's happened over the
past decades or centuries as well, I think, over and over
again. I can see that.
Yeah. I've really enjoyed this
conversation, Martha. It's fascinating the way you
think, the way you have a strong vision and kind of are driving
forward this community and in and now and I'm some
specifically, I'm going to look forward to it.
We'll, we'll keep in touch. Hopefully it's going to be
successful. Hopefully it's going to be
successful. We'll see you as we go.
Thank you so much for coming on and sharing.
Thanks a lot this. Was a blast.
Well, round off here. Thank you so much for listening.
If you're still here, let us know in the comments section
what you thought of this episode and we'll see you in the next
one.