I was doing a consulting project for Walt Disney Parks and
Resorts. We kept finding ourselves in
this situation where we were supposed to integrate with an
API and it wasn't ready or the team delivering it said it was
ready, but it didn't behave the way that the specification said
it would. And I actually went on paternity
leave and I kind of thought that this is an HR being wanted to
scratch for a while. And so I spent quite a lot of my
paternity leave kind of hacking away on what became the first
version of Why I'm. On So Tell us what is why I'm
not why people should consider using tools like wire more?
The benefit is that you're bringing into your inner loop of
development the ability to learn and then get feedback about that
real world integration. You know when you're working
with something that closely resembles a real API, rather
than your own abstraction with your own set of assumptions
about it as the thing you're integrating with.
If you refer to the testing pyramid, where do you think
Wyomops sits in that pyramid? What I've observed in practice
over the years, you get the testing trophy shape more often
than you do the testing pyramid shape.
One of the sort of points of resistance around mocking that
you hear a lot is this idea that it can't really be trusted.
It's not really realistic. And you see some organizations
who will do the vast majority of their testing in a sort of
complete the integrated arrangement.
And the advantage of bringing mocks into the equation is that
they kind of flatten the economics out.
The difficulty involved in testing some weird edge case is
about the same as the difficulty involved in testing your sort of
#1 happy path. So.
WI MOC itself was created in 2011.
It's been like 14 years now. It has like 6,000,000 monthly
downloads. Lots of contributors, right?
How do you actually make this open source project successful?
There are a number of things that make open source projects
successful, but it's kind of about.
Hello guys, Welcome back to the new episode of the Technology
General Podcast. Today I have with me Tom Akers.
He's the creator of Wire Mock. If you don't know, it's an open
source tool for API testing, API mocking and things like that,
which has been created a long time ago.
So it's like in 2011. I think I will learn from Tom
today how he made that open source project successful, and
we'll talk a lot more about API testing and API development in
general as well. So welcome, Tom to the show.
Thank you for inviting me on. Right Tom, I always love to 1st
invite my guests to share a little bit more about you, maybe
by telling your career turning points that you think we all can
learn from. Sure.
So I'm sort of, I guess a career software developer.
I've been doing it for my whole adult life.
I've usually worked in enterprises that have complex
and often distributed network systems integration problems to
solve that kind of thing. And to give a little bit of, I
guess the Genesis story of why I'm not, which is probably my
most significant career turning point.
I was doing a consulting project for Walt Disney Parks and
Resorts, working on what what is now the magic band system.
So their system of kind of RFID wristbands that you get as a
guest that grants you access to rides and your hotel room and
various other things. When that system was first being
conceived of and built, I ended up working on that programme and
Disney needed to do a sort of huge digital transformation in
order to enable this. So they had a load of systems
that sort of operated different bits of the park and hospitality
experience, but they were kind of quite siloed.
They, they did some integration, they had some sort of existing
service oriented architecture in there, I guess, but the, the
degree of integration needed to be far greater in order to, to
support this initiative to build the magic band system.
So I got involved in that program and it was sort of
before micro services have been coined as a term.
And it was when rest was really just catching on as the sort of
hot new architectural style. One of our senior engineers from
my company actually persuaded the Disney senior engineers, the
architects, that they should be using Restful design principles
rather than continuing to build things on SOAP.
And that was great and that was forward-looking.
But the downside of this was that nobody really knew what
they were doing at this point. And it was a, a huge programme.
Lots of developers kind of parachuted in from various
integrators and consulting companies very quickly onto this
programme. And the the result was, to put
it politely I suppose, friction being caused in various places
and things not being quite as productive as they needed to be.
And the team that I was working on was building a set of apps
for cast members for Disney staff members to help customers
with their experience. And these systems needed to talk
to lots of different APIs that have been provided by various
different vendors. We kept finding ourselves in
this situation where we were supposed to integrate with an
API and it wasn't ready or the the team delivering it said it
was ready, but it didn't behave the way that the specification
said it would. A lot of the time environments
would be unstable, things would be broken.
This whole sort of litany of problems that are really common
in micro services now or so or anything networked really that
you kind of need to address in order to to, to be productive.
And I actually went on paternity leave and I kind of thought that
this is an itch I've been wanting to scratch for a while.
There must be a, we need a tool that's that's going to help
decouple us from this sort of slightly kind of chaotic
environment in which we're trying to build software.
And so I, I spent quite a lot of my paternity leave kind of
hacking away on what became the first version of wire mock.
And it was really the, the first requirement for it was I just
want to be able to copy and paste the examples on the wiki
that formed the specification of these APIs into a tool.
And then I want to be able to test my product against these,
these simulated responses. Rather than having to rely on
all of these other teams to provide working services, to
provide test data, to provide me documentation telling me how to
operate these things, all of that kind of stuff.
So that was kind of how Weimart was born.
This episode is brought to you by Swim dot IO, and I'm excited
to have its CTO and cofounder Omer Rosenbaum with me today to
tell you more about Swim. Hi Henry, very nice to meet you
and thank you for having me. So tell us a little bit more,
what is swim dot IO? At Swim, we want to help
companies understand their code bases.
We combine static code analysis with generative AI to create
comprehensive documents that help you navigate the code base.
As an engineer myself, I wouldn't want engineers to spend
so much time understanding existing code.
I would want them to spend time creating and building new stuff.
When you have code that has accumulated over decades, and
especially in legacy languages that not many people are adapted
nowadays, then the problem is even bigger.
Swim dot IO is specializing into helping mainframe developers to
understand their code base. Why mainframes?
We actually didn't start there. COBOL had been by some people
obsolete for a few years, and I discovered that it's not really
obsolete, not at all. There are more than 800 billion
lines of COBOL code that are in production and they drive lots
of the business in the world. And we got more and more
requests from customers to help them understand the legacy code
business that was written decades ago and get accumulated
over a very long period of. Time So from your customers so
far, what are the some of the success stories that you can
share? So we worked with an analyst who
shared with us that it took them a year to document a single
mainframe application, and using SWIM they were able to document
a similar application in a matter of hours.
So saving that amount of time enables them to focus on.
Other tasks Thanks Amir for sharing with us about SWIM
today. To learn more about SWIM, check
out their website at swim dot IO.
Right. So very exciting that you were
sharing with us. Like how did you start with this
WI MOC tool, right? I could still remember back then
as well, like when early in my career we used to work with SOAP
UI and things like that when SOAP was still the main thing in
application development. And SOAP UI back then was kind
of like the go to tool for API mocking, testing or just doing
things like WI MOC is doing, right?
So I think when we switch to REST, definitely there are a lot
of gaps, for example, the tools, you know, the maturity of the
tools as well. So I think it was really great
to see such solution like Wyomok exists because as we adopt more
Restful AP is I think we need such tools in place.
So Wyomok itself was created I think in 2011, right?
It's been like 14 years now. I think it's such a successful
project. It has like 6,000,000 monthly
downloads, lots of contributors, right?
And a lot of companies and maybe other tools integrating while
mock as part of their systems as well.
So maybe if you can share a little bit of your insights, how
do you actually make this open source project successful?
Yeah, it's a really interesting question.
I think there are a number of things that make open source
projects successful. I think there is a degree to
which obviously you need to be solving a problem which is
really important and top of mind for people in a given moment.
There is an element of timing around that.
I started building wire mock at the point that rest started
becoming popular. And then a few years after that,
micro services became this thing that everybody was talking
about. And it particularly with respect
to micro services, I think it had reached a maturity at the
point that people really got into doing that, where it was
the tool for the moment and it became very useful in that
regard. I think there's a number of
design choices that I'd made that turned out to be good ones.
I guess that made it a popular and useful tool.
I'll go into that a little bit more in a moment, but I think
there's another element as well, which is to some extent about
kind of showing up and doing the boring things and doing them
repeatedly. So writing documentation, fixing
bugs, being available on a support channel of some kind so
that people can interact with you over it and demonstrating
that you're there for the long term to support the project.
I think in terms of the API design, it's interesting you
mentioned SOAP UI. Actually, I think there there
was a, because it's sort of in addition to the the transition
away from SOAP as the kind of predominant API style.
There was also around that time a sort of transition in the way
people were building software and the devil movement was
getting into full flow around that time.
And the sort of the desire and the kind of the impetus to
automate things and to make things a lot more kind of
developer centric and work that way was also a kind of a trend
at the time. And I think the tools of the
sort of SOAP UI generation that tended to be kind of customized
IDs essentially were not always necessarily a great fit in that
context where you wanted a sort of code based or an API based
primary interface onto the tools that you you used rather than
AUI. And one of the ways in which
Wymock thrived is that it was right from the outset that there
was this design principle I wanted to kind of hold on to
which was everything should be representable as data.
So there should be an externalized data format.
It should be well documented and something you can check into
source control and something which is kind of legible by
human beings rather than just this thing that gets dumped out
wholesale by a tool just as a way of kind of persisting its
state into a file system. And the idea was that this data
model was at the core, but then you would have ADSL kind of over
the top of that as a nice way of expressing that data
programmatically and getting all get all the benefits you get
from a full programming language.
And I think that combination yielded a lot of benefits.
Some of them that I was maybe predicting when I made those
choices and some of them that I wasn't having everything is
expressed as a sort of externalized data format allowed
a degree of interoperability in ways that I think I wasn't
predicting. So maybe some more obvious ways
a lot of people went off and wrote other programming language
bindings. So I think to take this sort of
eight or nine third party programming language is
supported as clients to wire mod that I had nothing to do with
writing. There are sort of some other
benefits as well, ones I've noticed much more recently
actually. So being able to by describing
things as data rather than as code, you can make kind of
inferences about what that data means that you you can't very
easily with code. So a good example is in our
commercial product, we have this kind of two way generation
between open API and mocks and getting from a stub definition,
which is describing in a sort of declarative way, I guess, how a
stub should respond to an input document.
It's far easier to take that and then turn that into a useful
open API element. Whereas the alternatives that a
lot of previous generation tools did where they'd kind of say if
you want to do anything complex, you kind of have to break into
scripting. You have to write a Groovy
script or a bit of JavaScript in order to express this rule.
You can't really then go and pause that and use it in this
other context. So there's some design elements
like that. I think putting lots of
extension points in there has definitely been a good choice.
I think there are times when there's an I can source
maintain. You get asked to put things into
the core of your product and you have to say no because it feels
like this is too much of A niche use case, adding a lot of
complexity. So you are going to have to take
on and maintain that is then maybe not kind of pulling its
weight in terms of the amount that it gets used.
But if you can provide good extension points for people and
encourage them and sort of help them to to take advantage of
those, then you're not just flat out saying no, you're kind of
saying, I'll give you some help towards this, but then you can
do the work to build this particular feature.
So I think that's been kind of quite important as well.
There's probably the main things.
I think there's maybe a few other things I could point to
about the design, but those are the key things really.
I think giving people the opportunity to use the tool in
ways that you didn't necessarily expect as a sort of general
unifying principle. I suppose the other thing that's
worth mentioning as well is focusing on the first run
experience, kind of making sure that when people first encounter
your tool that they get some value from it very quickly.
I think that the same kind of heuristics that you apply when
you're first building a start up, I think also apply here
where you sort of get 10 seconds of someone's attention when they
first become aware of the thing that you're building or
promoting or whatever. And then if you can, they
capture their attention in that 10 seconds.
And then they might give you 2 minutes or 5 minutes or
whatever. And then if you can give them
something of value in that time, then they might give you an hour
in order to try and set the thing up.
And you have to kind of make sure that at each of these touch
points, you're allowing people to make meaningful progress or
to sort of have a moment of enlightenment where they realize
what the value is that you're delivering.
I don't think I was concretely thinking about it in these terms
when I first built it, but I think I just had this
instinctive sense that if you can give people little snippets
of code that they can paste into to their projects, and the thing
will just work. Because when I first started
building Wyomot kind of programmatically set, standing
up an HTTP server in Java was quite an onerous process.
You had to understand slightly opaque APIs, I suppose you could
say, in the web servers that were available at the time.
So part of the real value of Wyomot was you could paste in
the original version. It was AJ Unit 4 rule.
So you could paste in these two lines where there was an
annotation and a class being UW. And in the background.
That would mean that the the web server would would start up,
bind to a port, configure itself in a, in a way that meant you
could start using it and then shut itself down at the end of
your test. And you didn't have to worry
about any of that and any of, you know, sort of jetties
internals or anything like that. So I think that was quite a
powerful tool to to to gain adoption.
Well, thanks for sharing all these learnings and insights,
right? Maybe for those people who, you
know, just started their career in the last five to 10 years,
right? Maybe you would think this is a
natural thing to do, like code as configuration, code as data,
and all this fast bootstrap, right?
You know, like starting up web server in one line or just
annotation. I think these days these things
are quite ubiquitous. But back then I think it was
maybe more revolutionary. So I think kudos to you for
thinking in that way. And I think looking at the
success of Wire Mock, right, it's been around for quite a
number of years. I don't get the chance to talk
to so many open source founder who have this kind of project,
right? And the last I checked, you have
about 246 contributors on the GitHub.
So that is kind of like quite a lot out of developers, you know,
like, and not many companies also have like that number of
developers, right, working as a team.
But you're having this as part of the open source team, open
source project. So maybe tell us a little bit
more share for those people who want to build big open source
project, how do you ensure that people are aligned or you have
good amount of people willing to contribute and expanding the
project? That's an interesting 1.
I don't profess to be very expert at this, I have to say.
So I, I probably didn't do anywhere near enough for a long
time in the Wymot project to sort of promote contributor
accessibility. But in Wymot, the company, we
had a head of community working for us for a little while and he
actually, he moved things forward enormously in that
regard. So he did all the sort of
obvious things around improving documentation, kind of
signposting people to contribution guidelines and so
on. Put things in places where you
would expect to find them in GitHub, helped clean up a lot of
things that just made it difficult to get started working
with the source code, all of that kind of thing.
So I think there's sort of a load of project hygiene factors,
you could call them that. I suppose it's kind of the first
run experience for contributors rather than users.
And while I'd focused a lot on the users first run experience,
I think I'd kind of neglected for a long time the contributors
one. Really, you want to be able to
check the project out, build it, run the tests, open it in an IDE
and be able to kind of poke around and see how things work.
And if you make it so that somebody's got to install a
bunch of tools that can't be easily installed automatically
and they have to jump through a lot of hoops before they can
even start doing that, then you set the bar very high for them
to show up and make a contribution.
And they probably won't. I think setting clear guidelines
around how you accept contributions and what you'll
accept and what you won't. I mean, I've always tried quite
hard to say to people, please come and so speak to us before
writing a load of code because it's always awful to when you
can see that someone's put loads of effort into building
something new and you have to say, look, sorry, this probably
doesn't belong in the core. So this is we're not going to
merge this. So I think trying to be as
upfront as possible about that kind of thing so that people
feel like their time's being valued while they're
contributing is definitely a good thing.
Yeah. So I think you emphasize again
the developer experience right before this term getting
hijacked lately because of developer productivity and that
thing, right. But it used to be a term, right?
Developer experience, you know, the first time you check out the
code, how do you get up to speed very fast API, the CLI
experience and things like that. Thanks for emphasizing that
again, because I can see that so many successful open source
project, simply they have this investment on developer
experience. And also not to mention the
contribution guidelines and being being like a safe
community for people to contribute, right?
I think that's also another key. Let's go to the wire mock
itself, right? So maybe tell us what is wire
mock? Why people should consider using
tools like wire mock? So in a nutshell, wire mock is
an API mocking tool. What it allows you to do is to
simulate the behaviour of networked APIs over the wire.
It's conceptually similar to kind of object mocking or encode
mocking for those that are familiar with those kind of
techniques. But instead of substituting
implementations of interfaces within your code or something
along those lines, substituting function implementations, you're
instead kind of going outside of the process and you're
substituting something on the other side of a network
connection. So if you're testing an app or a
service that needs to call out to an API, it can still call out
over a network interface it and you're still, when you're
testing with API mocks like the sort of WI mock provides, you're
still exercising kind of all the production code paths around
networking and serialisation and all of those things that kind of
have to happen in order to integrate with an external
service. And there's a number of benefits
to doing this versus doing object mocking.
I would argue that the particularly kind of micro
services and systems being increasingly composed from large
numbers of networks components, a lot of the complexity of our
systems has kind of moved onto the wires or has moved into the
code which governs what happens on the wires.
While you can create abstractions within your code
base for the things that happen on the wire and then do all of
your testing with respect to those, you know, you're
essentially abstracting away a lot of the real world complexity
which is going to bite you in production if it isn't correct.
The the the benefit of of doing the kind of over the wire
mocking is that you're bringing into your inner loop of
development, I suppose, the the ability to learn and get
feedback about that real world integration.
You know when you're working with something that closely
resembles a real API, rather than your own abstraction with
your own set of assumptions about it as the the thing you're
integrating with. Right.
So I think it's really interesting, right?
For people who may not experience using this kind of
game mocking tool, if you refer to the testing pyramid where you
have the unit test, integration test, end to end test, maybe API
test somewhere as well, where do you think why a mock sits in
that pyramid? Can it be in multiple layers as
well? Like how do you think we should
apply this kind of API mocking tool in the testing pyramid
paradigm? It's funny you should mention
this actually. I wrote a blog post on this a
few weeks ago which garnered a lot of vigorous conversation.
You could say what I've observed in practice over the years of
doing lots of projects and using API mocking as as part of them
is that the really useful kind of band of testing migrates to
the middle. So you get the testing trophy
shape more often than you do the the testing pyramid shape.
I kind of have this sense you shouldn't be aiming for a
particular shape at all. You know, the the shape as a
heuristic is probably that I think there are better ways of
figuring out how you should balance your testing strategy, I
suppose. Whereas I, I think kind of back
when I first started developing, well when I first started doing
test driven development, anything above the unit test
layer, anything integrated at all tended to be very expensive
and difficult and tests were costly to build.
The infrastructure around them was costly to build and running
them tended to be difficult and slow and error prone and so on
like that. So the reason for the testing
pyramid being the shape it was is that if you push as much as
possible down to the sort of unit layer, then things will be
sort of fast and easy and so on like that.
But I think the combination of, as you mentioned, sort of
frameworks and servers and all that kind of thing shrinking to
be small and fast boot up and all that kind of thing.
Combined with the the availability of tools like wire
mock, you can now do far more of your testing.
What I think of as the Goldilocks zone in the middle,
where you're isolating individual apps and services by
mocking out sort of outside of them on the network, but you're
still get getting the benefit. Most of your tests are
exercising real production code paths.
Combine that with sort of improvements in things like
observability and debugging tools, you can get the sort of
the maximum bang for your buck I think a lot of the time by
writing tests that way, if that makes sense.
Right. And I think the emphasis here is
also not in terms of the so-called the functional
accuracy of the input output, right?
But it's more also like to simulate like what you mentioned
like the Http://behaviours the networks, right?
So it's more like an integration kind of thing, right?
And I think these days people are also familiar with this
thing called contract testing. How do you actually
differentiate between API mocking and contract testing?
Are they the same or are they different?
If different than in which part? Yeah, it's an interesting one.
They're they're kind of adjacent concepts and mocking is often
used as part of contract testing.
So actually why mock is used as part of the Spring Cloud
contract testing module. The way the way sort of those
kind of consumer driven contract testing tools tend to work is
that you define a mock, which is kind of just sufficient for your
client code, the thing that you're building.
And then you run some tests with the tool and the tool then sort
of derives a set of tests that it can apply to the server based
on what you mocked. So mocking is kind of integral
to the process in that regard. More abstractly, the notion of
contract testing in the context of mocking is really important.
I think that there are a whole load of benefits to kind of
building things with mocks beyond.
There are benefits around sort of things like environments,
stability and so on like that as well.
But I think there's also a similar benefit that you get to
kind of mock driven test driven development where you are kind
of saying, this is my interface definition, this is the thing
that we're building to and there are going to be two
implementations of it right from the off.
There's going to be the mock implementation and then
eventually there's going to be the real implementation.
Yeah, I know there is a contract that both must adhere to and
contract testing is valuable in that context as well.
The conversation happens a lot. 1 of the biggest sources of
scepticism about mocking, I suppose.
I'm sorry, I'm going off on a slight tangent here, but
hopefully it will make sense. One of the sort of points of
resistance around mocking that you hear a lot is this idea that
it can't really be trusted. It's not really realistic.
And you see some organisations who will do the vast majority of
their testing in a sort of completely integrated
arrangement. And it's this kind of testing
where all of it is kind of, I guess the sort of slightly
diffuse goal of we're going to exercise the system in a way
that the a user would and then we're going to see if anything
goes wrong with it. And we're not going to attempt
to kind of categorize tests by risk with managing or anything
like that. We're just going to do a whole
load of testing and maybe things will break or maybe they
weren't. This tends to be a very costly
and inefficient and, you know, in some ways very kind of error
prone way of testing. And an alternative to that,
making effective use of mocks, I would say is to have an explicit
kind of segmentation of what risks you're trying to managed
with different kinds of testing. 1 type of testing you're doing
where you're saying, given that my assumptions about the
external contracts I'm integrating with are correct,
does the thing that I'm testing actually work the way I expected
to? Is it functionally correct?
And then outside of that, you can have a usually much smaller
set of tests saying, are my assumptions correct?
There's, there's still the, the, the place for kind of integrated
testing in order to say both in terms of contract testing and
some fully integrated testing, where you're saying, are these
contracts being adhered to by the, the real system where you
know, the mock contract and the real contract the same thing.
But by sort of separating those two risks out, you can save
yourself a whole load of, you know, energy and runtime and
labour and all sorts of things versus sort of saying, I, I just
don't trust this. I'm going to do everything fully
integrated. Yeah.
So I think when you mentioned risk based approach, right, I
had a previous episode also covering about this.
I think tests should be driven by the kind of risk they want to
cover, right. So it's not like you can test
any different permutations are available out there.
And also not to mention when you integrate with third party APIs
that you don't actually have control or you don't have like a
line of communication with, it's very hard to simulate corner
cases, H cases behavior that and sometimes could happen, right?
But it is random instead of, you know, like you could drive it
from a certain input, right? So sometimes all this can be
simulated through a mock behaviour, right?
You know, using wire mock or some other tools available out
there. So I think thanks for mentioning
about this risk coverage, right one.
More observations about you make about that actually.
So it's, I'm glad you mentioned that aspect of it because
another challenge, I guess within testing is the economics
of testing sort of different scenarios.
And quite often there are things that are relatively
straightforward to test where there's happy paths or near
happy paths and don't require a lot of data in order to
implement that kind of thing. And then as you kind of step
away from those cases, often things get progressively more
difficult if you have cases where you need to load the
systems you depend on with very specific data or very large
amounts of data. And then as you alluded to, I
think the cases where you want to simulate failures or errors
that you can't really make these systems produce on demand, they
become sort of difficult or impossible.
The other downside of integrated testing is you tend to gravitate
towards the tests that you can practically implement rather
than maybe the ones that are actually most valuable in
managing risk. And the advantage of bringing
MOX into the equation is that they kind of flatten the
economics out. The difficulty involved in
testing some weird edge case is about the same as the difficulty
involved in testing your sort of #1 happy path.
Right. And it's, it's very important,
especially in the integration kind of scenario, right?
There will always be certain cases that you never thought
before and sometimes when it happened, you want to cover it
as well, right? So maybe in the future so that
it won't happen again. So again, like this is a quite a
common, typical use cases whenever you integrate with
third party APIs, right? In the explanation you just
gave, you mentioned that actually a mock driven
development kind of thing, right?
So I think this is quite similar to the concept of API design
first or something around that, right?
So you know, I trader of Weimock, what kind of
development workflows should people do whenever they want to
integrate with third party APIs? Or you know, in the micro
service environment, is it always the case that we should
always come with the API first? So I I think in certainly in
cases where you're in environments where there is an
API that needs to be built and there is some opportunity for
collaboration between consumer and producer.
So very typical in micro service environments, you have this
problem of a new product feature needs to be built, but there's
this API and it doesn't have this API feature yet.
And the mobile team need it and the web team need it, the data
team need it. So there has to be this kind of
collaborative design process. And I think it can be very
valuable to build things upfront.
So, so design things upfront and to actually think about these
things rather than writing code first and then generating the
design from it for all of the usual reasons.
You know, sitting and thinking about things and designing them
and actually sort of mulling over the consequences of
designing things a particular way generally produces better
results than just kind of bashing the keys and going for
it. But I think also specifically in
the context of APIs, I guess, and this is another facet of
kind of organizations that rely heavily on integrated testing.
I think there are places where internal APIs are kind of like
these opaque pipes where two things talk through them, but
nobody really cares what's going on as long as the system
functions overall. And that's maybe OK when there's
sort of a producer and one consumer.
But then if The thing is to be genuinely an API, if at some
point there are going to be 10 other teams who are consuming
this, then they're not sort of driving the design forward
thoughtfully. So if you continue to take that
approach where if these two endpoints can communicate, then
it's all fine, then you'll probably make erratic design
choices that aren't really consistent with the organization
style or the API style. You'll probably introduce
breaking changes. You'll do things which are going
to make it harder and harder for more and more people to adopt
this. So actually treating it as a
proper API is something which is to be Co designed and evolved in
a way which avoids breaking changes and surprises and all
that kind of thing is really important.
Where I think mocking fits into this is that mocks can be
prototypes of APIs. I mean, this is something that
our cloud product is particularly oriented towards is
the idea that when you're designing an API, you want to to
kind of get it into people's hands so that they can try it
and they can validate it as quickly as possible.
I think a lot of API first type tools stop shorts of actually
giving you something practical you can work with, you know, so
you have a design document and maybe you have a bunch of
governance rules that you can run against them, which is
great. The way I think of it is they're
kind of validated by inspection. You know, you have a lot of
people looking at them and going yeah, OK, this looks about
right. Whereas really what you should
be doing is giving it to developers and saying OK, go and
code something, try and build some version of the thing that
you want to use this API feature for.
You can guarantee that nearly always there'll be that oh
moment where you realise that despite the fact you spend 3
hours in a design session talking to people about what
this API design should look like.
The second you try and write some code that uses it, you
realise some really obvious thing that you've missed some
fields that you're absolutely going to need in the data or
otherwise you can't proceed with your workflow.
And it's that kind of thing. You know, the sooner you kind of
get it out to the, the woodwork, the better.
Particularly in organisations that are built where APIs are
being built is kind of as facades over legacy stacks and
so on like that. Or where the the cost of
implementing an API feature is very, very high and itself
involves kind of lots of coordination, then the value of
sort of shifting left, that kind of feedback point where you
discover whether or not the API design is right.
It's huge. You know, you see these banking
environments and so on like that where an API is facade over
decades and decades of legacy tech and it can literally take
months to surface 11 new piece of data.
I mean, there was a friend of mine who works for a big bank
who was saying he had to sort of project manager a kind of three
month piece of rework because there was 1 field missing from
an API, and that involved this stack of five teams going all
the way down to some really old text to expose this field.
So yeah, personally I'm a very strong believer in using mocking
as prototyping as a way of surfacing those problems early
so that you can deal with them as cheaply as possible.
Yeah, I used to work in a bank as well and I could really
relate that experience, right? So changing especially when it
involved a lot of coordination with different teams, especially
distributed teams some more, right?
So it's always a good idea to nail down the interface, the
contract, the API spec and things like that before you
actually start implementing and figure out the issue later on,
right? So if you can kind of like shift
left the API design, I think that will be definitely useful.
And besides, if everything really to the spec, right, like
you nailed it down, you implement as expected by the
spec. I think when you first
integrate, it would work typically quite beautifully.
Like everyone will just be happy simply because you have verified
everything in a simulated behavior.
And when it comes the real production one, it actually
works really, really well. So thanks for mentioning about
this, right? Another aspect of using tools
like wire mock you mentioned is about developer experience and
developer productivity. So maybe we are not talking
about, you know, like open source developer experience, but
it's more like the developer experience team, right,
engineering team and productivity.
So tell us what are some aspects that you think tools like Wild
Mock helps in terms of improving developer experience and
productivity? Sure.
I think we've touched on some of these already, but the biggest
aspect is simply that in a lot of environments where you have
lots of APIs and lots of different types of API, varying
levels of sort of developer experience and accessibility
themselves. They come from different
sources, different vendors, all that kind of thing.
Your development environment, the one that you as the
developer are working within, where that environment is
made-up significantly of kind of other people's APIs, the
stability. And as I say, they think the
kind of the developer experience that those APIs themselves
expose has a huge impact on your ability to be productive.
So, you know, concretely, if you're integrating with a third
party API, which is old, it's maybe run by a vendor who didn't
put a huge premium on developer experience.
And as a result, it has a sandbox which is slow, flaky,
maybe not always running quite the same code as the production
system, hard to get large amounts of data into, doesn't
have the capacity to do any performance testing.
All of these things will impact you directly as a developer, you
know, by destabilizing your environment and as if you're
working on a sort of highly integrated piece of software.
Every one of these external AP is that isn't presenting you an
excellent developer experience is kind of degrading the quality
of yours by a little bit. It's a little bit like sort of
availability where you're, you know, if you're in a highly
networked environment, kind of every non perfect sort of
availability number for each individual dependency reduces
yours by a proportionate amount. And it's similar with developer
experience. And I think if you're wrestling
with dozens of different third party sandboxes that have all
got these different sets of problems associated with them,
then you can spend a lot of time not actually doing your job.
It's not just third parties, it's APIs built on top of legacy
systems, sometimes sort of commercial off the shelf
software that's sort of installed on premise.
And maybe you've only got one kind of non production license
for us where everyone's sharing the same environment.
It's running on some ancient server infrastructure that no
one wants to buy any more or so you can't run any more of it and
all of these kinds of problems. So mocking really get lets you
kind of build this sort of insulating wall, I guess around
your own environment. You can kind of say, I don't
need all of that while I'm doing my development.
I'm going to keep it. I'm going to sort of build an
environment that I can fully control and, you know, get that
kind of determinism and performance and all of those
things that I need in order for my developers to remain in flow.
Yeah. So I think maybe it's slightly
related to that as well as if let's say those third party APIs
or systems, right, needs kind of like a hardware driven, you
know, like you have to install something on a certain location
probably that's also hard to simulate, right.
So I think if you are able to mock that, I think that would be
perfect as well. And I think you mentioned
something quite interesting, right?
These days, I mean, many people work with, you know, SAS, APIs,
even micro services within their environment.
And as we can see, the trend is going more and more towards that
distributed system, right? And there are a lot of
challenges definitely working with distributed services.
So maybe in your view, what are some insights that you think we
can think about in terms of improving our experience in
working with this type of distributed system?
Yeah, I guess a lot of the things I've already mentioned,
you know, I think treating the API is the messages you pass
around between these systems as kind of first class artefacts.
However you do that, you know, ones where you make them visible
and legible, you make them things that you treat as
independent artefacts of design, you apply governance rules to
them, you try and make them consistent.
All of these kinds of things will make it easier.
And then I think, like I say, adopting testing strategies that
take advantage of this. So having this notion of I will
do most of my testing out to the edge of my boundary for my
service or my app or whatever I'm interested in.
And I will assume that my assumptions about my contracts
are correct. But then I'll have other
supporting testing strategies that will be about validating
whether that's actually true. It's a sort of a vintage time at
the moment, I suppose for tooling generally in the API
space, I mean, but there's lots of progress happening around
standards at the moment. So open API has added a Razzo
and overlays and all these kind of things.
So I think the richness with which you can describe AP is and
you can increasingly sort of describe facets of APIs in ways
that are useful for them, verifying them, observing them,
documenting them, all that kind of thing.
You know, I think that both the standards and the tooling around
those standards is improving a pace at the moment.
Yeah, there's this sort of diffuse area, you know, kind of
referred to as API observability, which I think
looks really interesting at the moment where the the sort of
consolidation around those standards and sort of open API
in particular that kind of allows you to then look at API
traffic and draw lots of kind of rich conclusions from it.
So I think it's going to be increasingly necessary to take
advantage of that kind of tooling where you have these
vast API landscapes, you can figure out what's going on
within, you know, when and how things are changing and and what
the direction of travel is. Yeah, you mentioned standard.
I think Open API has been kind of like the go to standard these
days, right? But I could remember back then
when REST was just starting to pick in terms of adoption,
right? There was no, you know, such
tools. And for people who work with
SOAP UI, you know, this SOAP land before, right?
I think one good thing about SOAP is like it's pretty
standardized. You know, it's an interface that
is well defined. You can actually use any kind of
tools as long as it conforms to the spec.
So I think thank God that, you know, Open API exists, right?
It used to be called Swagger, by the way.
So I think that these tools and standardization definitely is a
good thing, right? For us developers so that we can
easily integrate with each other.
There's another trend that I see in API world called API
virtualization or maybe a simulation, that kind of thing.
So maybe if you can tell us what is this actually?
Is this something about virtual machines or something like that?
Yeah, I'm glad you brought this up actually, because this is a,
there's definitely kind of a language problem in API mocking
or simulation or virtualisation depending on how you you want to
look at it. So to to give a very brief
history lesson that might explain this a little bit, there
was the sort of SOAP UI and even kind of before that, I would
argue that past a generation of tools that called themselves
service virtualisation. The product that really
popularized it was Lisa from it was originally from a company
called ITCO and then it was Computer Associates.
And that was they used the category term service
virtualization. And I think this was before kind
of VM Ware and that kind of virtualization really took off.
So it was a lot less of a confusing term back in those
days. Mocking obviously sort of came
out of the agile movement came out of the kind of London
extreme programming way of doing things, I guess.
And I, I thought I was very into that around the, the, the time
that I first started building Wymock.
And so, you know, mocking is a sort of language in a set of
idioms kind of made most sense. And one of the reasons for
Wymock's appeal was it sort of spoke that the language of that
kind of growing cohort of agile developers.
So the mocking thing sort of came from that period and that
set of practices API virtualization, I guess is a
kind of reheating of service virtualization, I suppose.
I, I don't think there's really, I don't feel like there's really
a sort of meaningful difference there.
And then simulation is this sort of term that some open source
tool vendors and commercial vendors have sort of started
using as a further break with the past that's maybe a little
bit more descriptive than than virtualization or mocking.
I think one thing is worth clearing when I talk about
mocking personally, and, and I'm, I'm probably in a,
something of a minority here, I have quite a broadview of what
that could mean. I think it can mean very simple
kind of. Canned responses, you know, the
thing that I think most people associate mocking with all the
way up to sort of quite complex, rich, dynamic behaviour.
We actually did the survey within the company recently
about this and we sort of discovered that most people seem
to see those two as being very different parts of the spectrum.
If you like, you know, the mocking is the sort of simple
canned example type thing that you do when you're writing a
unit test or a narrow integration test.
And then simulation and virtualisation.
And all of that is where you're doing things that are
data-driven or sort of templated and dynamic or stateful or
introducing any of those kind of sources of complexity.
So yeah, I hope that's cleared up a little bit.
I think maybe the convention we're going for is mocking if
it's if you're doing it in code in your inner loop, and then
maybe simulation if you're doing more complex stuff and you're
doing it in your outer loop as a simple rule of thumb thing.
Right. And sometimes I think I see
people also introduce this kind of like fault injection as part
of their simulation, right? So that you can kind of like
simulate the failure behavior that your API sometimes is not
designed for, I guess, right. So thanks for clarifying that
and a little bit of history as well how all this naming
actually came. So thanks for that.
So I think talking about technology these days, we can't
run away from AI. So what do you see AI
involvement in this API mocking and API development?
I don't like making predictions about AI because it feels like
everything's changed within 5 minutes of you making any kind
of declaration. But I guess from what I'm saying
at the moment, one thing that seems to be becoming clear is
that the LLMS need to interact with APIs in order to get things
done. And while there's some
discussion happening at the moment about whether APIs will
go away and agents or LLMS or whatever will just kind of
become web scrapers and we we won't need to build separate
APIs anymore. Firstly, I'm, I'm not so sure
about that because I, I think when you're looking at sort of
consumer facing web-based systems, then there is an
argument that they generally have much better human UIS than
they do APIs. But I think it's a whole huge
swathe of software out there that is only accessible via its
API. So in order to make those kind
of things available to AI applications, it's going to be
necessary to provide APIs that AIS can use.
And it seems that AILLMS are supposed kind of they have a
different taste in APIs and developers do.
And the kind of very normalized DRY style that we tend to go for
in developer API design is not very AI friendly.
That AI tend to prefer things where all the context is kind of
there and present and made explicit in one place rather
than it being, you know, achieved through references and
shorthands kind of all over the place.
So I think there's going to be a move to start building APIs
which are intended for agents and that adopt this very
denormalized style relative to previous generation APIs.
So I guess that's one thing I would say another thing I think
is happening generally is the AI coding assistance.
And I guess you know, the sort of agents that are just starting
to come into the fore now for assisting with coding, I think
are producing a a lot of demand and revealing bottlenecks and
kind of enterprise software delivery systems.
So it's interesting that Google Dora report that came out, I
think at the end of last year, you know, which is quite a big
sort of quantitative study of developer productivity and the
sort of various influences on it.
And one of the things they looked at is the use of AI
coding assistance. And they found that on average,
it was actually there was a net negative productivity impact
where these were present. And my hypothesis on this is a
sort of theory of constraints. You know, that if you make the
thing, it's something faster, which is not the thing which is
constraining your system's throughput overall, then you
will just build up work in progress kind of around wherever
the, the, the actual bottleneck is.
And that a lot of organisations don't have the downstream kind
of software delivery throughput to be able to cope with the more
untested code being produced. I guess add to that that we
probably at the moment trust the code produced by LLMS a bit less
than that produced by most human beings.
I think there's this big problem to solve generally in in terms
of how organisations do end to end software delivery in a way
that can take advantage of the the productivity benefits of
coding assistance. Right.
So I'm sure, I mean, many people would have tried, you know,
coding assistance, right, including generating tests.
And I'm sure people will be able to generate while more compliant
tests through coding assistant, right?
So I'm actually interested in the one thing that you mentioned
about testing API that is going to be used by the LLM or the AI
agents, right? These days people talk about the
genetic AI, you know, agent that can collaborate with each other
and solve a particular task. And these days there are also so
many new tools for doing deep research, right?
Gemini, ChatGPT, Open AI. Also, maybe I don't know whether
you have experience or not, how do you typically see this kind
of API mocking that we need to do in order for us to simulate
an LLM agent actually using our API?
Is there such thing or is this something that we have to kind
of like still figure it out along the way?
I'm not aware of anything built and mature and in the public
domain that's doing this yet. It is something that we're
looking at as a company and trying to figure out.
I feel like I'm not yet in a position to make any strong
statements about what this is or how it works.
You're right, it is possible to get.
So I suppose back to my earlier comment about why I'm not having
a kind of externalized data format.
One of the big advantages of that, and because it's been
around for a long time and the internet's full of examples of
how to create this data, LLMS are actually quite good at
producing Wymot mocks. You know, you can ask for
something in Wymot Jason format, and in my experience, it will
produce something pretty good, pretty useful.
So one of the things we're looking at is this idea that we
should be asking our AI coding assistance to generate the APIs
that they want. So if you're using an AI coding
assistant in order to build an agent for something, then we
should be sort of asking the LLM for an API design and then
expressing it in what I'm not. So then we can go in and
immediately test it in our Magentic workflow.
This is an idea we're exploring. Whether it will take us anywhere
or prove to be viable, I guess time will tell.
But being able to use mocking to remove non determinism when
you're testing these flows is a valuable use case for it as
well. So where you have a whole load
of workflow steps and some of them involve calling an LLM and
some of them involve calling out to fetch data or perform
operations in external systems, our APIs, you have a difficult
enough problem with non determinism anyway with the
LLMS. And if you're also integrating
with a bunch of sandboxes whose data maybe isn't completely
stable and so on like that, then you're sort of multiplying the
problem of non determinism and how you can sort of repeatedly
test. So I think there's sort of a
more mundane use case there just in terms of limiting the scope
of the things that can change between one test and the next.
Yeah, I think what you mentioned is kind of like a cool use case,
right? So using AI LLM to actually
produce this more compliant, you know, basically creating a lot
of APM mocks that you can test, simulate since the very
beginning. It also aids in terms of, you
know, this API first design that we just talked about earlier.
So I think a really good use case for people who probably
want to build a more robust API development experience within
your team, right? Please also try using this
AILLM. So I think AI LM for me also is
has been my go to thing as well. Like sometimes I feel addicted
these days. So I'm sure like once you
generate a lot of things using AI and maybe one day you kind of
like make it more natural in terms of the workflow, right?
It's not like something that you have to think about South Tom, I
have one last question for you before we wrap up our
conversation. Normally I ask my guests to
actually share this thing called the three technical leadership
with them. You can think of it just like
advice that you want to give to us.
So maybe if you can share your version to us?
Sure. OK.
So I did a bit of thinking about this before and I hope these
don't turn out to be completely trite.
The first one I was going to talk about, it's not really
directly related to the things that we've talked about already,
but an observation about kind of hiring engineers for a team,
particularly in a startup. My bit of wisdom I wanted to
share was to sort of challenge a bit of conventional wisdom, I
guess, which is that if you're building a startup that you
should look for these kind of scrappy, very customer focused,
very business focused engineers who will kind of move fast and
break things. I suppose if that's not too much
of A sort of tainted way of putting it.
You know, while there are developers out there who can be
both scrappy and great engineers, my experience of
hiring is that they there tends to be a bit of a spectrum of
people where you have people who are a great engineers who are
very rigorous and so on like that.
But it may be not sort of business and customer oriented
in the way that the the people at the other end are.
And then the people at the other end have more of those focuses,
but maybe don't produce such great quality code or tend to
introduce technical debt at a high rate when they build
things. The point I, I want to make is
that I think you can hire the rigorous engineers and then you
can teach them sort of how, I guess, help them to, to expand
their comfort zone to, to do the kinds of things that are needed
to make a start up work. So adopting more of a commercial
focus and more of a user and customer focus, but also that
kind of flexibility around things like quality and use of
technical debt strategically and all of that kind of stuff.
And I would argue that actually it's easier to hire people who
are, are great engineers and then show them how to do that
stuff rather than hire people which are scrappy and terrible
engineers and then try to make them good engineers.
It's harder to move in that direction.
So that's number one. The second one I think is I've
probably mostly covered it already, but I, I guess I wanted
to say the making an, an open source project successful.
It's almost a sort of product management cliche really, but
it's kind of about making your own users successful and making
them look good amongst their peers.
And I think a large part of that is if they are going to kind of
stake their reputation on introducing the tool that you've
built into their organization, then you need to show that
you're behind it and that you're serious about it.
And that this isn't some developers whim.
It's a serious project that you're going to document and
support and stand by and like I said earlier, kind of do all the
boring work in order to make a success out of in the long term.
So I think that's number two. And I guess the third one kind
of relates to some of the things we talked about in terms of
working in larger engineering orgs, being productive in
organisations that have multiple teams and many services.
And all that kind of thing is about a couple of key things in
my opinion. 1 is decoupling seems to some extent individuals
from the sort of broader context of the technology enough that
they're, you know, their own sort of working set their
context is at a manageable size. And then secondly, kind of as I
lead you to earlier as well, look for as many opportunities
as you can to sort of shift feedback left, you know, to get
meaningful feedback about the quality and fitness for purpose
of your code to try and make that happen as early as
possible, even if actually that means doing a bit more design
and planning around things like API's and interfaces.
Well, so thanks for sharing. I think it's pretty unique.
I would say the last one, right? And especially also the hiring.
Actually this is my first time hearing about hiring the maybe
more good quality engineer, right?
But try to train them to be a little bit more scrappy, right?
A little bit more fast in terms of producing, maybe not so much
up to the quality standard that he aspires, but at least you
produce business outcome and generate value faster.
So Tom, for people who want to connect with you or learn more
things related to Wyomop and API mocking in general, is there a
place where they can find you online?
Yeah, so I mean, LinkedIn is my main watering hole these days,
so you can find me there. And yeah, if you want to drop me
an e-mail at any point, I'm Tom at wyomop.org.
Thanks for that. I'll put it in the show notes.
So thank you so much, Tom, for sharing this conversation today.
I think we all learn a lot about API mocking and best practices
on doing that. Thank you very much for having
me as well, it's been great fun.