We talk about this a lot in the book, but the communication is
the key to this, and contract testing facilitates that.
It's that communication facilitator while you're
developing software, things can change.
It's very easy for humans to make mistakes, and so that's
where contract testing kind of comes in, is we're saying that
we know things are going to change, and so we want to
capture that and help people debug those issues because
breaking changes can be difficult to identify.
One of the main ones is that faster feedback because you're
not relying on an environment and you can point to the
contracts that are potentially not even live yet in an
environment and you have control over your own code base.
So getting that fast feedback early is a huge benefit.
But you should definitely have more contract tests than
integration and end to end. But it's not to say that it
replaces those. So you still need integration
tests for your configuration for some of your business logic.
Hello everyone, welcome back to another new episode of the
Technical General Podcast. Today I have with me the author
of Contract Testing in Action. It's still in early manning
release, right? But I think we can talk about
the contract testing today. So if you're into testing and
testing techniques, I think today we'll learn about this
technique, which is kind of like getting some tractions because
of the micro service trend. And Louis Prescott is here with
me. So thank you so much for your
time and looking forward to learn from you about this
contract testing. Thank you so much for having me,
so pleasure to be here. Right.
So Louise, I always love in the beginning to ask my guests to
probably share more a little bit about yourself.
So if you can, mention any highlights or turning points
that you think we can learn from you.
Yeah, of course. So yeah, I started out, I did a
degree in psychology and then I stumbled upon working in
software for a small business that worked on like psychometric
profiling, so analysing people's behaviours.
And obviously they had the software that went along with
that. I worked out that the software
side of things was really interesting and obviously the
software testing. And from there I joined a
graduate scheme with a testing consultancy who trained me up
and gave me the foundations that I still use today.
So that was a real big turning point for me, right?
Complete career change to what I thought I was going to be doing.
And then that's really helped me kind of push through my career.
Worked in lots of managerial roles within testing, learnt to
code on the job as part of the consultancy, which was
completely foreign to me when I started.
And I, yeah, it took me a long time to get to where I am today.
And then, yeah, I've worked for some really great businesses, so
some start-ups and also a company called ASOS, the online
retailer works in a really great environment there, kind of cross
functional teams, very forward thinking.
And then worked in the healthcare space for a long time
and I've recently joined IBM and I now work on the NHS app as a
test specialist on there. So yeah, lots of variety of my
journey, but yeah, kind of consistently in testing.
Thank you for sharing your story.
So one thing if I may follow up right from what you shared
earlier. So in the beginning you started
studying psychology and you even work on a system that work for
psychometric test. So just wondering how do you
actually test this kind of system?
Because I mean like the input is mainly human behaviors and
they're answers and how do you assess the output?
Yeah, of course. So all of the assessments are
based on research. So all of them are validated and
verified based on their like personality profiles.
I can't remember the name of them at the moment, but like the
kind of well researched, well backed ones and then the
software side of things with purely people being able to
access the form and being able to enter the results.
I wasn't doing any of the like interpretation or any of the
research, just purely the software side of things.
Right. So I could imagine it might be
quite difficult to actually assess whether the accuracy of
the calculation or the categorisation, right.
So I think that must be a challenge altogether.
Leave that to the professional issue.
Right. So today we'll be talking about
contract testing. So you're writing this book at
the moment, right? So in the 1st place, maybe many
people might have heard about contract testing or might not
have heard about contract testing.
I personally have heard about it, but haven't actually seen it
in action in any projects that I have.
So in the 1st place, maybe tell us why contract testing exists,
what kind of problems that it tries to solve.
Yeah, absolutely to my journey was started learning about
contract testing, completely foreign concept, really like
found it hard to get my head around it until you realise what
the problems it solves. Obviously most of the people
that come to concept testing have familiarity with API
testing, integration testing, that kind of background.
And so those tests are quite unstable, rely on test
environments to be available, integrations to be configured,
and all those kind of factors contribute to those tests, be it
flaky or taking a long time to run.
So that's one of the areas where contract testing comes in.
Another kind of area is breaking changes to your API, knowing who
is consuming your API, how they're consuming it, what
changes are going to break that contract, and generally building
software in a way which is backwards compatible, so having
that confidence to deliver that software going forwards.
So contract testing comes in and it kind of solves that because
it gives you fast feedback, it gives you the versioning of your
APIs through the contracts. And yeah, we'll go into more
detail about what contract testing is and stuff, but yeah,
they're the kind of main areas where it really helps you in
that software delivery life cycle.
Right. I, I could imagine last time
when I, we used to work a lot with service oriented
architecture on micro services, right?
So I mean, in the beginning, maybe it's easy when you have
one to one relationship where you have producer and consumer
and any change you can just communicate, right.
But I guess since the era of micro services and
interconnectedness with, you know, so many SAS applications
or maybe API is available out there, right, You start to have
this permutations of possibilities of API
integrations. And I think one of the challenge
that you mentioned, right is breaking change.
Initially, I think we all just rely on the contract, maybe open
API spec that you just distribute around or you just
tell somebody, hey, we are going to change this and you need to
change that. So I think all these definitely
is one of the problems that we foresee when you have a lot of
API integrations. So tell us a little bit more how
contract testing is being utilized maybe in this kind of
era of micro services? Yeah.
So microservice is definitely like the biggest use case for
it. Obviously, the more the
complexity goes up and the more integrations you have, then
obviously you want to scale those things.
And so contract testing comes in and effectively the most similar
comparison is when you're using a mock to mock out your
integrations. That's where contract testing
comes in and it says, OK, you're interacting with the service in
this way. That forms part of the contract
and this is what you expect to get back from that service in
order to render that on the page or store that information in the
database. So the contract is formed from
the request and the response and that forms that contract.
The web application or the mobile application then
generates that contract and puts it in a central place.
So using the tool packed, it's stored in a packed broker.
And then the person who creates the API or is the author or
whatever the owner, they then replay that contract on their
side. So it's about forming all of
those different contracts with all of the different scenarios
that you interact with that integration point, for example,
error scenarios, different query programs, all those kind of
interactions and putting those into contracts and then getting
the the provider, the person, the owner to replay those
contracts. So yeah, it's passing around
contracts which have those expected results in basically.
You said something about mock, right?
So I think if I get it correctly, right, So we are
actually not testing the actual service that you are calling or
maybe you are returning back to, right?
So you're actually utilizing mock and something like a replay
kind of mechanism where you send a particular request and you
expect some kind of a response back.
So I think 1 area as well that I learned from what you just said,
right? There are many kind of
components that exist in a contract testing.
You mentioned something about central place or broker, so
maybe tell us a little bit more like what are these components
that exist in a contract testing so that we get familiar a little
bit more about it? Yeah, absolutely.
So the terminology we use in contract testing is you have a
consumer, so that's your web app, your mobile app, and as a
service that's your consumer. The provider is the API or the
event mechanism that generates those events, and then they are
to provide them. Then you have to store the
contracts. So where you're going to store
them, that's called the broker. That's where they're all stored.
And then there's another component because as you
mentioned, the services aren't communicating with each other,
they're just communicating via a contract.
So then there's the CLI tool which allows you to query the
broker to say, has this contract BeenVerified?
And within the packed world that's called can I deploy?
So yeah, it's just a way of querying it and saying, does
this contract has it's BeenVerified by both parties,
OK, we can release. So they're the kind of four
components. Right.
Just to recap a bit. So there's a consumer, so which
is the one who consumes the API, right, The providers, the one
who will produce the API that says broker where you store the
contracts. Of course, there's a contract
itself there, the component, the Great Expectations between both
parties. And then the last one, you
mentioned there's ACLI tool that we can utilize to query the
broker to check about the contract and the expectations
and all that, I guess. So I think in your definition in
the book, you also mentioned something that I think is quite
interesting to highlight here, right?
So contract testing is actually a form of testing where you are
verifying 2 systems have actually the same shared
understanding about the expectations.
So tell us why this is important to have the shared
understanding. I think in the requirements
world it is. I've also mentioned a lot about
shared expectations, you know, from the business and also the
technology team, right? How does this share
understanding also play a part in the contract testing?
Yeah. So it's, it's, it's super
important, right, when you're dealing with multiple teams,
distributed teams remotely, all that kind of thing that we share
the source of truth and also why you're developing software.
Things can change as you're developing.
So the way that we mentioned about generating contracts is
via an open API specification, which is potentially generated
after the code is built. Or you could mock it up before
the development starts and share that around.
How do you share that around, right?
You share it in a wiki page or you share it in a central
location where in a GitHub repository or something like
that. All these things require human
intervention, right? It's all, I've changed the
contract. I need to remember to
communicate that I need to update it in the wiki and all
these kind of things. Like it's very easy for say,
humans to make mistakes. And so that's where contract
testing kind of comes in is we're saying that we know things
are going to change. We know that the specifications
might change over time. And so we want to capture that
and help people debug those issues because breaking changes
can be difficult to identify. And you may have lots of
consumers and they will use different pieces of the API and
stuff like that. So that sharing of that contract
is super important to enable this.
And then the other scenario, right, is you have teams that
are working on two different code bases, front end of the
back end and they're writing in two different languages, they're
developing at exactly the same time.
So like the rapid changes that are happening, you want that
shared understanding to continue over time.
And so yeah, creating that central place to store it, to
enable you to version it and everything like that.
That hopefully answer your question.
Yeah, certainly. So I think thanks for explaining
that, right? Because it's very important to
realise about this shared understanding because I think
maybe one to one again, it's quite simple, right?
You can just back and forth converse or this is my
expectations, this is the behaviour and this is the
response of the output, right. But once you have when you start
to have a permutations like for example, even multiple versions
of the APIs, right, the evolution of the APIs, the
backward compatibility. And then once you start having
more and more consumers, this is where it gets tricky, right?
Because you will not be able to trace all the changes that
happens and how it will impact those teams, right?
Or those consumers. So I think contract testing
definitely is very interesting. So for people now who understand
a little bit about contract testing, so tell us maybe some
of the benefits that maybe you have seen in the real world
where you actually implemented contract testing.
So what are some of the obvious benefits that you can share so
that people can understand exactly what kind of value they
will get out of this? Yeah, absolutely.
And one of the main ones is that faster feedback because you're
not relying on an environment. You can point to the contracts
that are potentially not even live yet in an environment, and
you have control over your own code base, right?
The consumer generates contracts from their side.
The provider spins up their local environment or however
they choose to do it to run the contract tests.
So getting that fast feedback early is a huge benefit.
And independently, right, In that scenario, there's no
dependencies, but you control everything.
The caveat to that obviously is the provider needs to have their
data and you set up their data in some way or a mechanism for
doing that. And then that may come with
complexities. But yeah, so that fast feedback
loop catching those issues earlier.
So not waiting for the changes to go into that integration
environment, which can be potentially later on in your
cycle. And one I mentioned earlier,
right, about the ability to debug those issues, it can be
difficult when you run your integration test because you're
pointing at it from the outside. You get a response back, which
is a 500 or a 400, and it just says bad request.
And you're like, well, I don't know what that is, what's wrong
with my request, that kind of thing.
So like having that contract in place makes it super easy to
replay those on both sides. So, yeah.
And another huge benefit of this flipping of integration testing
to the consumer is it's consumer driven, right?
And that means that the users in this scenario, because they're
the users of the API, are the ones creating the tests, right?
So in terms of being as lifelike and as close to production as
you can get, then these sit nicely in there and give that
consumer view of how the API is being used.
So there are a few benefits that you get.
Yeah. So I would also imagine that you
can deploy things in a much independent way and confidently
as well, right? Because before you actually go
to production, you would have verified all the interactions
that you have with either your consumer providers, right?
You would have verified that all can actually work still, right?
And that's this tool. Can I deploy?
It's like a fancy term, right? Can I deploy?
And then when it's green, then you can safely assume that
everything will just work as it is compared to the previous way
of working where you have tested everything like your unit test,
integration test, E to E test, all working fine.
But then once you deploy to production, right, it's always
the case where something broken because the integration actually
changed the behavior and then you have to quickly rollback.
Yeah. So I think this is very
interesting. I just mentioned about unit
test, integration test, and E to E test.
So there's this testing pyramid concept, right, in the testing
world. So what do you think?
Where does contact testing actually reside in that pyramid,
right? Is it unit test, is it
integration test or is it E to E test?
Yeah, it's a great point. And like contract testing kind
of sits next to the unit test. So when you're generating your
contracts on the consumer side, it sits next to the function
that makes that request because it needs to intercept that
request. And then on the provider side,
it's a bit more like integration level because it pulls down that
contract and replays it on their side so that you have a running
service. So it's kind of a mixture of
unit and integration. So in the pyramid, it kind of
sits in that just below integration tests and just above
unit. So yeah, again, it sits nicely
in that way in that you have more control and you get that
fast feedback a bit further down the pyramid.
So I think it's in the same spirit of a pyramid, right.
So do you think we should have more contract testing versus
more integration or E to E test or like tell us about this shape
or ratio of number of tests? Yeah.
So you should definitely have more contract tests than
integration and end to end, but it's not to say that it replaces
those. So you still need integration
tests for your configuration for some of your business logic of
this API needs to be available at this performance level and
stuff like that. The contract is testing the
request and response. And it's not testing the exact
values that come back, right. So like there's still lots of
unit tests that you need and there's lots of integration or
not lots, some integration that you need.
So it's definitely more, you can fit more scenarios in that
contract level where it talks about the error scenarios, the
different query parameters that you're passing.
So there's lots of different scenarios that can fit into
there and then you still need some of your integration and end
to end. Yeah.
I just want to highlight one more time what you said, right?
So this doesn't replace integration or E to E test,
right? And one thing also important to
note, the contract testing doesn't verify the business
logic or the correctness of the output or the value, right?
So those things should probably be more in the unit test or
maybe integration or the E to E And also it doesn't test all the
other maybe quality attributes like performance, maybe
security, whatever that is, right?
So please take note about that. And I think I'm sure some people
would have tried, you know, like a verifying correctness of the
behavior in the contract testing as well.
So one thing about contract testing that I also learnt from
your book, right? You mentioned that for API
providers that have public consumers, you know, like you
don't know actually how many the consumers of the API or maybe
third party, external third party that you don't have
control over. Contract testing probably is not
suitable for those kind of scenarios.
So tell us a little bit more about this, why that is the
case. Yeah.
So as I mentioned, the consumer driven approach.
So what we've been discussing so far is the kind of consumer
driven way. And so the person who creates
those contracts is the consumer. And so obviously in the public
and the kind of third party integration scenario, you don't
have that input and you can't drive it that way, especially
with the public way, right. Like you don't want it driven by
the consumer, otherwise you'd have an endless tack log of
things like it would just get insane.
Yeah. So from a consumer driven
perspective, it's not necessarily the right choice,
but there's still value in in contract testing.
It just sits in a a slightly different flow which there are
solutions for. It's just not in this consumer
driven methodology. Yeah.
So you have mentioned a few Times Now consumer driven,
consumer driven, right. So I think many people would
probably be puzzled, right, Like what is this consumer driven
like? Maybe now is a good time to
actually introduce how many different types of contract
testing are actually out there. Yeah, there's quite a few
different types. And it's also good to call out
that there are other people coining contract testing, but
they are doing it at a slightly different level and it can be
confused with like schema based testing, which does that kind of
static level stuff. So yeah, it is quite confusing
and it can get quite blurry along with a lot of other terms
in the testing space as well. But yeah, there's two main types
in terms of what is offered by Pact and pack flow.
So consumer driven, whether the consumer creates the contracts,
the provider that verifies them. And then there's what we coined
in the book, Provider driven contract testing, which is also
known as bidirectional because you can kind of do a combination
of both. But yeah, the provider driven
approach is kind of more catered to those public API kind of
scenarios where the API can generate contracts, verify their
own contracts. So open API specifications, they
verify those with their own testing framework and then they
can publish those results to the broker and then the consumer can
then verify those against their own contracts.
So the provider doesn't have to do any additional work and
replay anything on their side. They're just saying here is our
contract, we've verified everything in here is true.
And you can also verify it via the pack broker, but that's only
available on pack Flow, which is the software as a service
version. And the team there who are many
of whom created packed and are part of that, they also have
this pack Flow which stores your contracts and does obviously
much more functionality. It's like bidirectional.
Right. It seems like in your book you
explain that the provider driven contract testing, right, is
suitable for those who start to learn about contract testing or
they come from the legacy of, I don't know, like things like
open API where you have the spec already and you just publish it
to broker and then consumer can kind of like verify it against
the broker, whether the behavior is the same.
And you mentioned about pack and pack flow.
Pack itself is like probably the most well known contract testing
tool or framework out there, right?
The pack flow is like the SAS, the product on top of pack,
right for people who are doing this provider driven contract
test. So maybe after knowing about
these two type, I think the first traditional one is
actually the consumer driven one, right.
So obviously many people would have a challenge in visualizing
how is the workflow like because you have a consumer, you have a
producer, you have a broker in between, right?
And both sides also make changes.
How is the workflow like and maybe in the typical CICD
pipeline manner, maybe if you can just illustrate a little bit
so that people also understand like how would they work with
this contract testing? Yeah, absolutely.
And the key is what we're talking about, right?
It's deploying with confidence. So there's kind of three or four
steps to get to that point. So the consumer generates the
contracts, publishes those to the broker.
The provider then pulls down those contracts, verifies those
against the latest version or whatever version is going to be
released, and then it goes back to the consumer.
So then the consumer will use the can I deploy tool to make
that check to say, OK, I'm going to release this new version of
the contract. The provider has verified it.
Can I deploy to an environment to production?
And so they're the core steps that you would have in CI.
So they would sit into separate places, right?
So the consumer, on the consumer side, they would have their
contract generation tag it with this, the version that they're
going to release and then publish it.
Then the next step they would have in their pipeline is the
can I deploy to say, OK, I want to release this now, can I
deploy that? And then they tag that with the
specific environment that they're going to deploy it to
and the specific version that's going to be deployed.
And then on the provider side of things, they would obviously
have their tests that run in CI which verify that specific
version and publish those results.
So they would only be publishing those in CI to make sure that
it's not sitting on a branch or anything like that.
So they tag it then with the version of the API which is
they're pointing to and the environment that that is
available in. And so from the provider side,
that's it from ACI perspective, unless they also have consumers,
right, which they may have if they're a service that shares
data two ways. And so then they would also have
a Canon deploy step. Yeah.
So that's the kind of cycle of things.
So then obviously once the consumer's ready, they will
deploy that to production and everything will be tagged and
ready. Right.
So I can visualise it a bit, right, The workflow, I have some
follow up questions because again, like this is probably new
to some people as well. The 1st is about the can I
deploy, right? Because there's this gap between
the consumer generating the contract and publishing it to
the broker and actually for the provider to actually run their
tests and verify and update the broker.
So I guess in the consumer CI pipeline, right?
Does it just wait or continuously pull, you know,
like, can I deploy, can I deploy?
Or like what's the trigger like to actually proceed to the next
stage of the pipeline, right? Because you might have, you
know, integration tests, E to E tests and all those things that
are also waiting. Yeah.
Usually you'd have two separate pipelines, right, One which is
building and then the other one which is deploying.
So it would usually be split across those two things.
Obviously if the two are connected together, then you'll
be running that step of the same pipeline just later on, right.
And So what is clever about the pack broker is if the provider
hasn't theirs is compatible with previous versions, then that
consumer can still deploy because it's BeenVerified
against the previous version and there's been no changes which is
going to affect that deployment. So it can then say, OK, I've got
this one three versions ago and nothing's changed since then.
So you can still deploy this or the responses are the same based
on different factors. So you can still deploy with in
that scenario, but obviously if there's a new version then the
can I deploy tool will just return a response to say this
has not BeenVerified and then you would need to trigger the
other pipeline. But that's easily done, right?
If you're working in a CICD environment, then you can easily
trigger another pipeline from yours to get those verifier
tests to run or communication, right.
Like we talk about this a lot in the book, but the communication
is the key to this, the problem and contract testing facilitates
that. If you get the communication
right, you don't need constant testing because everything works
and you don't have any issues about fricking changes or
anything. But it's that communication
facilitator. So that's where, yeah, in CICD,
if it hasn't been verified yet or if it fails, then you can
jump on a call with that team and start discussing it.
So there's obviously a delay and getting that information, but
you should be able to move pretty quickly.
So thanks for highlighting about the communication and the
agreement, right, between those two teams.
So having contract testing doesn't mean that both parties
just don't talk to each other and just rely on whatever tools
that exist, right? So I think communication is
still the key. Again, the key point here is
about share understanding, right?
And you would probably not get shared understanding if you
don't talk to each other. So I think human aspects of the
software development is still key.
So I think that is one question that I have.
So the CI, I would imagine that you can proceed with the rest of
the stages of the pipeline, but somehow before you deploy the
production, you have to join with this.
Can I deploy parallel track to actually be confident about your
change as well? So the other aspect is that the
provider side as well, right? Because now if let's say you
have to support multiple consumers, I would imagine that
your pipeline will get triggered by multiple consumers updating
their contracts. Is that the case?
Or like how does the producer actually work by having all
these contracts that they need to serve and verify?
Yeah, yeah. It's essentially that could be
the case, absolutely. But as we discussed earlier
about it all being isolated and working independently, right,
there shouldn't be any dependencies and obviously that
pipeline gets queued behind another one, then the
verification process will still have occurred, right?
So shouldn't delay anything on the motor for consumer side of
things. But the area that I mentioned
earlier as well about the data side of things, right?
So from a proprietor's perspective, the heavy duty
lifting is the consumer may be sending you lots of scenarios
where the data needs to be in this state and you've got a
hundred of those scenarios, right?
And that's one of the big hurdles in terms of contract
testing because you have to be able to set up that data as part
of your test. And that's a limited factor
really. Obviously there's a way around
that with sharding your database and things like that.
But like, obviously that that requires development time and,
and there's complexities to it. So yeah, the provider's
definitely going to be doing the heavy lifting in that respect
and the pipeline will have to accommodate for that.
Yeah, just by having a better understanding about the workflow
now, right? I can actually imagine this can
be quite tricky to implement, especially if those two teams
are not like close to each other, right?
Let's say they're different departments, they don't use to
work together. So if let's say only one site
wants to implement this, but they cannot convince either the
producer or consumer to also do the same, will this still work?
Or do you think getting the buy in is something that is very
important before you actually embark on this contract testing?
Yeah, the buy in is potentially the hardest part of this whole
journey and we talked about that a lot in the first few chapters
of the book is what we're selling as part of this
contract. Testing in action is not
necessarily just the technical side of things because that is
only part of it. And lots of people will say I've
already got an integration test suite or we already test our
schemas using schema testing tool.
So there's only a narrow window of stuff that we need to cover
that's currently not covered and that's absolutely fine view,
right? It's just if you want to seek
the benefits that we discussed earlier on, then that's where
you'll want to implement contract testing.
And the problem is that usually teams embark on this journey
after they've already got these suites of things.
So yeah, that upfront selling and getting on board early on
definitely facilitates that process.
And then if you start to hit these hurdles later on, then you
can start implementing bidirectional provider driven
approach which will enable you to convert your integration
tests to contract tests. The selling again is part of you
need a broker, right? So you need someone to pay for
that or someone to host that. Obviously there's an open source
version which you can host yourselves, but that still has
running costs, so you've got to be able to sell it.
And so that's why we spend so much time in the first couple of
chapters talking about what the benefits are, what problems
we're going to solve for that, and what the solution looks like
in terms of that life cycle of the CICD.
So you can work out, OK, will this fit into our delivery life
cycle and will this solve the problems that we're currently
having. So yeah, the selling is
definitely a big part of that. In your view, who should own the
broker, right? Because then you have consumer,
you have provider like who should own it actually.
Yeah, it's a great question. I mean, it usually falls under
DevOps, right? Like that kind of team will
usually host it. If you've got a open source
version or if you pay for the software as a service station
with pack flow, then it really doesn't matter who's owning it.
It just comes out of some of the budget.
And obviously because it's going to be shared across multiple
consumers, it will probably be leadership or someone in the
management will be signing off on it because it will sit across
multiple teams. So yeah, but what I say to
people in terms of that question is until you start using it and
like start seeing the benefits of it, then you're never going
to get started, right? So like, even if you have 5
brokers to start with, because you've all set up your own ones
and then you can consolidate them at a later date, like
you've got to get started because otherwise you'll just be
sat there thinking like, what benefit are you going to get out
of this? And also the concept as well,
the concept's hard to understand how it work in practice until
you actually get a proof concept.
So yeah, it's a really good question of one get asked a lot,
but yeah, just get started. So for people who wants to get
started, I know you have mentioned a lot about PAC.
Is PAC the right tool to get started or are there other
alternatives out there if I'm wrong?
I've heard something about like mounted bank before.
I don't know whether that's a contract testing tool, but like
tell us maybe what are some of the tools we can explore?
Or is PAC actually the de facto standard that we should try to
use in the very beginning? Yes, it's a really great
question. So Pact is like the go to in
terms of it has lots of different language supports and
has a core like Ruby base. So they all just build off of
that. And so that's definitely the one
that I would start with if you're looking for one.
But yeah, there's lots of other places which offer some form of
contract testing, like Postman has a version and Mount Mac
might have a version, I'm not sure.
Hacked has got an adapter for the mock, so it can definitely
facilitate that. And then you've got like Source
Labs, Test Sigma, all these now have contract testing solutions.
I read the small print about what it's actually doing behind
the scenes. But yeah, and The thing is, you
don't actually need at all. You could do this with a GitHub
folder, or you could do this locally in a local folder where
you verify your request and your expectations against the
providers. You just need somewhere to share
it. So you don't actually
necessarily need a tool. Obviously the tool provides lots
of shortcuts and it's been doing it for a long time, so it's got
lots of functionality built in to facilitate that process.
But the actual concept of contract testing is just the
concept. You could implement it
yourselves. You could use any of the tools
that are available to you. It's just time.
And communication for sure, right?
So don't forget. So you can use all the tools,
but if you don't communicate, I guess it's not optimal as well.
So I could imagine, yeah, I mean, pack probably will help in
terms of workflow and automation, right, for some of
those steps. But I would imagine you can also
do it manually, although, yes, it's a bit tricky.
You need to coordinate and you have to put the things into the
right place and triggers your pipeline and things like that.
So I think definitely for people.
Thanks for the tips, right? Maybe pack is something that you
should explore. And I think pack also support
some of the advanced use cases, right?
So you mentioned in your book something like can I deploy
maybe web hook, right? And also about this versioning.
So tell us a little bit more what kind of versioning support
that maybe pack supports and why is it important?
Yes, we discussed this bit earlier when we're talking about
that life cycle view of when you're deploying it, what
environment you're deploying it to.
And all that kind of thing. So obviously whenever you
generate a new contract, that generates a new version, and
whenever the provider verifies it, that verifies against that
version. And if a provider makes changes,
then they will update their version as well.
So all the versions are stored within the contract.
So you wouldn't necessarily be changing your version of your
API if you don't have version in built in.
And on the consumer side of things, you don't generally have
a version for I'm changing my interactions.
You just have, I'm releasing a new version of the web app and
you don't have that physical versioning.
So that's built in. And so the versions are then
supported by tags. So if you're developing
something and it's just part of APR or it's on a branch, then
you would tag it with that. And so then you can test new
features against what's in production because the
provider's tagged a specific version in the broker with
production. So then you can point your
feature branch to the production version of the API.
And then the API might move forwards, but they might not
release that to production until later on.
So now you've got that matrix and the Pepper relies a matrix.
So you can see OK, which versions of the consumer are
verified with which versions of the provider.
And it gives you that nice visual.
And so you can develop in isolation without having to rely
on that communication of OK, I've deployed this version.
I'm going to deploy it there for an hour so you can test it and
then we'll switch back to the production version so that we
don't break any of the other consumers.
So yeah, it's that versioning which becomes very complicated
very quickly in a micro services.
Environment, right? So yes, this is probably
something that pack also helps you out-of-the-box, right?
So if you have to support multiple versions, you don't
have to maintain your own matrix, right, All these
permutations of different versions.
So I'm actually still intrigued about this.
Consumer driven, right? Traditionally in the past, the
provider is the king, right? They would just tell everyone,
OK, this is my API, you use it or whatever, right?
So obviously normally there's no room for kind of like the
consumer to drive the API. So how come this time is kind of
like flip, right? I mean, consumer driven contract
testing, the consumer is the king, right?
They can actually shape the contract and the provider has to
follow them. So tell us why this interesting?
I don't know evolution in terms of, you know, the power between
the API provider and the consumer.
So why this makes more sense instead of the traditional one?
Yes, a really great point. So the IT kind of comes from
Postel's law, Postel, however you pronounce it.
So it's be conservative in what you do, be liberal in what you
accept from others. So it's kind of come from that
concept. And the idea is putting those
users first. So when we're developing our
APIs, there's often in requirements.
It's like, OK, this consumer needs this information in this
format. And so it's being driven from
there. It's like, OK, the consumer
needs this information. So how do we put that into our
API? And then it's like, OK, this
consumer needs it in this format.
OK, well, maybe that doesn't sit inside this API because this is
for this purpose. And so it's trying to narrow
down those requirements and trying to make it purely based
on what's going to be consumed. Another example of that, right,
is how often do you have information within the API which
is never used? Obviously you can see within,
you can ask the other teams like are you using these data points
or in the logs you could potentially see, OK, we filter
the data by this, but no one ever filters the data by that.
And so it's like also enabling you through the data to say, OK,
these are the only areas which are being used.
So maybe we can reduce down our API or maybe we need to split it
in two because we're getting lots of traffic and they're
requesting this filter, which is really expensive.
So let's just put that into another API.
So it's trying to have a data-driven approach to breaking
down those requirements to what do we actually need to provide.
So it makes, yeah, it makes a lot of sense.
And then, as you said, about like the power of the provider,
right, is usually the provider will create their tests, which
will say the API returns this. And then when they make a
change, they will update their own tests.
How often do we say like, don't test your own code?
And it's like, well, why don't we do the same with APIs?
Because the consumers should be the one testing it because
they're the ones that rely on it, right?
So yeah, there's lots of examples that I've had in my
career, right, where you just change your own tests because we
did make a change, so we know it's broken, so we change it and
then we've broken the integration.
So yeah, that's where kind of the consumer driven approach
comes from. Right.
I would also imagine maybe sometimes I don't know whether
this is true, right? You can correct me if I'm wrong,
right? So even though the consumer
generates the contract, the provider can still negotiate,
right? So it's not like every different
version of contract generated by consumers, they will just
satisfy, right? So again, communication is the
key. Is that correct by the way?
Yeah, that's a really, really good point.
So obviously if there's a failure and providers fails
because the consumer is asking something ridiculous or even
asking something small, right, like date format back, we return
it in this and you're expecting it in this.
Well, we can't return it in multiple different formats.
That's impossible. Or if there's a specific null
value or like there's a empty or whatever, there has to be a
consistent way to do that. So the provider has to say at
certain points we cannot do that.
And so the provider still has lots of power, but it's just
driven by that communication of, OK, we would like it like this,
how can we come to a solution? Right.
Thanks for clarifying that. I think it's very interesting
take for those people who want to try this contract testing,
right. I think in this era, not just
REST AP is or HTTPAPIS that people normally use for
integrations. So there's this asynchronous
communication as well, or message driven or event driven
architecture, right? Does contract testing help for
that kind of scenario? And if so, how does this
contract testing now play a part in that kind of architecture?
Yeah, absolutely. So event driven is another one
of the ones that recommends for people who are getting started
with contract testing because an event is like a small section of
data and it's like a small piece of communication.
And so it's a really nice way to get started because the
parameters are a bit less and often you're just passing it
around. Sometimes the service isn't even
storing anything or passing it into another system.
So a Venturefin is definitely supported within contract
testing. The message part is all we care
about, right? So it's not tied to any tool or
any event driven architecture kind of facilitator.
So like Kafka or AWS SMS, anything like that.
It's not tied to anything, it's just a message, right.
So if your events are heavily tied to those and you can't
separate the contract message from that, then contract testing
doesn't work. But if you've got a good way of
constructing your event message, then you could definitely verify
that that contract is supported. So I think in EDA you will also
have this concept of producer and consumer, right.
So is it still the case that the consumer is the one driving the
event? Because I would imagine is the
producer that kind of like push the message, right?
So is this still the case for the event driven?
Yeah, I always get this so confused, so confusing.
But yeah, it's flipped on the event of it, right?
So the producer is the consumer and the consumer is the provide,
but check the book because it's definitely right.
But no, yeah, it's flipped. Obviously the terminology is
very confusing, but the producer generates the contract and then
the consumer is very fine. Interesting.
So, OK, so maybe we'll use the different term publisher and
subscriber so that you don't get confused.
Exactly. Right.
So thank you so much for this overview about contract testing.
I think for people who deal with a lot of integrations, this
definitely is one of the tools that you have to utilize in
your, I don't know, CICD pipeline or development process
so that you can get the confidence to deploy your
integration changes right as we reach the end of our
conversation. I have one last question for
you, Lewis, which I called the Tree technical leadership
wisdom. So if you can think of it just
like an advice that you want to give to ask the listeners here,
maybe if you can share what is the version of your Tree
technical leadership wisdom. Yeah.
Thank you. Yeah, hopefully what I've spoken
about makes sense. But yeah, the theme throughout
the conversation has been communication.
So that's definitely my number one is be very open with that
communication and keep those, the avenues of communication
open. And I do think that the large
majority of software delivery is about communication.
So that would definitely be my number one.
My number two is breaking down those barriers of quality.
So dev and test is one thing, right?
Like at the end of the day, we're trying to deliver a
quality product at the end. So breaking down those barriers
and making sure that everyone has that quality bindset built
in from the start. So that would be my number two.
And then, yeah, my third one would just be to keep a kind of
general holistic view of software development.
So I've always been a generalist.
I have my fingers in the lot of pies, but I think that's really
helped me, especially in leadership, is having that
holistic view and being able to contribute to lots of different
conversations. So yeah, that would be.
Thanks for sharing that. I like the second one, right.
So remove breakdown all the barriers for quality result or
quality mindset, right? So I think this is key still in
any software development team, right?
You want to produce product of high quality, right?
And contract testing definitely is one tools that you can
explore for that. So for people who would love to
learn more about contract testing or maybe they are
waiting for your book or maybe a place where they can find you
online, is there a place for them to find you?
Yeah, absolutely. Obviously, you can go to Manning
and search for contract testing in action.
There's live discussion on there.
And then you can find me on LinkedIn, Lewis Prescott and
also on my website, pacman.co.uk.
And we should also mention that the co-author of my book, Marie
Cruz is having a baby at the moment.
So she can be here today, unfortunately.
But obviously, yeah, you can connect with her as well on
LinkedIn and we're more than happy to answer any questions
that you have on contract testing.
Right. So let me put that all in the
show notes. So thank you again for your time
today, Louis. So I hope people learn about
contract testing and they get excited to actually give it a
try. So thanks again for your time.