The API design centers on effective communication and it's
not just communication between developers but it's
communication that combines product thinking, Business and
Technology. All in what?
Hey, everyone. My name is Henry Surya, we Robin
And you're listening to the tekhelet Juno, the show will be
bringing you the greatest technical leaders practitioners
and thought leaders in the industry, who discuss about
their Journey ideas and practices that we all can learn
and apply to build a highly performing technical team and to
make an impact in your personal work.
So let's dive into our Journal. Hello to all of you.
My listeners. I'm happy to be back here again
with another new episode of the tekhelet journal podcast.
Thank you for spending your time with me today, listening to this
episode. If this is your first time
listening to technology, you know, I would invite you to
subscribe and follow this show on your podcast app and social
media on LinkedIn, Twitter, and Instagram, and if you have been
regularly listening, and enjoying this podcast, and love
the type of content that I'm producing.
Will you support the show by subscribing as a patron at
technology, you know, dot Dev, slash Patron, and support my
journey to produce great episode.
Every week building, web apis has become ubiquitous whenever
we build Tech products, almost everything is nowadays available
on the internet, and it's just one API call a way to integrate,
disparate software systems in order to make them work
together, to perform a series of tasks or activities.
Think about your mobile apps software as a Service electronic
payments or even smart, home devices that you use frequently.
They are all connected seamlessly through, apis, that
power the integration. And the importance of the apis
is not all about choosing the underlying Technologies or
protocols such as whether we should choose rest API, graphql,
grp C and so on, there is something more beyond that.
My guest for today's episode is James higginbottom.
James is the author of principles of web API.
Design, and an executive API consultant in this episode James
explained, why it is extremely important to design, apis,
properly and share the five key important principles of API
design such as not to design it. In isolation importance, of API,
documentation, Etc. James also recommended the API
design first approach, which is a rapid and lightweight outcome
based API design, process to design and deliver apis success.
Really, including the addr process, aligned Define design,
refine establishing API boundaries, and how it relates
to domain-driven design or DDD towards the end.
James also shared some recommendation for API testing
strategies and also some API testing anti patterns that we
all should avoid. I really enjoyed my conversation
with James understanding the importance of a good design
principles for building successful apis including his
First approach and the addr process.
And if you also find this episode useful, please share it
to someone, you know, who would also benefit from this.
And you can also leave a rating or review on your podcast app
and share comments on the social media on what you enjoy from
this episode. I always love reading your
reviews and comments and they are one of the best ways to help
me spread this podcast to more people.
And it is my hope that they can also benefit from this podcast.
So let's get our episode started.
It right, after our sponsor message.
Are you looking for a new cool swag?
Tackle, it Journal. Now offers you some swags that
you can purchase online. These wax are printed on demand
based on your preference and will be delivered safely to you
all over the world where shipping is available.
Check out all the cool tracks available by visiting package,
you know, dot, f / shop and don't forget to break yourself.
Once you receive any of those tracks.
Hello everyone. Welcome.
Back to another episode of the technology on our podcast today.
I have with me, a guest named James higginbottom.
He's actually an expert in API design.
He has been a software developer and architect for more than 25
years. So that's really a long time and
mostly helping people to develop deploy apis apps.
Also helping companies even to go through Transformations based
on these API. So James just also published a
book titled principles of web API design.
It's something that you can look for.
You need to build a web API for your applications, for your
team's or for your company. All this will be covered in this
episode. So thank you so much for coming
to the show James looking forward for this composition.
Yeah. Thanks Henry.
I appreciate you. Having me and look forward to
our discussion. So James, maybe in the
beginning, you can introduce yourself for those who haven't
known you yet. And then maybe you can highlight
starting points or major highlights in your career.
Yeah. So for those that are not
familiar, I've been Consulting for a number of years, ranging
from startups and software as a service company.
To Enterprise it. A lot of what I do is helping
organizations to establish grow and mature their API, programs,
as part of a larger digital transformation initiative.
Part of that is coming in and working and I 101 or team basis
to help organizations to grow their expertise.
And that also, oftentimes includes training.
So to doing live in person or remote workshops on API design
techniques interesting enough. I've got started in software
quite a few years ago. You're talking about turning
points in the career. My grandfather, gifted me.
A Commodore 64 computer when I was about eight years old.
He said, I want to get you this because I think computers are
going to be big someday, and I'd like for my grandson to learn
how to use them. I took to it and turned it.
Into a career is been an absolute amazing opportunity to
go from kind of the early client-server days through
client networking. We were figuring out how to
network systems across Either a private Network or over the
internet and then eventually into web development and that's
also led into web API design and helping organizations.
Think about how to treat their web apis, not just as a
technical asset. But really, as a product that's
owned and matures over time. Thanks for sharing your story.
Interesting enough. This is not the first time, I
guess actually shared about Commodore.
So it seems like that change a lot of people's career.
I personally haven't seen it. So, looking forward one day
maybe to see the real product. Of comodo, right.
Let's talk about your book itself.
Principles of web API design. What do you think are some of
the important things that led to to writing this book?
A lot of this book comes from practical application that I've
spent with organizations and myself.
Personally, designing, apis. I come from kind of a blend.
As I mentioned, in my career of through the older School, way of
doing things where we used to build or design and build
everything up front, which we don't want to do.
We learned other techniques for it.
But I've also seen a lot of opportunity to think about API
design and new and interesting ways.
I've always referenced an old partner in co-author of mine.
Keep Casey who helped me put together a self published
version of what turned into this book.
When we put it together years ago.
He always said that API design represents the effort required
to design the third user interface or referred to an API
design as the third user interface.
It's the UI for developers. What I've really taken from that
is API design. Enters on effective
communication and it's not just communication between developers
but it's communication that combines product thinking,
Business, and Technology all-in-one.
And so, it's always been a real interest to me.
So getting an opportunity to put a full book together after
having years in the field working with teams, coaching
them API design and different techniques that you can use to
get there effectively and quickly without bogging down.
The delivery process was always been a passion of mine.
So that's been Focus of this book.
So first of all, why is it important to actually design?
An AP are. So you mentioned that your
co-author of the book told you about.
It is actually one third of the user interface.
Why is it so important to design a pi properly?
I think it really does go to communication.
We don't often times think of our API design, as part of the
communication effort. We think of our documentation,
the web portals developer, portals marketing materials and
other things. But our design itself
communicates. The words that we choose the
patterns that we apply, they all communicate to us.
And I think as developers we all encounter or have encountered
web apis, or even just helper libraries in our favorite
programming languages. There were either a pleasure to
use or really difficult to use in really frustrated us.
So I think taking the time to consider your API design and
sintering it on an outside in thinking process thinking about
the outcomes that are desired. And we're trying to deliver with
that API, really contributes to that effective communication,
and also enables great documentation as a result
because it makes it easier to document the API in the
communicate, how to do things. You don't spend as much time
saying, here are the little nuances or caveat sore things
that are kind of non-standard that I put in here.
Instead, the API design becomes very intuitive for the person,
trying to use it to build a solution to solve a problem to
automate something. So I think there's a lot of
important steps. Be placed in effort, centered
around API design first, and IMS also assume when you say about
all these communications, it didn't just happen for a lot of
people who build apis, maybe also based on your consulting or
expertise in people, doing this stuff.
So, why do you think those people do not actually empathize
on getting the API or communicated with so many people
who are connecting to it? I think four years API design
was really pushed to the technology teams over the years.
we've had to make decisions about Frameworks and helper,
libraries even entire programming languages platforms,
Cloud infrastructure, anything along the way, a lot of those
decisions tend to center around technologists, but as our
organizations start to think about how we expose web, apis to
allow others to integrate with us, whether it's Corp to Corp,
just business-to-business kind of Integrations, or if it's more
of allowing consumers to access the room data or to automate,
Oceans, whatever the case is. We really have to spend more
time. Emphasizing the API design.
So I think up to this point. It's just been kind of pushed
off to the side. And now, we're really seeing not
only Tech leadership software, architect senior leaders, any
developer on a team, now, responsible for producing
delivering an API, either for internal use, or maybe external
use. But we're also seeing product
owners, get involved as well. So now, what we have is this
opportunity, Think about apis as products and that is requiring
us to take a step back and to rethink the approaches.
We have to the way we approach apis from the design
perspective. From the coding perspective,
slow down and meet those parties that have input subject matter
expertise to bring them along with us in the API design.
And not just to forge ahead with the code.
But rather to take the opportunity to turn it into
something that centers around more product oriented
experience. So, you mentioned that product
owners, sometimes get to be involved.
But is this only for the case where you are building an API,
as a product or is it also applicable for people who build
like, back-end apis, where you just connect from your mobile
application as well. Maybe your web front-ends.
So any thought about that, we're starting to see more and more
product owners get involved. Whether this is an API for a
software-as-a-service and apis, are product offering that an
organization is putting together and releasing.
Or if it's just internally facing, it's still taking a bit
of time because I think from the product owner perspective, their
response beliefs and what they're held accountable for
tends to still be related to either a partner to partner
integration or a web or mobile app.
So we tend to still focus on the wire frames in the features and
the function of the application, but it's starting to become more
and more apparent that through these digital transformation
efforts. Anything we do, whether it's
meant to have a human Ur face in some way Voice Web mobile
whatever it is, or whether it's automation that there's an
opportunity to consider the API as part of the product offering.
And so that's resulting in a lot of product owners starting to go
beyond their initial focus of just the product as its seeing
inexperienced and more of the holistic view from front to back
and that includes the apis that help to power those Solutions
swell, so let's go to some principles that you at clean.
In the book, The First principle is that you set API should never
be designed in isolation. What do you mean by that?
That's really what I've been talking about here.
And that's why I've spent a good deal of our interview so far,
discussing it because we have oftentimes required.
Our technical teams are delivery teams to shape the apis
themselves. While there are quite a few
developers out there that have designed well-thought-out apis
before I think there's an opportunity.
Opportunity to expand our influence and to incorporate
other parts of our business in the process.
So in this case, we're going to have, as I said, subject matter
experts from product owners to perhaps even security teams or
the office of the sea. So that is looking at the
security aspects of an application and what kind of
security concerns the are with an API, you know, personally,
identifiable information and non-public information other
bits. Of information.
We may not work with leaked. There are different subject
matter, experts that should be consulted.
When we think about the apis too, often.
We've released web apis out and either not secure them or didn't
think about the design from those different perspectives
from security from product ownership, form, the user
experience, and developer experience.
So the first and foundational principle is, we should never be
designing apis in isolation. We should be looking for those
other team members to come with us.
We also You have to kind of slow down as developers and if
necessary educate team members that may not be as familiar with
some of the technical details of HTTP.
And other things, just kind of help meet them where they're at,
and part rep with them and move forward with the API design,
with them involved, rather than just going off in a corner and
building something. Because there's a lot of
opportunity to make apis. Better to make them more secure
and to make them longer lasting. Yeah, which I think also led to
the second principle, which is you mention about the design
should start. An outcome-based of Focus,
right? So when you gather all these
people around, I'm sure there's a common outcome that you want
to get out from this API. Is that correct?
There is when I mean by the outcome based focus is not just
what these team members are including or wanting to factor
into the API design, but there's also the aspect of who's going
to be using the API and what they're trying to do.
So, part of what I do, is coach on a technique called jobs to be
done, which started with Clayton.
Ensign who wrote Crossing Chasm and several other business
books, his perspective was whether you're building a
start-up or whether you have a large Global organization,
anything that you do service or product is solving a got to be
done and the best way to be successful at it in a find
something that is going to do well is to understand the
customer, listen to the voice of the customer, understand their
problems and what their desired outcomes are if we take those
things into consideration. In the piece in the middle, is
the job to be done. If we understand what the
situation is and what they really want the situation to be
the in state, then what we put in the middle is that job to be
done. So when we think about our API
design, we want to think about the same thing.
There's a tool called job stories richer and similar to
user stories, but they start with the when which is that
situation, the I want to, which is the job to be done and the,
so I can the outcome that they desire, when I talk to teens of
The API designer when I'm designing my own apis.
That's where I like to start. That means that all those team
members that I mentioned from principle.
Number one, that are subject matter, experts and have input
to provide or going to be contributing to that
understanding. And I might require our product
owners to take a step back and start talking to the target
audience. Get feedback.
Just like we would any other product and use that to help
Drive literary kids. I needs to deliver and what that
will kind of outcomes we can produce as result of using that
API either as a standalone or until a Raishin, with others is
very interesting. You use this concept that is
normally used for building products.
So like when we want to gather user stories or requirements, we
use this jobs to be done framework, but you mentioned
that you actually use this to actually come up with the API
design, what you call Job story, you mentioned, but when I want
to, so I can. So it's really interesting that
you actually bring all these the best practice from product
development of product, specification, to actually web
API specification as well. The next one in your principal
is a Actually, you mentioned that you need to select API
design elements that matches the knee.
So I'm not very sure about this principle.
Maybe you can help to explain about it.
Yeah, I think this really stems from the idea that there's been
a lot of discussion over the years.
Do I use a rest API style. Do I use graphql?
What about grp? See?
I've shared discussions in the past and spoken at conferences,
where I kind of dispel the myth that there's any one particular
API style that's best. So when we think about the
design elements of API, we need to think about how people are
going to be using the API. What's the context in which
they're going to be using it? Is it primarily from a web?
Browsing device? Phone tablet, laptop.
Whatnot. Is it a voice-based?
Is it that we're going to be integrating with other systems?
And so we need some events. So we might think of things like
web hooks or server-sent events or websockets, maybe Kafka
streams. If you're sharing messages
within an Organization across teams across systems, all those
things need to come in together to match the needs of the
consumer. So as we're taking a product
based approach, part of what we need to be thinking about is not
only what the API design should look like, but how or what a pi
style, or Styles, more than one style.
Should we apply to make it the best match for our consumers?
So just because an API producer a team that's designing and
building. AP, I fell in love with graphql.
Love it. If Their audience that's going
to be using API is not familiar with it.
They have two choices, one. They proceed with graphql and
they invest in educating, the audience on how to transition to
graphql for those. That may be more familiar with
rest or other styles or to set aside their desire as a provider
as a single team and think about holistically.
All of the different things are going to be consuming this API
and determine if a different style will be better match.
Or maybe it's offering Choice offering both rest and graphql
and maybe even some event capabilities with web hooks for
callbacks when event notification and reactive style
of interaction makes more sense. So, it's just the idea that we
want to recognize that. There's no one single solution
that there's no one thing that is best.
But if we factor in our target audience, then we'll be able to
determine what the best fit is at least for today, and we can
continue to listen. To them and then offer new
Styles as the needs arise that keeps us from being a little too
focused on ourselves and a little bit more focused on the
variety of developers around the world.
That may be one in consumer API and would prefer to do it in the
way that they would, like, rather than the way that you
might want to mandate. I know that you mentioned as no
best API styles of API Technologies, but for those
techies, who listen to this, I'm sure they will have the same
question. Is it rest?
Is it job? You see?
How is it graphql, right? The new Oh, darling in the API
world. But from your view, maybe you
can give us some summary. When do you think we should
choose each of those popular Technologies?
I found that rest or just basing a web API on HTTP.
Typically using the rest architectural style is a great
approach because the tooling exists in so many languages.
It's so easy. I can use a variety of developer
tools to assist me in the development process in the
design processes in. Assuming the API and trying
things out. I just see that as a popular
option for a lot of organizations that said I'm
seeing some organization started to go down and explore graphql,
a little bit. They're finding that it's a
really great fit for the response shaping aspect of it,
even though there's other aspects that tends to be where
people gravitate toward the idea that I can craft a query, that
looks a lot like a database query.
I can be very specific about the elements that I want.
So if you have a very deep resource structure, Where you
have lots of nested resources and you don't want to have to do
query upon query, to go, deeper and deeper, and you don't want
to implement your own version of that on top of rest.
Oftentimes the teams will choose to use rest as a foundation
complement it with some graphql. The maybe, if they need some
high-performance service to service communication, they'll
decide to go grp, see, speeds up the development process with the
code generation that's built in. And also, some of the
performance improvements, the grp see takes advantages of from
http. To some of the compression of
headers and other things that are built into it and the
bi-directional communication, it gets teams a lot further.
So again, it's not one size fits all but I do see the majority of
apis that I work with tend to start with rest and then they
look at other styles to complement it and then we see
things like web sockets and webhooks used for venting quite
frequently. And I think you mentioned, also
when you explain about it, you can mix and match.
So not only choose just one style, but you can actually
start with, providing rest and maybe some job PC on top of Of
it, maybe graphql for those people who really wants to shape
the response of the web API. So thanks for sharing that
summary. So let's move on to the next
principle, which is I think a very very interesting because
you mentioned also in the beginning that API documentation
is probably the most important user interface for developers.
I've seen in my whole career. It's a documentation seems to
vary, some are really good. Some even has the capability for
you to test it on the documentation itself, but some
are just like, okay, what? The end points where the
parameters and all that. So maybe you can explain a
little bit. Why do you think the
documentation is the most important UI?
You mentioned it as the UI for developers.
The reason being is there's different paths that we all take
to work with an API. So as a developer that's looking
at a brand new API. The reference documentation is
very important. I think all of us know whether
it's a helper library for our favorite programming language,
or whether it's a web API is allowing us to take advantage of
a software as a service solution.
Or It is the reference documentation, is where we spend
the bulk of our time. However, there's additional
documentation on top, that's really important getting started
guides that help frame up and position.
The API is to what it does, what it's capable of doing cookbooks
or, or step-by-step guides that show you how to do common, tasks
are really important as well. So we sometimes forget,
particularly when we're designing our own API that, just
because we've spent a lot of time thinking about it design.
It implementing it deploying. It troubleshooting getting it
ready and push it into production and we've Managed IT
production for some time and we have people successfully using,
it doesn't necessarily mean that next developer then encounters.
Your apis going to know exactly what your apis meant to do.
What you've designed it for. What it's good at.
Maybe what it doesn't focus on or that you would recommend
complementary apis to pick up where you leave off and how to
get started. Those type of aspects cannot be
be underemphasized when we think about an API.
And the last thing we want to do is developers is document.
I don't know why a lot of us have developed this habit that
done is when the code is done, and the tests are passing, and I
can get a pushed out to a cloud resource in its up and running.
But if you don't document it, then those developers that are
trying to understand how to use it or get very frustrated, just
like you probably get frustrated with a particular website,
that's poorly designed or, you know, a word processor
spreadsheet. This is not super intuitive.
A mail app that just doesn't work the way you want it to
work. I think all of us have probably
used over the last 18, 24 months, a variety of video chat
and collaborative messaging platforms from slack, two teams
to WebEx to everything else that's out there.
And we have our preferences for what we like to use in what we
don't that user experience or that frustration that we have is
oftentimes for web apis rooted in great or terrible lacking API
documentation. So taking the time to document
Even if it means asking for outside help getting a technical
writer. That's really good.
At asking you the right questions and getting the copy
so that people can understand it.
And it's approachable will make your API very popular because it
stops the frustration cycle. And it helps people get things
done. Some of the most popular web
apis. We know of the software service
apis that we know of, that's what they did.
So even if you're building an internal API, you want other
people to use your API internally, right?
Little bit of documentation. If you're not standing up a
whole portal, just take the open API spec or whatever it is.
You're using to capture your documentation, spend a little
time, put a few examples or walk people through some basic use
cases or usage examples. Help people get started.
I have some empathy for the people that are seeing your API
for the first time. Maybe they're having a bad day
and now they have to get something done.
And this API is going to help them.
Give them a good day. Not a bad day.
So spend some time documenting or get some help if you need to
get an assist. From somebody contract somebody
and get somebody else in your organization, might be better at
it to give you some tips. Whatever it is and really focus
on the documentation they can occur over time.
It doesn't have to all be done immediately.
But over time you should be improving as a start.
Simple, grow it, improve it. Don't let it be.
The last thing on your to-do list when you release a new
version of the API or make an improvement constantly look for
ways to improve it. Take feedback from
troubleshooting sessions, or support sessions with developers
trying to use your API, take what?
Struggling with back and try to improve the documentation and
just continue to do that. It'll go a long way to making a
successful web API. Yeah, so from my experience as
well, it's not just the tooling actually to build these API
documentation becoming more varied and becomes easy as well.
So like you mentioned open EPS specification, what used to be
known, Swagger, I think, and also tools like Postman, so they
evolve into building ecosystem or building tools for people,
collaborating building API or using other people's API.
Why? So I think look for tools as
well? Not just like building by
yourself. I think that's also a key here.
Absolutely. Yeah, the last principle that
you mention in the book is actually this is very
interesting. Definitely, you mentioned, apis
are forever, but it tends to stick.
So you need to plan accordingly. Yeah, I think many of us, in the
API, space is dealt with web, apis, can understand this one.
And I know that Verner vogel's from his days at AWS, really
kind of solidified. This one through various posts
that he's made over the years. Regarding the challenges and the
opportunities and the successes that AWS has had with their
apis, that as really codified, the idea that apis are forever.
I mean, if you think of the S3 API from AWS just that API
alone, imagine if you woke up tomorrow and they broke
something. They changed in API signature.
They renamed something. They drop an operation that you
needed and that you were dependent on you woke up and
things just stopped working. We just can't do that when we
think about Web apis. We can't just think about a code
base that we can refactor and change things with and as long
as it all starts to compile and work again.
We're okay. We're talking about distributed
systems many systems of which are outside the control of other
people using it. So those that are consuming your
web API are out of control of what you do is the provider and
vice versa. In that case.
We have to have a strategy for how we manage versioning how we
approach it being sure that we're not breaking people and
also making sure that we're not creating brand new versions of
apis. Every week, or every month, it's
great to enhance an API with non breaking changes.
You'll additive changes those types of things are great.
But removing things renaming things.
Those things that would break someone who's using that
operation or using that particular field in a response
or using a particular feature that this part of your web API.
And you just decide one day on tired of maintaining it, or I'm
tired of the way. I named this.
I'm going to refactor the code that refactoring ends up
changing your web API because you haven't separated the let it
get caught. Tracked from your implementation
details. And so you're unable to make it
still work the way it was designed even though you've
changed things internally. If you start to introduce
breaking changes, then you're going to introduce opportunities
for customer Charter. So, if yours a
software-as-a-service, you have a web API and people are
integrating with your web API to automate or integrate your
software as a service capabilities.
With other things that they use other software as a service has
and they're doing things to help them and automate their lives
and you make a change. They now have to go make a coat.
X. Well, they now have an
opportunity or a question. A choice ahead of them.
Do I make the code change? Or do I go find someone else?
Who treats customers that are integrating with their web API
with more respect and prevent breaking changes from coming in
and then me waking up to a bunch of problems.
So the way that we treat our apis, the way we version and the
way we manage change is really important and we have to plan
for that their strategies that outline in the book for that and
different techniques you can think about than a Ford you
Unity's to make breaking changes before you're committed to those
designs before this designs become Frozen where you can only
add new things, but you can't go back and change those things.
You've already put out there. There's ways to do it.
But keeping that in mind is absolutely important too often.
We just plunge headlong into the code.
We think we've got it all figured out.
We put it out there. It doesn't work.
Or doesn't meet the needs or expectations yet, people start
using the API in different ways. And then we want to break it to
introduce the right way of doing things because we don't like the
way it turned out. End up causing too many problems
and we want to avoid that. So I myself personally have
experienced. For example, there's an API
deprecation announcement or emails.
You have to move to the new API by this.
So personally I find sometimes it's very annoying because what
is working? Well in Michael Bay's, suddenly
you have to change because of this, I think it's very
important that you have this mindset that API that you design
and publish them to stay forever.
I don't change it too often which brings us to a good
Segway. To the next discussion, you
mentioned API design first approach for many this doesn't
come intuitive because normally they will just start from
product requirements code and showcase, especially when you
integrate with the body between services.
So you mentioned here that important Concepts that I
learned throughout my career as well.
That API design first approach tend to be the best practice
here. So maybe you can elaborate on.
Why do you think it's the most important thing?
I think, when we think about building software, we went from
sitting down and Something about a design and spending a little
time designing out our software. Now granted a few decades ago.
We spent too much time there and not enough time actually
producing or delivering. I think the pendulum swung the
other way where we said we'll code is most important thing.
Our code is our documentation. We can just kind of evolved our
design as we go and figure things out.
Yeah, we can do that in small bursts.
The small increments inside our code base.
Many of you listening may have experienced success in that way,
and I don't discount that. But you also have to recognize
as we said with all these different principles that we
can't design in isolation that we have to think outcome-based.
Do we have to think about how are consumers, want to interact
with our apis, how we're going to document them and that once
they're out there and we have the first integration with our
API. It's out there forever.
We don't want to be tasked with having to keep up with your API
changes because you keep changing your mind on how your
design should look. And I've got to keep up with it
because you're deprecating a version and forcing me to write
more code and adjust my code to make things work.
So, there needs to be a balance between what we do with upfront
design, and how we deliver the book, outlines an API design
first approached, the idea that we need to spend a little
thought in incorporate this input from other team members
and to think about an outcome-based focus of what our
API is going to do. If we spend a little time doing
that, then we don't have to experience what many of us have
experienced. And I know I have is writing
code that never makes it into production because we completely
missed it. We met too many assumptions that
were incorrect. They're weak jump too fast to
write the code and we didn't ask any clarifying questions.
So, what we delivered missed the mark and now we have a few days
to a couple of weeks to try to make something work to fix the
things we need to fix. And so we, then start saying,
well, that's going back on the technical backlog.
Well, that's going on the backlog while fix it on the next
version. I've experienced that way too
many times. It was frustrating.
So finding that balance with web apis in particular because they
are forever principle. Number five, apis are forever to
think about. How do we A lightweight.
Rapid API design first approach that will get us closer to the
right Mark, prevent us from writing or designing an API.
That's incorrectly or misaligned with what really is needed and
still get feedback quickly. So that we're not doing this
totally in isolation in our own corners and it's too costly or
too late to make those changes. So that's what the aligned
Define design refine or addr processes all about.
Yeah, so maybe you can go and explain He's a DDR process.
It's a line defined design. Refine.
The aligned is how do we ensure that?
The API that we design is alignment?
With the outcomes that are desired by the developers are
going to using the API by the end users that will be
indirectly using that API maybe through a web or mobile app or
anything else gaining alignment by starting with the job
stories. Detailing those job stories out
to the next level where we have step-by-step.
We understand the stepwise process is involved to achieve
the outcomes that we found in the job.
Stories. And then defining our API, by
looking at those finding our API boundaries and the API
operations were going to need independent of any one
particular API style. So I had mentioned earlier that,
you know, we can mix and match things, rest, and graph, qmg,
our PC. So if we align first and
foremost to make sure that all of our assumptions are dealt
with that were not writing code, that has assumptions built into
that. Are incorrect or not designing
an API, like why's it have incorrect assumptions when we
understand what the needs are with the outcomes.
Are we break those down to the next level, understand, step by
step? What we need to do.
Then the API design will evolve along with it.
So we go through the Align phase.
We going to Define where we turn that step-by-step process into
apis that have operations to fulfill those step-by-step
processes that produce those outcomes.
We found in the alliance phase and then we apply one or more
apis. Styles to design the API.
So we might apply a rest approach for the majority of our
operations, maybe offer a graphql.
All using the output of the Define phase an API profile that
defines with the API supposed to do a high-level protocol
agnostic. And in, once we designed it now,
we have this high level design that we can quickly.
Push into an open API, spec or graphql.
Sdl, schema definition language or a gr.
PC IDL, interface definition language.
We can capture it. And then socialize, we can use
those artifacts to socialize it. We can generate mock
implementations of our API using tools that can take in one of
those definitions and create like an in-memory mock
implementation of it, just so that you can kind of try it out
without it really being completely coated up yet.
Now allows us to refine the design.
I go back and rework that. So the aligned Define design
refine is the four major phases. We go through, there's different
steps outlined in the book that take you step-by-step.
Up how to go through the Align phase and make sure that
everybody is in alignment, from business product, owners to the
tech teams. And then how to define the API,
find those boundaries, find those operations, then, apply
the apis Styles. You want to create the design
and then to circle around gain feedback and do that in a rapid
fashion. So you spend a little bit more
time up front, but you save a lot of time on the back end
where it's more costly to change code and it can be very
frustrating to have to take some well written code.
Well, designed code. And make Um last minute changes
in really mess it all up because we misunderstood something the
beginning of tries to encourage communication throughout its
really applying the agile. Manifesto principles right into
this design process without it being too front-loaded or too,
heavy weight. So that's what the book outlines
and that's what I really recommend.
It allows us to have that design first approach so we can talk
about the design. Look at how we would integrate
with it. Make sure it's going to hit the
mark and we haven't missed anything.
And then we can go write our code and do what we do best
there. In the different delivery
phases. So I also find this kind of
approach very useful whenever you build both apis
independently, and also starting from scratch, for example, so
especially the Tema dispersed geographically, for example,
different time zones, but you need to collaborate through the
apis for those services. Having gone through all these
aligned defined design even providing mocks testing for
people to integrate with the AP art.
So it's speed up the development so that they can all do in
parallel at certain point, you Because it in a staging or maybe
some kind of testing phase where you can ensure that.
Okay, actually, these two Services can talk to each other.
Thanks for sharing this addr concept.
I think it's really useful. Another thing.
I pick up when you mention about this addr is you mentioned about
API boundaries. What do you mean by API
boundaries? Is it something that is related
to bounded contacts and all these DDD?
It is here. So people that were looking at
the add or processing techniques that we train on will recognize
some of them from the DDD. Base what I wanted to do.
And what I have tried to prevent is isolating those that are not
familiar with DD or that are only practicing some of the
techniques and preventing them from being able to take
advantage of this process. So it is designed to be able to
incorporate many of the concepts of tdd.
We use event storming and courage of in storming during
the Align phase to try the surface things.
If it's a domain that's a little bit newer.
For some people bringing in subject matter experts from the
domain and having collaborative sessions to explore and
understand it. It is really valuable, as well
as what you mentioned, what the API boundaries, the idea is that
I've seen a few apis over the years that may have hundreds and
hundreds of operations. Those are just so unwieldy to
understand and get started with. So unless you just really
surround all that, really large API with a lot of copious
documentation that guides the reader to where they need to be.
Unless you do that. Then it's really hard to get
started is really not approachable to have to take on
an API. That's really large.
So the Is to bring in boundaries where we can segment parts of
the API. That doesn't necessarily mean
that you have more than one API product.
It means that you're finding different apis or different
operations that are cohesive. So I go back to the fundamentals
of software development, High cohesion.
Loose coupling loose coupling says we don't want to tie into
the internals of our systems into the implementation very
much. Like encapsulation.
We want to hide the internal details from the external API.
We want to loosely well in just couple to the API itself not to
the internal implementation details.
Hi could weejun is about having related functionality grouped
together in the same code base, rather than scattered all over
the code base, where it's hopping between module two
module, what we call Spaghetti code or big ball of mud were
things. You just all over the place and
there's no Rhyme or Reason for it.
All those techniques that it exists in PD are really
beneficial whether the team is early in their DD Journey,
whether they're not on a journey at all.
Only. Just want to design.
Only great software or whether they're DDD the experts, taking
the time to evaluate where those boundaries are for your apis
will enable you to be more efficient and decompose a very
largely scoped product or surface area of H operations
into smaller, bounded areas that are more digestible and easier
to manage and easier to decompose for delivery by the
API provider. So we bring out some of those DD
Concepts in throughout the process, but if you're not
practicing DDD, no need to be concerned.
It doesn't require it. TD for you to be able to take
advantage of it. Maybe for those people who are
interested in the DDD if we can go just slightly deeper.
He's a pi resource like the term resource from rest here
correspond, very closely with the entities.
They do. What we typically do though is
we try to get teams to look at the Aggregates.
The Aggregates tend to be where the operations and commands and
queries that you send in tend to Cluster around.
They tend to focus more on transactional boundaries and the
Nation of multiple entities together.
So the resources typically represent the entities, but the
apis tend to scope to the aggregate.
So we use events storming as a great way to find that we can
lay out a particular process. Start with our events and the
start to expand and find the commands that generate those
events and then identify Aggregates and detail out the
event storming process. Often times.
Those Aggregates are really good hints of where API boundaries
are at, we can look at what we've discovered.
And then reassess. So we can always make some
changes so that we have better cohesion if we've identified
multiple Aggregates, but maybe they belong together in a web
API because we have to keep in mind a lot of web apis because
they're going over a network must be more coarse grained than
our DDD code bases that tend to be more granular and can operate
in process. So when you go to a distributed
architecture where we have web apis, we don't want to have too
much Network. Traffic won't be too chatty.
So we have to start looking at how What are we interacting over
the network, which is different architectural style, than how we
might, architect a standalone code base, and process that may
expose that API. And how we coded up internally
looking at DDD. We really want to focus first on
the Aggregates and then allow that aggregate to help Drive the
API operations. And then behind the scenes, we
continue to apply d. D d, Ed smaller and smaller
levels as we realized the API during delivery.
That's why explaining all this and clarifying.
How How does these two concepts actually help each other all
aligned with each other? So thanks for sharing that maybe
the last point about API. So you kind of like design, your
kind of like develop it, you kind of like put the
documentation but how about the testing strategies?
Because at the end of the day people just want to integrate
with your EPA. First, of course, they will need
to go through the testing strategies.
Maybe you can elaborate some of the testing strategies that you
commonly see throughout your Consulting.
Yeah. Yeah, one of the anti patterns
that I see is that teams just In off with building apis because
they're often times have a front end to them.
They'll use their UI testing sweets, whether it's selenium or
something like it to drive front end test Suites to build those
and they'll assume that their apis tested because they tested
their friend. You are the problem with that is
that often times our front-end application whether its web
interface or mobile or whatnot, they're doing front and
validation client, side validation.
So a row Aeneas request. We'll never make it to the back
end server because on our client is catchy and time.
So you're testing the client that it's validating the input
before it goes to the server, but you're not really getting a
chance to test the server. So just relying strictly on UI
based testing to exercise. Our API is just not sufficient
enough. It's a good start the
integration from front and the back end making sure
everything's working but it's not the complete story.
So the other things that I recommend is one is looking at
Contract testing. This can be automated.
There's a lot of tools out there.
Now we're starting to merge that you, if you capture your API
contractor, you had scription and say, open API spec what you
speak of swagger. As you mentioned earlier, if we
use that, there are some tools that will read that description
which is machine readable and we'll turn it into a starting
point for a suite of automated tests.
There's some commercial tools like that is, if you open
source, libraries and others that can help make sure that the
API is doing what it says. It will do that.
Your code matches. What the spec says.
That's captured an opening Prospect.
The other thing you could do is integration or acceptance.
Testing. I prefer to really just focus on
acceptance testing which takes the job stories and the activity
steps from our line, phase of the addr process and turns those
into executable code. So those that are familiar with
behavior, driven design bdd things, a cucumber Frameworks
like that are probably familiar with the wind and and so one
kind of those types of formats. You can likewise take the work
that you did upfront during the design the alive.
Unfazed and turn that into acceptance tests that can verify
that all your apis. When combined together can
produce the outcomes that you initially defined in the job
stories during the Align phase. So you're taking all the work
that you do upfront and that's going to in turn drive your
testing to make sure that your API delivers on what you
initially identified in the aligned phase is what the API
needs to do. So doing acceptance, testing
around our API in that way is really valuable.
Some teams will do it with you. Seeing tools, like you mentioned
in like Postman and others that have collections that are built
up that exercise each step and maybe script the transition from
one call to the next pulling data along the way.
So the output of one called turns into the input of the next
call, so they can make those orchestrated calls and make sure
that things end up in the final outcome state that we want, or
they might write it by hand or some combination to do that.
So those are just kind of the high level ideas I have there
but there's more that's going to detail about the book, but it's
really important to make sure. Have a solid testing strategy.
And if you do this design first approach, then you have all the
artifacts. You need to help inform you of
what kind of test you need to write.
You just need the right them. So it really makes a lot of
sense to build a pull all that together.
Thanks for sharing some and Tibetans like, you know, testing
from just the UI. So I think here also it's
applicable to apply the test pyramid principle right?
Where you don't just have all the UI based testing, but you
have lesser integration or acceptance test part of the you
iPod. Thanks for sharing that so
James. He's been really great to talk
about all these apis. I learned a lot all these
principles and the tools and also the concept that you
mentioned. But like job stories, addr.
Unfortunately, we have to end this conversation.
But before I end this conversation, normally I would
ask all my guests to share the three technical leadership
wisdom. So this is mainly for you to
actually advise other people based on your experience or your
knowledge. So what will be your three
technical leadership wisdom. Yeah.
I think the first one and it's one that I try to really
incorporate into my workshops is Be curious not only of new
technology, but of our software development history.
I think we tend to always strive to see the new and interesting
and that shiny new things. But if we go backwards a little
bit sometimes and look at what people have done in the past and
learn from those that came before us.
There are a lot of lessons, we can learn that we can then
apply, as we move forward with software development.
There's a lot of wisdom out there and sometimes we just
ignore it because we feel like, it's maybe two older or not
relevant today, but a lot of it still is relevant even though
Though, maybe the technologies have changed.
We're doing were distributed computing than we used to.
There's a lot of history out there that I think people should
be students of and so I try to incorporate that to my workshops
a little bit as I go just so that people can gain that
Insight if they haven't been exposed to it previously.
The second thing is to always evaluate new technologies and
light of what we've learned from that history, oftentimes will
see new things, come up development team puts them
together and you open source project, put something together
and they think to the first one to have ever done it in reality.
Have it. In fact, when story is that,
I've had a client that ended up re-implementing a message,
broker over HTTP. So they effectively got pretty
close to AWS has sqs, and they didn't even know it.
They didn't know those Technologies existed, you know,
it's no fault of theirs. All of them on that team had
never really had a need for message-oriented middleware or
traditional message. Brokers think rabbitmq, things
like that. They just never encountered it.
And so they ended up re-implementing the same thing
themselves, but it was a little less reliable.
So we I evaluated that and consider how we could learn from
technology has been around, will be the best fit for them.
The third thing I would say is passing what you learn to the
Next Generation. I think there's been several
studies that have indicated that a half-life of a developer in
our industry tends to be around five years.
So that means that we're not really always doing a great job
of helping the newer generation become more familiar, not only
with our history and knowing how to evaluate new technologies,
but just the wisdom that we've gained in learning how to code.
So software development can be A very solitary role at times.
We can put our headphones on and get into that zone and really
have a good time. Writing code.
That might be our passion and I enjoy that as well.
But it's important to step out and help others, and listen, and
learn from others as well. So passing on what you learned
and then learning from other people.
That's the third one. Thanks also for spending your
time. He had to pass on what you know
to all those software developers who are still learning or maybe
developing web apis. So, thanks games again, for
people who want to To follow you online or get to know more about
your work. Where can they find you?
My company website? You can reach me in find
articles that I've written. Another insights is at launch.
Any.com., That's Lau in ch a in why.com., That's my consultancy.
I also have a newsletter called API developer weekly.
So you go to API developer weekly.com.
It's a weekly hand curated newsletter, where I focus on a
lot of API related articles and non API articles whenever I find
something interesting. Doing.
So I just send it out once a week.
It'll keep you up to date on what's Happening and interesting
developments case, studies other things to get written by other
people. I share that out every week.
So make sure to put that in the show notes.
So, thanks again. James.
I wish you good luck with all the consultancy and helping
people to build a proper API. Thanks Henry.
Appreciate it. Thank you for listening to this
episode and for staying right till the end.
If you highly enjoyed, please share it with your friends and
colleagues who you think would also benefit from listening to
this episode. And if you're new to the
podcast, make sure to subscribe and leave me your valuable
review and feedback. It really, really helps me a lot
in order to grow these podcasts better.
You can also find the full show notes of this conversation on
the episode page at technology. No, the death website.
Including the full transcript interesting quotes, and links to
the resources and mentions from the conversation.
And lastly make sure to subscribe to the show's mailing
list on technology. No, the deaf to get notified for
any future episodes. Stay tuned for the next
technique Journal episode. And until then.
Goodbye.