Developer productivity is not lines of code written.
It's not number of commits. Its kind of irreducible to any
sort of low-level metric like that.
It really has to do with the ultimate problem.
You're solving and the users that you're solving it.
For. Hey, everyone.
My name is Henry Surya Barragan. And you're listening to the
tekhelet Juno, the show will be bringing you the greatest
technical leaders practitioners and thought leaders in the
industry to discuss about their Journey ideas and practices that
we all can learn and apply to build a highly performing
technical team and to make an impact in your personal work.
So let's dive into our Journal. Hey everyone, very excited to be
back here again with another new episode of the tekhelet journal
podcast. Thank you for tuning in and
spending your time with me today, listening to this
episode. If you're new, to the podcast
know that technology, you know, is available for you to
subscribe on major podcast apps such as Spotify, a podcast
Google podcast, YouTube, and many others.
Also, please check out and follow technology, you know,
social media channels on LinkedIn, Twitter and Instagram.
Everyday, you will find words of wisdom from the latest podcast
episode and I share them on those channels to give us some
inspiration and motivation for us to get better each day.
And if you'd like to make some contribution to the show and
support the creation of this podcast, please consider joining
as a patron by visiting technology.
No dot f /, Patron. I highly appreciate any kind of
support and your contribution would help me to add sustainably
producing this show every week. For today's episode.
I am happy to share. My conversation with Bianca, you
beiong is the CTO and co-founder of sauce graph, a developer
tools company that brings universal code search capability
for developers in open source and every software organization
including leading companies like uber, Dropbox, Yelp, PayPal, and
cloudflare. In this episode, beiong, shared
with me his story on why he started building Source graph,
and the kind of problems that he's trying to solve it learning
from his Has experience working at Google and experiencing
firsthand a great developer experience using Google internal
tool called code search. He shared with me, his
perspective about developers productivity and how we should
go about measuring. It, including some words of
caution for measuring productivity, using proxy
metrics, such as lines of code, number of comets, or pull
requests Etc. We then discuss the rationale
for universal code search and why he thinks there's a massive.
Need for such developer tool to increase developers productivity
and especially to cope in the current ERA of big code.
A new kind of challenge that every software team and company
has to deal with related to the ever increasing size and
complexity of the code bases that are typical organization
has to maintain towards the end being shared, how individuals
can improve their personal developer productivity, and what
the future state of developer tools would look like Duo.
So listen and check out some of the source craft cool use cases
that Beyond shared with me. Based on the feedback given by
his customers. Those use cases, are pretty cool
to me and you may understand why universal code search.
Might just be a tool that you need to increase your developers
productivity. I enjoyed my conversation with
be young and if you also enjoy listening to this episode,
consider helping the show by living it a rating review or
comment on your podcast app or social media channels, those
reviews and Comments are one of the best ways to get this
podcast to reach more listeners and hopefully they can also
benefit from the contents in this podcast.
So, let's get this episode started right after a sponsor
message. Are you looking for a new cool
swag tekhelet Journal. Now offers you some swags that
you can purchase online. These wax are printed on demand
based on your preference and will be delivered safely to you
all over the world where shipping is available.
Check out all the cool swag is available by visiting.
Eating technology. No dot, f / shop, and don't
forget to break yourself. Once you receive any of those
tracks. Hey everyone, welcome to another
new episode of the package. You know, today.
I have a special guest with me. His name is being Lou, he is the
co-founder and CTO of sauce graph.
So, if you haven't heard about Source, graph is actually a tool
for developers to do code searching.
And basically, helping the developers in order to be more
productive with their code. I'll let beiong to A mobile
Source graph later. So being welcome to the show.
Hey, Henry. Thanks for having me on.
So, beyond before we start, maybe let me hear from you your
introduction. What's your background?
What's your highlights selling points in your career for us to
hear about? Yeah, definitely.
So I've been programming since maybe like Middle High School.
I think I got into it first on, like a TI-83, graphing
calculator. There's like a dial, like, the
basic that you could code in that kind of got me into.
It is really fun. I was always kind of like a And
science nerd in my elementary middle school days and this kind
of fit right in with that. After I got to college.
I took cs101 and fell in love with it and things unfurled from
there. I guess from there.
I ended up doing this internship at Google during my college
Years. I just mentioned that because
that was where I first got exposed to this concept of code
search, which is a thing that started me down this path, that
ultimately culminated in the starting Source graph with Quinn
and building developer tools, but that was cool because it
gave me a bunch of Experience playing around with like
different developer tools that Google had built in internally,
one of which was with this Google code search thing.
After I left Google and work at other companies.
I kind of realize that code search was not the standard
thing. And so it always felt like this
piece of the toolkit that was missing.
That was the case at palantir. I joined palantir shortly after
graduating college and worked on the commercial side of the
company or building Big Data analysis, tools for large Banks,
and financial institutions. And that's where I got to work
closely with Quinn. My co-founder for the first.
Time we were on this kind of like small and startup within
the startup team, trying to tackle the data related problems
of these large Banks Quinn and I were both programmers by
background. Both had this kind of like side
hobby of trying out different tools, different editors
different command line utilities.
And we were always talking about how much of our time was spent
reading through existing code and making sense of that.
And also, this like gap between. When we first got into
programming. I think a lot of people get into
and fall in love with it with that sense of being able to
create something. Sit down type away your desk and
then couple hours later or maybe a couple days later.
However, long the coding session is you walk away with something
that you built and created. That is almost like this living
thing that does what you want contrast that to like the
experience of typical professional software developer,
which we're getting more and more acquainted with.
And they're like, man, we spend very little of our times doing
that and way too much of our times, just trying to make sense
of the existing code base. We saw also that pain reflected
over in the software engineers at our Customers that were
working with, that was the kernel or the initial
conversation that ultimately grew into Source graph.
So wanting to get back to that awesome flow states.
That every developer loves that inspired us to get into
programming. Make that more accessible to
day-to-day developers by helping them understand their existing
code. So we end up starting Source
graph from there. At this point.
Probably the majority of my professional career has been
working at source craft. So we started the company back
in mid 2013 and now it's seven going on eight years later.
Thanks for sharing your story. Definitely.
It's interesting because you bumped into this opportunity.
Also, by chance, seeing how Google does the internal tooling
for developers. You mentioned a lot of times
about developers productivity. Maybe let's start by the
definition. What do you mean by developers
productivity? Because these days, there are so
many things about productivity, right?
Everyone is into productivity by the, your to-do list, your
calendar, whatever apps that you use.
What do you mean exactly by developers productivity?
Is it any different than any kind of productivity out there?
Yeah, that's a great question. Developer productivity is a
notoriously difficult thing to Define.
So I will try to come at it from maybe a couple different angles
and see if that can help hone in on roughly what it means.
First off. I just want to say developer
productivity is not lines of code written.
It's not number of commits. Its kind of irreducible to any
sort of low-level metric like that.
Because if you think about it, at the end of the day, what we
do as developers is, we're really creating new technology
every single feature or product. That we work on.
Is this new thing. It's never existed before our
job is to build a program that solves that new problem in a new
way. It's almost like a mini R&D
project rather than a factory model where you just like
turning out Widgets or whatever because of that property.
You can't really break it down until okay.
This is just the sum of all the functions and classes and files
that you author. It really has to do with the
ultimate problem, you're solving and the users that you're
solving it for. So So that's one angle.
It's not lines of code. It's not like any sort of
low-level metric organizations that end up, treating that as a
proxy even like a first-order proxy.
I think end up introducing perverse incentives into their
development teams. That's how you get a lot of code
written, but not actually solving user problems.
The other kind of angle to look at.
This is just view it. In terms of the overall picture
in some sense. Every piece of software is worth
some amount of time saved or money because time is money in
Certain sense. So if you look at the value that
you're creating, one thing that you can look at is, how much is
this piece of software being used by users who's using it?
And how much are they, paying you or how much revenue are you
generating from this piece of software?
Because that gives you, I think a better proxy for how much
productivity you're generating because at the end of the day
software is all about solving and automating problems for
people, saving people time having the computer do
something. So that a human being doesn't
have to do this wrote in repetitive process.
And so if you want to measure productivity, Ava T at a high
level. You start with the end user
think about time, saved or money, saved for the use of your
software. Now, that's the high level
principled way of looking at it. How does that actually help you
if you're an engineering manager or a program around a
development team, inside a large organization, the connection
between your individual work or your team's work to the overall
revenue of the company or the time saved across all the users
of the product is often a hard line to draw and that's why you
get things like the Organization coming up with points like
Sprint points. If you use a scrum or a drawer,
one of those processes, one of the responsibilities of the
product organization is to roughly map user value time
saved, whatever it matters to the company into the customers
to this point system, that reduces all those complex
factors and their interrelationships between
different teams to rough Point. Allocation.
That's easy for engineering teams to reason about.
And so then you're talking about how many points are you doing in
a given quarter or so? I don't know if that's a
satisfying answer but that's roughly how productivity is
measured organizationally, but I think at an individual level
it's almost like this feeling. It's almost like this binary
thing because a developer you either feel productive, you
don't feel productive. If you've done any amount of
software engineering, you know, that feeling the difference
between being in this Flow State where your brain feels like.
It's firing on all cylinders, you know, you have all the
contextual information you need is coding away and you're like,
holy crap. I'm getting a lot done versus
the other. Which is when you're like, I
don't actually know how this piece of code that I'm
contributing to works. So all the changes I make are
like to local or I keep breaking things, or I keep having to ask
this other person, a bunch of questions or the iteration
Cycles too long for me to get feedback to get into that, kind
of Flow State. So, an individual level, I think
it often reduces to just, which mode are you in?
Are you in kind of paralysis mode?
Or are you in getting stuff, done?
Shipping code mode? I like the way that you Explain
that. At the end of the day, it has to
bring value either to your users or brings Revenue to your
company. But obviously, this is a very
tough thing to Define because how do you actually map back
what you do as a developer, you know, like churning code, maybe
debugging testing and map that back into how you bring value to
the user? And I think it's taken for
granted and many companies that they don't actually do this kind
of measurement. I know, and you say for
individual, we know it. When we feel it.
There will be a day when you feel like, oh, you're on fire,
you getting along Things done, you could just works pretty
fine. But there are days where you are
judging to maybe processes. Maybe.
I don't know like very difficult box to solve or even competing
priorities between multiple tasks that you need to do.
But in the first place because in many companies, I'm sure that
there's no such thing as a good well-defined developers
productivity metrics or measurement, but do you think
that we should start measuring and investing our time in coming
up with this developers productivity, numbers match?
Alex, or even like some kind of rough proxy.
Yeah, that's a great question. I think it is important to
measure things in to quantify them because I think numbers,
keep people honest. Obviously, they're never going
to capture everything that you care about.
But in order to have something falsifiable and checkable, it's
good to define a metric. And then track your progress
toward it. Because if you just Define
things in terms of qualitatively, it's very easy to
trick yourself and other people into thinking that.
Oh, they actually hit that where maybe you didn't actually
deliver. On what you were trying to
deliver on so metrics and quantifying things is important,
but it's really important. What metric you choose and how
you quantify that right now. I don't know if there is like a
single Universal metric of developer productivity, that it
makes sense to adopt across every software organization,
because, I think, whatever metrics that you use are in some
sense, tied to your unique organization, and in some sense,
tied to the problem that you're solving or the type of software
that You're building. So the way I think about it is
if you're a single developer working on a solo project.
That's completely self-contained.
Let's say you're an indie developer.
You're going to build a simple tool and then you going to sell
that for money your metric. The one that matters is really
Revenue. How much money are you
generating? How much our users willing to
pay for this piece of software? Because that tells you how
useful this piece of software is to them in some sense or it's at
least a lower bound on how useful this thing is to them.
And you scale that up to a hundred person or or thousand
person or At the top of your product engineering
organization. There's a person may be their
title is VP. Maybe the title is sitio.
Maybe it's director, who is responsible for delivery of the
whole system and is in some sense responsible for ensuring
that system is delivering as much value as it can to its
users. Whether its measured by Revenue
that product generates or time spent using the product.
Whatever it is. They probably have one or two
metrics that really are the kpi key performance indicator of
how. Well, They're doing their job
when they report to the CEO and they report to the board.
That's how they're evaluated, at least in most well-run suffer
companies. It's that person's job to then,
go break it down, like the people reporting them.
Okay, what do I need them to do in order to achieve this goal?
In some cases? They might be able to break it
down, such that. Okay.
This part of the product is very independent from this other part
of the product. So I'm going to have to reports
of mine. You're responsible for this one
and you're responsible for that one, and I'm going to look at
the engagement numbers of each separately and I'm going to tie
your compensation. Shinto those metrics, because
that's a really good proxy of the overall usefulness of each
part of the system. Now, that's the lucky case, if
you can take apart the things and it breaks down nicely
without interdependencies. I think more commonly is that.
There's a lot of interdependencies.
You can't really say that you can't evaluate any piece of an
end user software application in isolation, without reasoning
about its impact on other parts. And there's also a cross-cutting
concerns such as technical debt test coverage and things like
that. You have to worry about like the
overall health. Our team.
And at the end of the day, it really is that person's
responsibility to figure out how this all breaks down.
Yeah. I unfortunately don't have a
satisfying answer here because I think every organization is a
little bit for any. It starts from the top that
person has a number that they're trying to optimize that as a
proxy for the overall, end user value that they're trying to
deliver. And then part of the art and
craft of building, an organization is figure out what
the right breakdown of responsibility is and what
metrics are going to. Used to evaluate how each
constituent part of that organization is working.
So, in the beginning, you mentioned things like story
points, right? These days almost every
company's running, some kind of a jaw methodology, scrum or
T-shirt size, whatever they use. A lot of them actually starts
using that as the measurement of productivity or either like the
product engineering team or even like individuals, and I know
these are like two edged sword. Sometimes it works.
But many cases actually, it doesn't work because, you know,
you can game the system, you create a As many points as you
want to, but actually doesn't mean anything or sometimes you
just take it. Okay, this story points.
Let's assume it's done. Let's create a new card.
So it's an illusion that you're making progress.
But my question on this is that do you start seeing these as a
problem coming up with the proxy that actually doesn't work.
And if so, what will be from your experience, some basic
indicators that may be any team could start with that.
Doesn't lead them into something that mislead the actual things
that they want to measure. Yeah, so if you're using any
sort, Points-based system, I think one of the things you want
to be wary of is ensuring that you're being honest with
yourselves about Point allocation.
A lot of. It depends on the specific
people. You have on the team if you have
someone who's experienced are running scrum or agile, who
knows how to work the system that can make the difference.
Between this being a working system for your team and it
being a not working system for your team.
Another thing that a lot of organizations do is, there's a
separation of product from engineering.
So that product is responsible for allocating the points and
then engineering is responsible for executing the projects
Associated those points. And because there's that
separation, you remove one of the incentives to game the
system because in some sense, engineering is measuring
themselves by velocity, but they don't get to choose.
How many points are associated with each project.
So that's if you're using a points-based system.
It's not for everyone, full disclosure Source graph.
We used a points-based system, the past we currently don't on
any of the engineering teams. We might use one on select
engineering teams in the future, not ruling it out.
It's perfectly good system, but it's not the one that we're
currently using. I think there are a lot of other
proxy metrics that you can. Look at that.
Give you a sense of is your organization overall in a
healthy or not healthy State. I think one thing you can do is
you can send out survey just surveying your developers to see
how satisfied they are and ask them how predicted they feel
relative to their ideal level of productivity, or the levels of
productivity that they attain working at previous companies.
I think a lot of developers have a And innate sense of how
productive they feel. Because again, it's this binary
thing where you either feel like, okay and getting stuff
done, or know there's a lot of barriers and paper cuts that are
hurting me, preventing me from getting my job done.
You can also look at the Ops and deployment side.
There's a standard set of metrics now, or I should say an
emerging standard related to how quickly you deploy and ship
software. So those I think they go by the
name accelerate or sometimes Dora metrics.
I think there's four of them. It have to do with like how
frequently you deploy decode, how much time there is between
submitting, a commit into the Upstream git repository, to when
it shows up in production, things like that.
Those are good things to minimize.
If they're too high, then that's an indicator that your organs is
likely, not as productive as it could be.
In fact, probably much less productive than it could be.
So those more quantitative things are tied, more to the
operational side. In my view.
The operational side tends to be a little bit more quantitative
because Ops is all about taking. Taking the software, the source
code that you've written and just getting it into production
that process tends to be these days.
A more automated process. A lot of humans have been taken
out of the loop there with things.
Like see ICD. You have these systems are
designed to take the changes to source code and then push them
through some sort of q, a pipeline, until they're
automatically deployed on the more product in application
engineering side. It's tougher to quantify and
that's why you have these Point systems where that's an attempt
at quantification through product prioritization.
But you know, unfortunately, I don't have a great answer other
than talking to your developers and asking them.
Hey, are you hurting right now or do you feel like you're being
really productive? There isn't really a good
Universal set of metrics that you can look at on the
application engineering and product side that gives you a
good idea of how good your team is operating.
I think that side of the picture is almost like emotional
intuitive empathetic and I think that's why I put these such an
important quality and Engineering managers because
they kind of have their finger on the pulse of the development
team. They can a know what the
potential of the people is on that team and how close they are
to realizing or actualizing that potential.
I like the way that you simplify all this just by talking to your
developers. So sometimes a lot of managers,
a lot of leaders. They forget this aspect, maybe
also relates back to the empathy thing that you mentioned,
because maybe they try to find the perfect system, either like
a gel story points. Technical debt lines of code,
being tested or whatever that is without actually thinking back
on The Human Side. And the developers themselves,
they might give a lot of insights into a daily
productive. Or are they being hindered by
processes or politics teams or even technical debt, which has
been surrounded over a period of time.
So I like the way that you relates back to Developers for
leaders out there. I think it's very easy.
Just talked to your developers. Ask them.
How do they feel when they work on some code lately?
Just real quick. I just want to clarify.
You're absolutely right. You should have your finger on
the pulse of the development team and you should be talking
to your developers daily. I'm not saying that you can't
come up with any sort of quantitative.
Metrics on the engineering side, like a lot of organizations do,
let's say you identify Tech debt as a big problem and then you
come up with a way to evaluate code for Tech tip.
Maybe there's some simple parsing or linting mechanism.
That is a proxy for that or if there's not enough testing your
code base. If that's a clear need, then you
can spin up a coverage tool like coveralls are code Cove and
track that metric down because that gives you a line by line,
but I think you can't use those as top-line indicators of
overall health. They are just kpis that you
track to Specific problems. And you always keep the human in
the loop here. You're always in touch with your
development team. To ensure that as the number
goes down, people also feel better about it.
So, in the context of that technology Industries these
days, right? Obviously, there's this
Enterprise to traditional Enterprises companies who have
been around a long time. They might have a well-defined
software development process and there's also the startups which
probably arguably do not have any process in place, and they
just figure it out along the way.
So, do you think they are? To set of problems where from
the Legacy Point of View, maybe you have traditional ways of
doing software development, either the waterfall or the
software development lifecycle that companies adopt and also
the Other Extreme, which the startups who are trying to come
up with a good way of making the app developers, more productive
turning out more features for their products.
Do you think that these two extremes that you need to think
about, especially for those who work on these set of companies?
Yeah. So between startups and
Enterprises, I guess the question is, how different are
they actually do? We need?
A completely different set of rules and best practices for how
to do great software development inside large Enterprises versus
startups. My cents on.
That is it's always yes, and no but I think fundamentally, it's
the same, right? Like software developers
wherever you are, whether in a big company or a small company.
There's a couple things that you really care about that.
Make you really productive fast feedback loop.
Are you able to build tests compile the code quickly?
How long does it take between offering a change and seeing it
in production and getting the Users, all those Loops you care
about making as quick as possible.
You care about automating as much as you can.
So if there's any certainty way that you need to do in order to
ensure that your suffer meets a certain quality bar.
Ideally, you want that automated without human beings in the
loop. So that everything just works.
You can focus on the creative parts of the job.
And you want a really robust development environment where
it's easy to look up the context that you need to have in order
to be productive. One of the things that might be
undervalued at both startups and large Enterprises is the amount
of the job of being a professional software engineer
or developer that is reading and understanding code before you
can actually write the code that builds the specific feature that
you're implementing. You have to understand the code
base that you're contributing to.
You have to understand how your change fits into that.
You have to understand what other shared libraries are
packages exist in your code base or perhaps out there in the open
source world that you can use in leverage because you don't want
to reinvent the wheel. And then when you go to submit
the change, most organizations these days, practice code
review. So someone else Team has to go
and review the code and understand your change.
Understand how it impacts the rest of the code base.
Ideally, understand it, as well as the person who authored the
change, which is no small feat. In some ways.
It's more difficult, but we can get into that later.
That's a problem. That applies.
I think universally from the largest Enterprise development
organizations, all the way down to the individual developer.
I think where it gets different between the two scenarios is
just the constraints that you have to work with.
So, if you're working inside a large existing organizations,
especially one that has an established product.
Duct, the ton of users you're going to be working in an
environment with more constraints.
Those constraints are sometimes bad, but often good, they always
exist for a reason, you have quality constraints to ensure
that code that breaks functionality for end-users
doesn't make it into production. You have the constraints of
fitting your change into the overall code base following the
stylistic conventions meeting the necessary test coverage bars
passing Architectural Review from the senior engineers and
architects, who are Possible for maintaining a certain level of
consistency and quality across the code.
These are all constraints that are necessary to make the team
functional at a larger scale, all in the service of building
more complex product, that's able to do more stuff from where
users and so dealing with those constraints, especially in the
context of a larger code base. I think it becomes a bigger part
of the problem solving role of software engineering inside
larger organizations, because you're not just solving, for the
needs of the users directly, but you're solving also for these
Strains imposed by the overall organization in the service of
the overall user set, or customer set of the
organization, definitely makes sense.
I think you brought a wonderful point for developers.
Most of the time you don't work with like Greenfield project.
We don't come up with code from the scratch.
Even if we do right. We normally utilize code from
somewhere either like open source projects, somewhere like,
stack Overflow, and we just search and maybe copy paste and
adopt. I think it's a good segue as
well to talk about. Code search, which is what
you're doing with Source graph. So maybe you can share, why do
you think companies should invest in building tool?
Like so scrap or having a code search capability within a
company? And how does it differ with
normal IDE, which can search code as well.
So, maybe you can share a little bit more about code search.
Yeah. So code search is a solution to
a problem and I'll start with the problem.
The problem is this collection of challenges and difficulties
that people who are starting to call Big code.
Big code is a buzzword. It's like big data in a sense, I
guess, but it would encapsulate Siz all the things that get much
harder when you're developing code at scale inside this world
and universe with way more code than there used to be and that
applies to both to co bases inside organizations.
And now you have more and more companies that are more and more
software driven, and they have larger and larger code bases
than ever before. And a lot of these companies, by
the way, are not like what you would call tech companies.
Traditionally. There are a couple He's in other
sectors, like agriculture, Healthcare or Finance, or
whatever. The some really old established
companies that are now mainly software companies.
So that's one side a bit code is the code inside your company's
these large proprietary code bases.
And the other side is, the exploding world of Open Source.
Open source has been around for a while now is decades old, but
the past 10 years have seen this, just like sharp uptick in
the volume and the number and the diversity of Open source
libraries in packages that are available a lot of this driven
by proliferation of the web and web services.
All of which is Howard by these open source.
Libraries. There's not a software
organization today that isn't heavily reliant on open source
code, which is amazing in a way. But with all this progress also
come challenges, right? So the challenges of working
inside a large code base. It means that there's more kind
of context that you might be unaware.
That there's more to kind of read it and understand before
you. And start writing your new
feature with confidence. There's often this like Paradox
of choice or embarrassment of riches issue that comes along
with you know, you start writing something.
You don't want to reinvent the wheel and you're like, oh, yeah,
there's probably an open-source package.
That already does this thing that I'm building.
Let me go find it. And turns out there's not just
one there's five and then you have to go and figure out which
one is the right one to use. If I choose incorrectly it could
cause me a lot of pain down the road.
So this important decision, but how do I evaluate things all
these challenges? Has are related to the fact that
we're operating in this world, of unprecedented, volume of
code. And all of it might be relevant
to the task. You're doing and you want to
take advantage of that wealth of knowledge.
And at the same time you have to also satisfy the constraints
that volume of code might impose on you.
So this is going back to the proprietary side.
A bit code. If you're contributing to a
large established codebase, you need to ensure that you follow
conventions Union. Sure that the change that you
commit doesn't just work in isolation but interacts.
Well with other components you need to And essentially like how
it's going to fit in. So anyways, all these
challenges, they add up and they add up so much that at some
point, reaches, a breaking point.
It's some point it reaches the point where the constraints that
have to work with and the context that you have to load up
and your head starts to be too much to fit into your IDE.
You can't just clone all the code that you need to know about
to your local machine and one by one set up their development
environments and spend up in your IDE.
And at some point, it can get so bad that inside some of these
larger organizations, the customers that we've worked with
the Developers. Is literally say like we don't
feel like we can get stuff done. Like, we're not shipping things
anymore because the burden of the existing code base is so
great. And so that's where code search
comes into play. Because code search, is this
tool that 20 years ago, there wouldn't be a need for it inside
most organizations because code bases were smaller back then.
But these days because everyone's working inside these
large code bases, where they want to know about the code.
That's not their local machine. They want to understand what's
out there. They want to see what libraries
exist. And how to use them.
They need a Rule to search over this giant Corpus of data.
Just like, in the early days of the internet.
You didn't necessarily need a search engine because it was
smaller and you could use something like Yahoo!
And that got you everywhere, that you might want to go.
But then as the volume of data and knowledge increase in that
Corpus of data, you wanted something that allowed you to
instantly jump to what you're trying to get to and make
accessible the kind of long tail of things available in that
Global Knowledge Graph. And so with code, it's things
like, okay, how do I discover the libraries that?
Are available either internal or external.
They might help me get my current job or task at hand
done. Once I find the available
libraries. How do I use them?
How do other people use them? The best way to learn a new
libraries Often by example, so I want to see like, how this
particular function is called, but other folks, my teammates or
maybe other open source contributors out there once I
start using it. Maybe this is a bug that tap is
in production. Now, I have to go and explore
this unfamiliar. Codebase this code that I didn't
write someone else wrote. Maybe I've never met that
person. Before.
But now I need to go and debug their code and figure out.
Okay, is this error message coming from within this library?
And if it is, how can I fix it? And sometimes you're doing it.
While production is down, so that your company is literally
burning money while you're trying to fix this problem.
So all these things go back to like you're operating in this
giant codebase. You never know when you're going
to have to understand a part of that code that's unfamiliar to
you or that you've never worked on before and in order to get
there as quickly as possible with minimal.
Friction, with minimal context, switching, you need something
like Code search. That's optimized to help you
discover and understand those unfamiliar pieces of code.
So, this is my first time I could a hearing, the term be
coat. I mean, we all know about big
data, but because I think the way that you explain it, I think
it makes sense. We see a lot of more and more
code, even if you just count GitHub repositories with the
open source world. I think it just increases
exponentially, it will keep growing as more and more people
are into engineering programming and things like that.
I think the other term it could certainly make Coming back to
the use case, here of the code search.
Maybe you can share some of the case studies from some of your
customers. How does actually something like
code search help to improve their productivity.
Is there anything for a particular research that you
have done that actually owe you? But using code search really
improve something by X. Yeah.
So there's a couple different case studies and are irrelevant.
I'll start with kind of the most concrete one, which is when your
site goes down, when there's a production issue, you want to
get this. Right back up as quickly as
possible, and you want to solve the root problem as quickly as
possible. But in either case, you want
that time to be as short as possible.
Ideally seconds. If not minutes, definitely
within the same day when an issue arises, the first question
engineer gets paged and often woken up in the middle of night,
and you wake up, wipe the sleep from your eyes and log onto your
machine. The first question is, what the
heck went wrong. You're trying to root cause that
issue. So you go through the logs, you
see an error message in the logs, and now you have to go and
actually make that change in. Code or maybe you have to
identify which commits caused that change and revert that a
lot of our customers. When they start using Source
graph. It becomes our go-to tool for
doing this. You just take whatever are is
appearing. The logs.
You pop into the source graph and nine times out of 10 that
gives you exactly what you're looking for.
You click on the result. You see exactly the line of
code, that's throwing the air and then from there, you can
quickly figure out what's going on and write the fix and commit.
It get production back online. So, we've heard from customers
who have honed in on that use cases.
Has we've turned some incidents. That would have been like
hour-long, downtime incidents, two things that were solved in a
matter of minutes, which is huge.
I think, especially in this age when a lot of applications are
web services. So if there's a production error
means, your site is literally down and means you can't make
any money off the site. In your users are also yelling
at you because this thing that they probably started to depend
on rely on is currently unavailable.
So that's the very like a cute thing that we've been able to
solve for at the opposite end of the spectrum.
There is what impact? How we Had on just like the
day-to-day developer productivity that, as I said
before, is a notoriously difficult thing to measure.
So, one of the challenges that we face is, when we go sell the
companies. They're like, okay, you know,
we'd like to measure the impact that you're having on our
organization and then we say, okay, great.
How do you currently measure the productivity of your developers?
Some people that use Sprint points in which case we'll say
after adopting Source graph. You'll see an increase in
overall velocity measured in points, but if they don't use
One of those Frameworks and they don't have a good way of
measuring developer productivity.
Then one of our roles is sort of becomes our responsibility to
help them determine a metric. That is a reasonably good proxy
for the productivity of their developers.
And sometimes it literally just reduces to developer
satisfaction. So we had cases, especially over
the past year, with covid and the pandemic.
We actually had zero customers churn through the past year when
we went and asked our contacts at a lot of customers and said,
you know, it's interesting. I knew that your business was
really hard. We expected you to be a
potential turn risk. They said well, when covid first
hit we did like a survey of all the third party vendors and
tools that we use to see which ones we could possibly cut.
Because remember February, March last year.
The sky was falling and everyone was like, okay, if we have to
cut things, what can we cut when they got to Source graph?
And I like, hey, development team.
How would you feel if we cut Source graph?
The response is just overwhelming.
This is a need to have tool. If you remove this, to, I
literally won't be able to get my job done.
The Us, was that intense for that reason.
It's saved our necks last year. So that's like the opposite end
of the spectrum. And then there's this like third
angle, which touches upon one of the other metrics that I
mentioned before, which are these kind of ad hoc
organization specific metrics. If a particular organization has
a goal in mind. Let's say it's like increased
test coverage by X percent, or maybe we want to deprecate.
This old hacky API that, we've been trying to get rid of for
the past couple of years, but somehow it's impossible to Stamp
Out. We've recently built into our
app, the ability to track usages of particular functions and
things like that and also the ability to track this
numerically on like a chart basis.
So if you have one of these metrics that you can Define, in
terms of a small piece of code, that takes your code, bases
input and spits out a number for.
What is this metric? We can actually help you track
that number as it goes down. Another case study is like
particular customer has this, use case of deprecating this API
and so we actually have a burndown chart over here are all
the out. Outstanding usages of this API.
Now your goal as an organization is to make this go to 0 by the
end of the year and here's how you're doing right now.
Wow, I left all those use cases that you mentioned suddenly I
can relate and I really like the true validation of your tool
when things are in crisis. Like in the covid situation, the
true validation of developers, most of them who would like to
have social graph still in place.
I think that's a true validation of how your tool or code search
in this place is the true measurement of how Developers
Can be productive, definitely some of the companies or teams
might find it fancy for these use cases, but I think if you
asked developers, these are definitely very important, but
there is just no tool around to solve this problem and thank you
for solving this problem. Definitely, few years ago.
You wrote this blog post about current state of developers
tools and I think code search with Source.
Graph is one thing. What are the other tools that
you think Worth to try out an experiment, whether you can
bring the same impact like Source graph into a team.
Yeah, totally. I think this is a really
exciting time to be working on developer tools.
And I guess to be a software developer in general because
what we've seen in the past five years or so, but probably even
the past like two or three years.
There's just been this explosion in the number of available open
source developer tools and developer tool companies.
It used to be very much the case.
I think that really good developer tools were associated
with a single proprietary. System of think windows in the
90s they had visual studio and dotnet and c-sharp, I guess to
some extent. You could argue like the Java
ecosystems fostered by Sun and then Oracle and that was the
case for a long time. But I think in recent years
perhaps driven by the proliferation of cloud and web
services, you've just seen this explosion of tools that are not
tied to any particular vendor ecosystem.
They're just vailable for anyone to use regardless of whether you
using this particular technology stack, that's been a huge Boon
because it's freed. He's up a lot of room for
Innovation, but it's also been a bit of a challenge because now,
there's like a bunch of different choices to choose
from, it's no longer. Just choose your vertical,
choose, which stack your on, and then adopt all those standard
tools and that ecosystem. Sorry.
I have degress from the original question here, which is what
tools would I recommend that are really useful.
Yeah. So I think there are a lot of
great tools. I think that there is a lot of
great work being done and the devops or op side of things
right now, so couple of tools that we use at source graph on
the monitor. Inside we use this tool called
Centre. It's like application are
monitoring think of it as like a stack Trace Explorer but on
steroids for your production systems.
That's great. We use this tool called
honeycomb, which is for observability.
Observability. Is this new way of thinking
about how you keep an eye on the state of your production
systems. It allows us to instrument our
application and then go and explore the data set of
production events in this kind of open-ended fashion, which is
very important for debugging. Being anomalous events.
Hopefully you're at fixing issues as they pop up in
production, but that also means that every new error in
production is one that you haven't seen before.
So, in order to be able to diagnose as effectively, you
kind of want this more, open-ended exploration tool,
which is what honeycomb is great for.
On the employment side. We use a tool called Palou me,
and we also use care reform from Hashi.
Corp. The infrastructure management
deployment tools have gotten a lot better since 2013.
The tools were using. I think it was like AWS cloud.
Nation and doctor didn't even exist back then.
Nowadays. You got tools like terraform and
plumy that make it much better to manage all the configuration
state of what you want deployed in kind of decorative fashion.
So easy to reason about and then the system takes care of
spinning up. The proper infrastructure.
There's obviously Doctrine kubernetes, which have made
deploying software a lot easier. I think especially multi-service
applications, which source graph is a multi-service application.
It's also been critical for making your software available.
Oil in a self-hosted way the conventional wisdom these days
is the world is moving to Cloud. But from our experience many, if
not, most large Enterprises still prefer a solution that
they can deploy into their own environment, especially a
product that indexes their code, which is very sensitive data.
And so without Doctrine, kubernetes it be much harder to
offer a self-hosted product because you wouldn't have as
much control over the application deployment
environment and a lot of this is going to be organization
specific, right? So what works for us might not
Certainly be the right thing for you.
But those are the tools that we use, right?
And I couldn't also relate that to another article that you
wrote which is an ex-googler this guy to develop a tool
during your internship. You exposed to a lot of internal
tools that maybe Google build, or maybe they used from it open
source or wherever they find it. Yeah.
Why do you think it's important for you to write it?
And what's your mission in? Educating people to know about
what tools exist in Google and how it helps back to the world.
Yeah, so I have my own experience as an ex-googler, but
to be fully up front. I did an internship at Google.
So I was there for all of three months.
It's been a while since I've been at Google and I had that
experience. But what we started to realize
as usage or Source graph group was that we were Landing in
those customers because an ex-googler would bring us in.
Because the pattern that we saw over and over again was someone
would leave Google, they would miss Google code search which is
Google's, internal code search utility index does all Google's
code. Accessible to every developer.
They would look around and they would find source craft.
And they would say, this is awesome.
I'm going to introduce this to my new company.
Google as a development organization, is one of the most
advanced and sophisticated in the world.
They really pioneered a lot of tools and technologies that have
since, either made it into open source, or the inspired, similar
open-source counterparts, but because all these kind of
Technologies are things were first built at Google and
there's like, Google specific tools inside, Google.
Google has become something of this like evolutionary the
analogy, I like to use is it's kind of like Australia, you go
to Australia, you look at the animals there.
They look similar to animals elsewhere in the world, but
they're all different. They got Kangaroos and stuff.
Google is like that where they branched off from the rest of
the world, a while back and they've been on their own
evolutionary path. A lot of the tools that they use
internally are similar to ones in open source, but they're not
the same ones. And so one of the challenges
that we saw in speaking with all these X googlers who became
source, Users and customers was like Hey, you know having some
difficulty mapping from Google internal technology to what's
available outside and that's a pain point because I think the
developer experience inside Google is so good.
That one of the first things that developers do when they
leave Google is they try to recreate pieces of that
developer experience, because like I felt the absence of code
search was super painful. And so wrote that blog post,
based on conversations with a bunch of such X googlers and
tried to put together a kind of a quick, Explainer for how do
you map from Google internal tools to ones available, outside
to be a front. This was not the first post that
touched on the topic. I think there's a GitHub repo
actually, which I think we linked to in the post that just
gives a comprehensive listing of all the Google internal tools
and their external counterparts. And our goal in the post was
just to present some of that information in more of a
narrative fashion and talk about, not only what tools you
might want to look at. But like how do you bring those
tools into your organization as an Blur entering the outside
world. So for those of you who would
like to learn more about all these mapping what tools exist
in Google and how it maps with others in the world.
I put that in the show notes also having seen inside Google
itself. I can validate that actually
these tools really brings a lot of productivity and they are
well integrated end-to-end from the beginning of your software
development life cycle to the end up to your code being
deployed on even maybe the locks and monitoring observability and
all that. So sometimes the integration
part probably is one of the most Crucial piece of all this
developer tools because without that, you'll still scrambling
insiders when you want to debug something that you go to this
tool, but when you find another problem, you go to the other
tools, so I think it will be great one day in the future.
Hopefully soon that we will start seeing all these
interoperability between different developer tools, and
it'll be great experience for developers to have that as well.
So, coming back to the developers productivity, right?
So, we talked a lot about from the product engineering point of
view, company, team and all that, but what if I as an
individual as a programmer. I would like to also improve my
productivity. What should I do?
Maybe you have some perspective here.
Yeah, so on an individual basis, I think you always want to start
with the pain that you're trying to solve.
When you're coding everyday, make note of what you find
difficult, what annoys you where you're kind of like context
switching away from code. Like anything that takes you out
of that flow state, where the state of flow is this kind of
mental state, where you just feel feel like your thought
processes is extremely fluid and you just cranking out code and
he takes you out of that or anything that prevents you from
getting into. That.
That's something that you should address and you should address
it in the way that programmers address.
Every problem, which is figure out how to automate it.
Chances are if it's causing you pain.
It's because it's something wrote or repetitive or manual,
or potentially unnecessary something that is boring to your
brain, but you have to do. And so when you go to automate
it, there's obviously two ways of doing that.
You can either build something that It's it for you, or you
could find a tool that already exists, that solves the issue,
do a quick Google search and see if anything exists.
So that's like a very needs driven way of addressing pain
points. The other angle, I'd recommend
going about, it is just find developers on the internet, who
write blog posts or maybe they tweet about their work flow or
the tools that they use and use the tools that they use.
Because I think one of the ways you get good at any sort of
craft as you learn from the Masters that craft and It's not
even about learning from masters of The Craft per se because I
think software development is one of those things where
there's so many tools that chances are if you talk to
anyone who's been doing this for at least, some period of time.
They'll probably have at least one tool that they are aware of,
that would be really useful to you.
That would be really cool to try out.
In fact, we actually started doing this thing during
quarantine on our development team.
We're a bunch of Engineers on the source craft team would join
like a live stream and we have one person just screen, share
their Dev environment. Up and walk us through.
Like how do you make a code? Commit?
Show us your editor? What command line utility is to
use on a daily basis? And that was awesome.
On the first one that we did. I walked away with three new
tools that I had to try out that weekend because it was like,
wow. You can actually do that.
I had no idea. I don't think it's announced
yet, but we're going to turn that into web series because we
think that that sort of conversation is so interesting
and useful especially in the covid world or as the world
becomes more remote. I think one of the things that
we lose out on On is the ability to look over.
Our teammates, shoulders walk over to the desk and be like,
oh, what is that on your screen that sort of like social
connection. I think it's just a great way to
discover new tools and Technologies.
I totally agree with you on this.
So from my experience, with the way I figure out other tools is
when I normally did pairing with other developers.
So, during the time when we can sit together side by side and
also yeah, like what you said on Twitter, people sometimes post
random things that looks cool or maybe it's just new open source
tools that someone We just wrote it and publish it.
You think that it will help for your productivity.
So I think I totally agree and I think the permutation of many
things that we can apply in our developer workflow.
I think it's just tremendous, there will be a lot of things
that you can try to improve in terms of your productivity.
So a little bit here. I know that you mentioned.
This is the time, the current exciting times for developer
productivity tools, right? What are some of your
predictions seeing this, maybe? I don't know.
Five years forward. What will be the state of this
developer productivity? Activity tools.
Yeah, so I think right now we're in this explosion of different
tools in the kind of diversity of offerings in the parts of the
software development lifecycle that they tackle and it's
amazing. I think that will continue over
the next five years. I think this is a trend that's
already playing out which is more.
And more of these tools are either open source or opencore
including the ones that enterprises pay for.
And so there's a lot of companies that are pioneering
business models built on top of Open Source Technologies, which
I think is really cool. As developers care a lot about
the openness of the software they build on top of, and also
the software that you use. I think that kind of gets at the
importance of being able to introspect into your tools.
If something goes wrong, you can poke in and see what went wrong.
This is also, this almost like philosophical feeling that.
Hey, if I'm building a substantial portion of my
workflow on top of this tool, then I deserve a certain level
of freedom in terms of ability to continue using that tool to
some extent. I think people are still burned
by Proprietary lock in which was a common strategy.
I think for software vendors in the 1990s, but increasingly the
consensus is that open is the way to go.
And so I think you'll see more and more companies that are
building tools where the tool that's relevant for the
individual developer. That part is going to be open
source, and then there's like an Enterprise component that is
more important for teams or organizations.
That might be kept proprietary. In order to be able to build a
sustainable business on top of the technology.
So that's one Trend that I think will continue.
I do think that with the explosion in diversity of tools
that we're currently looking at there, will inevitably be the
pendulum swing backwards, a little bit into more
consolidation, especially around particular, ecosystems and
targeting specific aspects of the software development life
cycle. You'll see essentially like
winners and losers to certain extent, and the winners will
probably buy up some of the losers and they'll be some
amount of consolidation and standardization, but I don't
think it will ever go all the way back because I think we're
really in this. World.
Right now. I think in the old world, the
model was you had these proprietary vendors that built
up. These vertically integrated
ecosystems that they control, they controlled the marketplace.
They controlled what Partners were able to get in front of
customers and they control the technology stack.
Microsoft built out the tech stack for Windows and made it a
really lucrative environment thing.
And I don't think that models necessarily wrong it obviously
yield a lot of value over the years, but I think now we've
gotten to the point where there is no single entity that can
Capture all the creativity and Innovation that can happen with
software and you see that with a web.
Why is the web? The platform that appears to
have won out over all the existing, kind of incumbent
desktop operating systems? And I think that some of it is
tied to the openness of the web. The fact that it's not under the
control of any single entity and that permits this kind of
flourishing and diversity of ideas that can spring up and
find users and use cases. That would have been
Unthinkable, even a couple years before.
And so I think that's going to very much continue.
You're going to see the continual flourishing of this,
like third-party independent developer tools ecosystem,
that's going to resist complete consolidation into that kind of
world. What's going to become more
important are these kind of like meta tools or aggregators that
unify multiple tools into a single coherent experience and
at some point I think as software continues to eat the
world as software development becomes ubiquitous at some
point. The devtools market will just
become like the Petit tools Market because code will become
so synonymous with knowledge work that the vast majority of
people in the economy could be building software in some shape
or form and that's going to be like in this huge vibrant
diverse ecosystem. The last thing I'll say there is
that in that explosion of different offerings, anytime.
You have a large data set or Market that explodes like that.
I think search is going to be an incredibly piece of
functionality because it's just going to be like so many things
to consider in a good search engine.
I think, we'll, I started to make sense of it all.
And I hope Source. Graphing, continue to play a
good role. They're definitely very
exciting. You're right.
I mean, software is eating the world.
So more and more people get into programming writing code, more
and more open source, projects are being published, and it's
going to be tremendously growing over the time.
And I'm sure developer tools will be a plenty as well for us
to try out. And I think, don't forget as
well, the code these days is not just sometimes, impacts, right?
There are a lot of audio, something like this, or even
like videos on Tube. I think those probably are some
of the opportunities that haven't been tapped by people
who are teaching tutorials on the videos.
Maybe sometimes they just want to search, which part of the
video that actually teach this. Maybe it's going to be cool.
So thanks so much for sharing all these stories.
I think due to time, we have to cut it out.
But before I let you leave, normally, I would ask this one
question for all my guests to share with my audience here,
which is the three technical leadership wisdom.
So being do you have any kind of wisdom that you want to share
with all of us here? Yeah.
The one piece of advice that I got early in my career, as a
tech lead or an engineering manager that really stuck with
me, is when it comes to motivating the people on your
team and unlocking their potential.
You always want to start with the why aspect of their jobs.
So, rather than telling people do this, do that.
You always want to give them a kind of high level goal that
they can creatively, use their creativity to find a solution
to, for a junior engineer. That high level goal might be
something that is Ultimately low-level go and make this
button work really well for users, but the should always be
that component of here is what the end goal is.
And I do not care how you do it. That's up to you to figure out.
That's up to you, to exercise, your human brain and creative
inspiration to figure out how best to solve that problem.
Because I think that Creative Drive is what motivates every
developer. It's like why we got into
programming, right? That Act of Creation, that
active creative problem solving, I think, oftentimes especially
as the organization Rose and you're trying to solve these
bigger problems and you're solving them by breaking them
down into smaller bits can be very easy to just become more
prescriptive or more imperative when it comes to telling people
what they need to do, but what I found works for both myself and
the people I work with is always keeping that goal in a way that
the person actually, satisfying that goal is able to work
creatively. That's good advice.
I would say for both, like tech leads when you're talking to
others and explaining a problem that you want them to solve.
And for the person who's receiving that If your manager
is explaining this problem to you, ask the questions about
what is the end desired State here rather than just taking
down the orders. Because I think that will
ultimately help you do your job better and lead to a happier
manager because you've done a better job, he found a more
creative solution to the problem that you were asked to solve.
I think another piece of advice is, if you find yourself doing
anything twice that you don't like, doing find a way to
automate it or look for a tool that automates.
It that is your kind of job as a programmer is to Make things.
And if you're not automating your own life, then you're not
living your own values. And then, lastly, just remember
to have fun. It's amazing that we get to work
in a job like this because coding is so fun.
You get to create these like applications that are used by
dozens or hundreds or thousands or maybe sometimes you millions
of users and you get to do all that just by sitting at your
desk and typing things into a computer.
I think that there's a lot of complexity to deal with
professional software. Engineering is there's a lot of
people issues. Shoes, there's a lot of
understanding existing context. There's a lot of slog to the
job. But I think at the end of the
day, the best way to make yourself, do a good job and also
not get overwhelmed by the complexity of deal with.
Is to remember what got you into program in the first place?
Focus on that, and try to experience that as many times
per day as possible because I think, if you do that, it will
drive you to focus on more of the creative aspects of the job.
It will also motivate you to automate the more mundane or
wrote aspects of the job. And it'll just make you a
happier person, because it'll feel like you're doing something
that's truly exercises. The human creative aspects of
your brain. Thanks for sharing the wisdom.
I like the last one. Definitely, we all got into
programming because of something something that's, you know,
spark when we try it, writing some code and something
magically happen. I think it's also synonymous
with the productivity concept of a developer, right?
When you are productive. Basically, you can generate
code, that actually works, may be impacted many part of
people's lives, but I think we have to be conscious over the
time. Let's say we don't enjoy what we
are doing the programming things.
Maybe the productivity goes down and we are not having fun.
So you always have to look back and think what got you into the
programming in the first place. So thanks again for your time.
Being for people who wants to find you online or learn more
about social graph. Do you have a place where they
can go to? Yeah, first graph just go to
short script that calm you should be able to search over
all the open source code from that URL or you can download and
run your own source graph instance.
And for me. Probably the best place to reach
me is just on Twitter. I'm at biamby ey A and G.
Feel free to reach out or DM me or tweet at me or whatever you
like cool. So, thanks again Beyond.
I hope one day I would be able to use your tools in my
development workflow, and it will be great.
I think to have all these superpowers for developer.
Awesome. Thank you, Henry, and thank you
so much for having me. This is great.
Thank you for listening to this episode and for staying right
till the end. If you highly enjoyed, please
share it with your friends and colleagues who you think would
also benefit from listening to this episode.
And if you're new to the podcast, make sure to subscribe
and leave me your valuable review and feedback.
It really, really helps me a lot in order to grow these podcasts
better. You can also find the full show
notes of this conversation on the episode page at technology.
No, the death website. Including the full transcript
interesting quotes, and links to the resources and mentions from
the conversation. And lastly make sure to
subscribe to the show's mailing list on technology node are deaf
to get notified for any future episodes.
Stay tuned for the next technique Journal episode.
And until then. Goodbye.