Hello everybody. I'm Shinobi, the technical
editor at Bitcoin Magazine, joined here by Stefan, also
known as Stickies, a Bitcoin Core contributor.
So we're going to talk a little bit about some of the work he's
been doing helping sedated with the Colonel project and then
kind of go into some of his review work before that.
So I guess kind of start like what?
What was the state of like the Colonel Project when you joined?
So the kernel project largely had two different phases.
In the first phase, it was all about identifying which code
actually belonged to validation, and then moving all of that into
separate directories, or at least a separate library.
And then the second phase was about building a public
interface around all that validation logic.
And so when I started looking at kernel in more detail, it was
somewhere halfway in that first stage of the code organization.
So the most work that I helped with since has been, yeah, just
moving everything into a more logical place, which also
involved a bunch of cleanups. Because you know, when you move
code around, it's also a good time to clean things up a bit
and remove some foot guns or some bugs or just make things a
bit nicer, more modernized as well.
And so like what, what was your kind of take on like the
progression of the project, I guess kind of from where it
started before Saturday took over and kind of the general
work that was involved in getting from the
conceptualization to like where things are now?
I think the pace for these projects kind of goes up and
down a bit based on where you are in the project and who's
contributing. I think in the first stage from
the part where I joined, progress was kind of moderate,
like it kept going at a steady but not super fast pace.
Also, because these changes were very critical, like if you move
consensus code around, even if it's just moving it around, that
can still introduce bugs potentially.
So you very much don't want to rush that process.
So yeah, not an excitingly fast pace, but I think quite
appropriate given the circumstances.
And we had a, a group of a few contributors who would regularly
come back to those PRS and, and keep making progress.
So it, it felt like it was, it didn't feel like it was stalling
for the most part. And then I feel when things
really picked up is about, yeah, six months ago when we already
started working on the public interface of the Bitcoin kernel
library. So thinking about how we expect
and want people to use the validation logic, that's when
things, yeah, really pick the pace.
I think for multiple reasons, but the main one probably it
becomes more tangible. Like if you just move code
around, except for people working on a project, it's not
that exciting because it doesn't add any functionality, it
doesn't enable anything new. It's important because it helps.
It makes it more robust, it helps us find bugs, it helps us
prevent introducing no bugs. When you make changes to what
should be non validation logic, it's just not that exciting.
It's important first step, but then once we started building
this public interface, we also asked people to try it out and
use it for small use cases. Like we had some people use it
for data science projects and then reports on findings on
delving, or it was used in an early prototype of Swift Sync.
And people could see that this was something that was working
even if in a limited fashion and still very experimental.
But it became tangible. And I think that gave us both
the user feedback that was very helpful, but also checked it
more developer interest. So we've had a couple of new
contributors joining who weren't contributing to Core before that
are not spending like really useful time on building new
features, reviewing and bringing everyone ideas into Corel.
And that's really been speeding things up at the moment.
Like we have a weekly call talking to my Colonel and
usually it's around 8 people on that call, which is a lot more
than I think what you would have imagined a year ago.
So that's been that's been really fun to see.
And so, you know, before you started contributing to the
kernel yourself, you did like a lot of general review of kind of
the project in general. I think that gives you kind of a
good view into core as an overall project rather than just
the like subsections or sub projects.
I guess to kind of start getting into this, you think you can at
least take a, a college try at kind of breaking down like what
are the major like different areas of core as a project in
the code base? Like what's their functionality
like how do they touch other parts of the code base?
And just like how that that kind of organization works in terms
of the different areas people work on.
Yeah, sure. Trying to give it some
structure. So I think a helpful way to
organize it is kind of a foundation of it all.
We have the the tooling and the the build system, the CI that
basically allows us to work to do the work that we're doing.
So over the last year or two, for example, we've had a big
project that moved the build system from all the tools to
Cmake, which was necessary for some technical reasons, but also
just made for a smoother development experience.
CI, we've expanded lots like automated testing.
Whenever we push and merge code into the code base, that's been
expanded a lot. It's been made faster, more
robust, So everything that makes that developer experience better
and helps us catch bugs faster before they even can be merged
into the master branch. Then we've had a lot of work
happening on everything like net processing, basically all the
peer-to-peer stuff is another big area of the code, maybe not
as much in recent years, but that's a big part of it as well.
We've had a lot of policy work happening and work cluster
obviously has been the latest one that has gotten merged and
is getting close to being in AI guess final ish state.
But then also of course the policy related work that's been
a bit more in the news recently. The IPC interface is a big one
that's been making a lot of progress this year, especially
multi process has been around for I think almost 10 years now.
It's the longest running project that's still open in Bitcoin
Core. But then thanks just just.
Real quick, you kind of want to like explain for your listeners
like what the multi process project is like, why that's been
such a long ongoing project. Yeah, sure.
So the the goal of multi process initially was to allow us to run
different parts of what is currently Bitcoin Core into
separate processes. So Bitcoin Core is an
application that does a lot of things at the same time.
So I think the initial outline of that project was to have a
separate process to run the wallet, to run the node and to
run the GUI. Those were the three main
components where for example, you could run your node on your
remote server and run the GUI locally.
You could spin it up and down how you want it to.
So it enables certain new kinds of functionality and it also
offers security and safety benefits that process isolation
offer. It was a very ambitious project
and so it took a while to, you know, be developed and
especially to be reviewed as we kind of became inactive.
Well, this is before my time quite a few years ago until we
had a new use case pop up about a year ago or so because there
was there's a lot of work happening on Stratum V2, the
upgrades of the mining pool protocol to help mining pools
communicate with hashers. And an initial approach on
making Bitcoin Core compatible with Strata V2 was to basically
incorporate a lot of Strata V2 logic into Bitcoin Core.
But this of course adds to the total service of the Bitcoin
Core code base. And so there was another
approach suggested to try and keep it out of Core and
basically have separate applications talk to each other
over IPC, over this inter process communication protocol.
And so that was like an actual real use case to help move the
multi process project forward. And that's also why we released
the mining interface. I think in version 29 it can be
off of the number as the first version to help strategy 2
clients connected with CONCORD without putting the strategy 2
logic in with CONCORD to help maintain that modularity.
Hey everyone, it's Shinobi and I'm here in front of the Chicago
Fed to talk to you about the core issue of Bitcoin Magazine.
The last few years have been a bit of a communication breakdown
between developers and people using Bitcoin.
Bitcoin exists to be an alternative.
To this institution. But to do that, it needs people
to actively maintain it. If you actually want to hear
from developers themselves, how they approach their work and
what they choose to work on, go to bitcoinmagazine.com and get
yourself a copy of The Core Issue.
And I mean, it's this is all kind of, I guess a giant project
untangling the mess. I guess you could say Satoshi
left like if you if you actually, you know, even if you
don't program or really understand software engineering,
like I think if you look at like the code base today or like
other typical code bases versus what Satoshi released, like it
was just a giant BLOB of code with everything all tangled
together. Versus like traditionally
software is made modular to begin with.
Like you have this process over here, that process over here,
like cleanly talking to each other through a defined protocol
or API. And yeah, I really, I really
think like viewers should, should appreciate like how
important something like this is just from the point of view of
this is software developers have to continue maintaining like
Bitcoin, even if the consensus rules never changing on.
Like it is not a piece of software that can just never
change, never have code updated to respond to changing realities
of the world. And like, that really is a
necessary thing if we want people like yourself to continue
spending your time actually keeping this working for us.
Yeah, no, absolutely. I think it's reality that like
software just needs to be maintained because you know,
change, compilers change and just keeping up to date with
that reality is already a reasonable part of the low.
But you also see the way that Bitcoin is used to changes.
I think the the min relay fee changes that were done a few
months ago I think are a good example of that.
That is not something that was initiated or pushed by Core, but
it was something that Core had to react to because suddenly
users decided to use Bitcoin in a different way.
And so you have to react to that.
And then, yeah, there is reality that like you're given a certain
code base to work with, which was not perfect in all ways, but
yeah, that's, that's what you have to deal with.
And so I think a lot of the work that we're doing now is to help
turn that into a foundation that we can build on for not just
next year, but for decades to come.
And a big part of that is just spending the time to, for
example, modularize code more as we do with the kernel project
and to for example, with the multi process projects to just
increase that robustness. And that is also one of my
personal goals for the future is to make sure that even with
reduced developer resources, we can still maintain and make make
sure that Bitcoin Core is as reliable, robust and useful as
it is now, even if in the future it will have more, more features
and more functionality than it has now.
And I guess like, are there other big areas of the code base
or projects you think are worth mentioning?
Oh yeah. So I think I already mentioned
to the the build system and CI is like 1 components.
You have all of the P to P logic that deals with, you have all
these different Bitcoin nodes that need to talk to each other.
How do you do that in a Safeway? Because you're dealing with an
authenticated protocol. So you have to build in certain
defenses that people can't bring down your nodes.
And that's a bit of a different paradigm than most other
software that is authenticated. Then of course you have a whole
lot of validation logic that deals with validating
transactions and blocks everything consensus related.
We also have a lot of effort focusing on testing these years.
We have our unit tests and our functional tests, but then also
rather recent focus has been the first testing, which has been
getting a lot of attention in recent years.
Basically in a smart way to hammer the system with all kinds
of possible inputs and see what kind of output it creates and
see if that can crash your application.
Which helps you catch a whole range of issues that are just
very difficult to do with manual testing because we're humans and
we don't see all the options all the time.
The wallet is not a big component of Bitcoin Core.
We also have a visual user interface, the GUI that helps
people that don't want to run Bitcoin Core in their command
line to give them a bit of a an easier access into the software.
And we have some other more utility like software, for
example, some indexes that you can build with the concord for
common stats for addresses. And I think I'm probably
forgetting something, but I think top of mind, that's
roughly the main areas of work. And So what you know, when you
go through reviewing PRS in different sections of the code
base, like given the state of things now in like the past
couple years, like how often does somebody who say just wants
to make a change to the wallets or something in the peer-to-peer
logic have to kind of cross the boundaries and deal with other
parts of the code base? Like, is that like as common as
it used to be? I think that's improved a lot.
So a lot of the work that has already been done for the multi
process project was to basically build those interfaces between
different parts of the code, for example, between the node,
between the wallets, between P to P to really separate those
concerns. So if you make changes in the
wallet, you really should not have to be concerned about most
of the other codes. But there are exceptions like
the isolation is not perfect. And so when you do make those
changes, you do need someone with a bit more in depth
knowledge to verify that what looks like a safe change is a
need to save change because those side effects are sometimes
hard to anticipate. So I'd say mostly it's in a
pretty good shape isolation wise, but not perfect, which
makes it. How much does it like slow down
the the progress in APR when somebody does have to deal with
that overlap and like consider what's going on in some other
system based on what they've changed and what they're working
on? I think it's maybe one of the
biggest factors in, yeah, which PRS make progress and which
don't. Obviously, you know, people
review PRS as a function of how important they think that
changes. That's probably the biggest
factor. But then also how likely is it
to make progress? And is it going to get merged
eventually? Because if you know like no one
else is going to review it or it's just too complex to be
merged, then it's kind of hard to convince yourself to spend
time on something because it doesn't really serve an actual
purpose. It's assuming that the change is
meaningful enough to satisfy that first criteria.
If the code that's being touched is more complex, as in it
touches different systems, then people just find it much more
difficult to be confident that a change is safe.
And then it becomes much more, again, much more difficult to
justify spending your time on that because you just, you have
to spend much more time reading the code and thinking about edge
cases and maybe adding tests. And yeah, that's what I
personally see. I think those PRS usually stall
the most because it's just hard to stay enthusiastic and
motivated about working on that. Yeah.
So the multi process project quite literally is something
that could drastically improve development pace and development
speed like finishing the last I guess string of things that need
to be finished there. I think it's actually already
done. The the isolation work, as far
as I've seen, is pretty much complete, the most outstanding
work that needs to be done. There is more the code that
actually deals with the communication between those
different parts, because you're suddenly talking about different
processes that need to communicate to each other
efficiently, reliably. You need to be able to debug it
and kind of monitor those processes quite efficiently.
And that introduces new dependencies, which then again
need to be understood and reviewed because we shouldn't,
we don't dependencies in the code base without knowing that
they are secure and reliable and have a, a minimum service to
introduce potential risks for the code.
And I think that's the main hold up with multiples at the moment,
is that that code is just fairly complex.
And again, with some of these complaints, people are less
confident that it's safe. And if they're not confident
that it's safe, then they're going to be much more reluctant
to, to leave your X. And yeah, that's a, that's a
difficult thing to get out of. But having the actual demand,
you know, we, we had the use case now with the mining
interface for Stratton V2. And so the complexity is still
there. But if demand increases, then
you're more likely to make progress on these kind of
projects that are important. But it's just it takes more
resources all. Right.
And I guess, you know, when looking at like this kind of
work and the fact that a lot of developers time is kind of spent
on this, I, I, I wouldn't call RE architecture completely, but
you know, the re architecting of things and kind of polishing and
streamlining things, maintenance optimization, like a lot of
people in the community kind of look at those things being
prioritized and see that as something to criticize, like
they're better things that developers should be doing with
their time or more important things to focus on.
Like what? What?
What do you think when you hear like that type of criticism from
people? I think it's definitely, there's
a lot of truth to it. I think it's important that
developers work on projects and features that matter and that
have impact. And we shouldn't get lost in
academic exercises on, you know, building stuff that is very nice
on paper, but in practice it doesn't serve real purpose.
And I think it's important to keep an eye out for that.
But I also think the people working on Core do that because
of and in ways that they think are important.
It's because they get excited about certain projects.
And yeah, some or maybe even a lot of those projects at the
moment are focused more on maintenance or improving the
robustness of Core rather than adding new features like
protocol upgrades or covenants, which is a big one.
And I don't think 1 should be done instead of the other.
It's more a function of what are the people working on the
software care about the most based on their experience and
their view. And I think a lot of people, at
least it's my own personal preference, my priority for
working on Core is to make sure that we can we have this
foundation that we can easily and reliably keep building on
for the next decades to come. And I think having software that
is easy to maintain, easy to understand, easy to review,
increases the pace of future development.
Because if you have code that you can easily grog, that you
can safely make changes to because you have good testing
infrastructure in place that is more modern.
So it can't introduce some of the bugs that you could have
with like older C code, for example.
That's kind of investing into. You invest time now and you
expect to get a return on the time in the future because
adding features on the code base that you don't understand, it's
just incredibly dangerous to do. And you know, after all, we are
building software for the future that should never introduce any
significant bugs, especially stuff like consensus fields that
that's absolutely critical to the system.
And so, yeah, I think neglecting the importance of those kind of
changes, having good testing, having good build systems,
having good modularity, having modern and easy to understand in
code is absolutely critical for being able to do the also
important work such as protocol upgrades.
So yeah, I just invite anyone that thinks that a certain
feature or change is important to conjoin and and help us do
the work, not instead of just in addition to.
Both things I think can happen at the same time.
Yeah, I mean, it's kind of like, you know, your car breaks down.
Do you just stubbornly go deal with the, you know, complexity
of public transit or taking a cab everywhere and the extra
cost? Or do you just take the time and
fix your damn car so you can get where you need to go?
All right. Well, I really appreciate you
sitting down and talking to me about this, Stefan.
And I hope everybody who watched this has a little bit better of
an idea as to how core is organized as a project and why
developers prioritize certain things over the other.
So thank you again. Thanks for the chat.