Welcome to the IBM podcast. AIM is the chartered body for
the project profession. My name is Emma De Vita and I'm
the editor of Project APM's quarterly journal and your host.
In this podcast, I'm speaking to Darren Dalcher, Professor of
Strategic Project Management at the University of Lancaster.
Few people have started project management as closely or for as
long as Darren, who's also a director of the National Centre
for Project Management and Co editor of the 7th edition of
OPM's Body of Knowledge. In a career spanning more than
25 years, Darren has become very interested in why projects fail
when it comes to Agile. Darren was there from the start
when it was being experimented with in software development.
Who better than to explore the origins of Agile and its impact
on projects? Listen on.
Welcome Darren, Thank you for your time.
Could you tell us how you originally got interested in
Agile, what its origins were? I was involved in software
development originally as an engineer developer, and then I
moved away from that and at some point I developed an interest in
failures. As a practitioner, I'd worked on
a number of on IT projects. They're all kinds of challenges
around them, and personally I was trying to make sense of some
of the challenges, some of what I was seeing around me.
The whole notion of success and failure were a little bit
dubious, as far as I could see. We had fixed deadline, fixed
ideas about what what we needed to be produced, and sometimes we
delivered on time, sometimes we didn't, Probably around
19818283. I was working on a number of
projects. We were using this new idea
called prototyping when we were spending a lot of time with the
users. We were trying to understand
what they were after and essentially a lot of people
didn't have an idea what computer technology could
deliver. So working with the users was an
opportunity to introduce to them different ways, different
options. Very often we didn't know that
much about the users world. We would have a client coming in
getting us to do a system. We came from the technical side,
so we had an idea about how to develop things, but not
necessarily the user's world. And essentially it was a way of
interacting with people. So for me, prototyping was a way
of interacting. Prototyping made the project
longer because once you started interacting with the users, once
you spend time with them, it was all about the dialogue, all
about the understanding. This is where you'd realise that
the requirements that you were given or the the need statement
that was passed along wasn't exactly what the users needed.
But we've kind of gone the complete circle now and we
recognise that we need to get the users on board.
There's no point delivering if nobody uses the system.
We deliver systems, we deliver projects so that it can be used.
So the results need to be deployed, the results need to be
used. There's no point delivering a
bridge if no one's going to go across it if no one lives.
In the area so, so it's all about the use, but we were
learning that as we work it to computer technology was becoming
more and more enabling. There was more that you could do
with it. And it became all about creating
the interaction, creating the relationships and understanding
what people wanted. And how they could do their job
better and what we could achieve for them through computers.
So really my journey there starts with an understanding
that we need to gain a better insight into what people are
trying to achieve. And it's only when we understand
what we're trying, what they're trying to achieve, that we can
deliver something meaningful and useful to them.
So is that really the core of what Agile meant?
So this to a large extent predates Agile.
This is prototyping, It's experimentation, It's something
that people have been doing for a long time.
We have been experimenting, we've been trying things out.
So rather than have a fixed idea about what we could deliver,
what needs to be done, it's playing with it a little bit and
and trying to make sense of what we're getting and seeing if we
can, if we can find a better solution.
Tinkering with the solution being empiricist, being playful
developers. The way we develop things has a
has the sense of a problem solving cycle.
We move some problem towards solution and we try not to
engage with this solution too early.
We go through different modes of development and to a large
extent computers enabled us to do a lot of things that we
couldn't do before. So they they enabled us to to
process large volumes of data, to do so very quickly, to do
amazing calculations, to change how we how we implement systems,
how we deal with people. They enabled us to do a lot, so
I was involved initially in trying to utilise computers in
useful ways, but also trying to make sense of what is success
and failure. And for me, prototyping was very
much an early precursor of of Agile.
We were doing exactly what our journey is doing.
The challenge was as soon as you introduced prototyping, as soon
as you engaged with the users, the user they wanted more.
So the challenge was not not a technical challenge, but there
was a management challenge. How do you control this, this
effort? It can go on forever.
And then the real challenge is how do you pay for it?
How do you structure it? So it became about.
How? How do we control this
prototyping activity? Do we only allow?
3 increments however long does increments are.
So we could say, well we will do 3 versions of the system we will
go through in three times. Or we can just do it for two
months and at the end of the two months what we have is really
the the answer unless you want to pay more and then we we will
do another increment. So it was really imposing some
kind of a managerial structure around it, and I was trying to
make sense of what seemed rather arbitrary at the time, the
notions of success. Failure, because if you say,
well, I'm going to deliver a system in six months.
And then you spend 10 months with the users because they're
engaging with it and you're delivering a better system.
Are you failing or are you succeeding?
More so? This is kind of in the 80s.
The Agile Manifesto was published in 2001.
So what was it trying to achieve and how did it fit with what was
going on with prototyping? You know, if you cast your mind
back to them, what what? What did you think its value
might be? To set the context in a way that
we are still dealing with project failures, they're quite
a few challenges around projects, primarily because we
are doing things that have not been done before.
So computers enable us to seventies 80s, really.
Even before that, some amazing things are happening through
computers. They're really exciting
projects. The problem is it's they're
quite difficult to predict. They're quite difficult to
control. It's back to this control
question. So there is a perception of
failure. There's a perception that
software projects fail a lot. And something needs to be done
about it. And they're various efforts,
various communities are formed and they're looking at different
kinds of solutions. So some of them are looking at
various ways of introducing quality.
So all kinds of quality structures, some of them are
looking. At what would later become the
idea of benefits. And some of them are looking at
ways of creating a craft, a discipline that is more
controlled. So software engineering is born
around 1969. In the NATO conference, which
brings together a lot of scientists who concerned about
suffer being out of control. So the idea of engineering
software engineering is to impose engineering structures.
So bring in that control over what we're doing and software
engineering allows us. To plan, go through the
requirements, design, develop and deliver.
And deploy meaningful software almost in an engineering kind of
way. So there are different
communities all over the place and the different communities
are engaging with different types of approaches.
Some of them more formal. Software Engineers will try and
find formal methods, but within that some would argue we still
need to work with people. Back to this if you like the
fight between. Creativity and control.
Various methods are created and throughout the Seventies, 80s
and 90s there are all kinds of structured approaches.
To obtaining requirement and structured approaches to design.
So the structured requirements, structured design and our
communities. Now those methods are becoming
more and more structured and more and more controlled in
response to the failures. We need to control the effort.
We need to know what we're doing.
We need to start here and finish over there.
And suffering engineering becomes quite controlled and at
a time I'm teaching systems and software engineering, so I'm
doing, I'm doing a lot of that. I've shifted from doing to to
teaching, researching, making sense of things.
And we we are particularly focused on some of the methods
that are emerging. So I had a strong interest in
methods. I had a strong interest in what
could be achieved and in this shift between control and
creativity and they have various efforts that are going on.
So in Japan, Takeuchi and Nonaka are saying, well to do, you
know, when you look at innovation, when you look at
creativity and how people work, it's not really a relay race
where you hand over to someone else.
So this is I think 1996, they're this, they're saying it's more
for. It's more like a rugby match.
When you pass the ball backwards and forwards that they come up
with the notion of it's a rugby match and this is the idea of
scrum being born. It goes in all kinds of
directions and at the same time other people are coming up with
other lighter methodologies. They're calling them, but
they're really methods. Lighter approaches to
development, lighter life cycles.
So what happens in 2001 is 17 people involved in the creation
and the development of of flight and methods.
Get together. What they have in common is that
they all believe that there's too much, if you like, structure
and control. And we need to be we need to
liberate ourselves a little bit. And we need to recognise these
are the people that come into the meeting, they've all created
their own methods or selling particular approaches.
That emphasise working with users.
Little bits of prototyping. Some of the methods can be
traced back to prototyping. Some of them are.
Simply lighter approaches. They're not project managers,
they're software developers. They're interested in software
and what they do is they they get together for a weekend.
There were a number of similar get togethers and this is this
was one particular group and they do want a lot of groups to
get together. Do they?
They come up with a manifesto, of course.
Wouldn't it be great if we could shape the world in the way we
like it and everyone used like methods?
So they come up with a statement, and a statement
reflects the kind of thinking at a time.
It comes a little bit from manufacturing, a little bit from
innovation. They're inspired by other
developments that are going on at the time.
And they're really saying we think we should be working in
different ways. We should be using lighter
methods. We should be doing things with
those lighter methods that without the control, without the
structure, without the imposition of of the.
Bureaucracy and they're all trying to sell their own
individual methods at the time. So the challenge for them, for
17 people in that in that room is to to come up with a coherent
statement that reflects. The flexibility, the freedom,
the that are common across the all the methods that they're all
representing. So.
So how do we create a sort of a statement of independence?
What is our our common cause? Did you know any of them
personally? Had you met any of these people
in that room? Yes, I knew a few, quite a few
of them at the time. I've met since.
I've done keynotes and I've spoken at conferences with many
of them, so I've so, yes, very much so.
And. I knew many others liked them.
It was pretty much the prevailing kind of approach.
Programmers don't like structure.
They don't like controls. So having come from that
background, I was familiar with. I was familiar with the
constraints. I was familiar with the ways of
thinking. So they basically work on the
manifesto over the weekend and the manifesto is 4 statements of
value. So we prefer something over
something else. We prefer people over tools,
essentially. The relationships and working
with people of the tools and methods and approaches, we
prefer delivering working products over comprehensive
documentation. So so we like to do things.
So the first one is saying we like to work with people.
And I suppose that's quite a strong statement for
programmers. Remember, programmers are the
people you keep in the basement. Now they're coming out.
They're saying we want to work with people.
Let me work with people. So they're working with people
now. And like me, I suppose they got
excited by the fact that ohh, I can shape things for you, I can
shape the world for you, I can use the computer to create a
better world for you. So.
So we work with people we prefer.
Delivering, working things, doing rather than documenting
and documentation wasn't a popular part of the software
process. It's the thing that came at the
end and it was you think you've done the thing.
Now go and test and then go and do the documentation.
We prefer customer collaboration over really contract
negotiation. And we prefer responding to
change over just following a plan.
If something changes, let's go with it.
Let's flow with it. So it represents an attitude, it
represents an age, it represents the people in the room, it
represents a long weekend in the cold February holiday resort.
It's a group of people saying wouldn't it be great if we could
change the world? So they come up with those
statements. Later on they will add the the
12 principles, but those four values really about 30, I don't
know, 40 words, something like that.
This is our manifesto. This is what we think we'd like
the world to look like. And one of them goes and posts
it on the web because you can now and reasonably quickly
there's some traction. Various communities realise
that. This is going on.
And they say, yeah, yeah, yeah, me too, me too.
I want to be part of this movement.
I believe in those principles as well.
So it becomes something within the software development
community, becomes a way of thinking about how we do, how we
develop software. I don't want to say how we do
projects, but certainly how we develop software, because there
is an implication that we're back to this creativity and
control. So it becomes perhaps an
alternative way of looking at how we develop.
I mean, I'm understanding that it was never intended to be a
management method or a project management method, but it has
permeated not only project management but management in
general. What's your take on this?
Do you think the people who originally created this are
surprised that this has happened?
What's your view on how Agile has influenced project
management and what it's become now?
Right. So when they post the manifesto,
it's the manifesto for Agile software development.
So that's quite explicit that that that's the title of it and
and and it remains like that. The document hasn't changed
since 2001. There was a conscious decision.
It reflects a point in time when they got together.
So what essentially was the the manifesto for Agile software
development became known as the Agile Manifesto.
So at some point we we dropped the software development.
But when I first knew it, it was this manifesto for Agile
software development. And within the software
development community there was a mixed response.
Some people got excited about what Agile?
And deliver. About the fact that it could
liberate you from some bureaucracy, it enabled you to
do things differently. It was quite an exciting
development certainly for programmers and a lot of the
agile literature from that time is saying we don't need project
managers, It's the end of the project manager.
Project manager can can disappear.
One of them is saying the role of the project manager is to
deliver pizza. So, so there are different
perspectives around that, but somewhere along the line.
People get excited by the concept.
Now. The software development
community, to a large extent, has moved on from Agile.
This suffering the software development community, if I
reflect back over the years, roughly every 10 years there's a
new trend and a new trend coming comes in.
And it could be structured systems analysis or it could be
structured design or software engineer.
Something will come along and it will grasp everyone and they'll
get excited by it. And then there'll be a new toy.
And we are looking at roughly every 10 years being there's a
new exciting development and to a large extent software
development and software engineering have moved on to
thinking about the delivery of services rather than products
and looking at continuous delivery, continuous testing,
continuous delivery, continuous deployment, So delivery becomes
more constant. The agile community has in
software development has certainly moved on.
But a lot of others, as you said, have have picked up on on
Djali. Do we need to be agile?
And that's without fully understanding what our journey
is. Our journey is this mythical
thing that we can do so we can be agile.
That that sounds good. We can be.
I want to be able to do magic. I want to do all kinds of
exciting things. So, so we can be agile, we can
do things, and we can pivot. We can respond.
And a lot of what our child gives you is the ability to.
Ready to to respond to change, Going back to where I started
and those experiments. Agile allows you to do little
experiments and on the basis of those experiments to amend to
some extent what you're doing. So you're being more responsive,
you're being more adaptive. And in terms of management,
management goes back to the bureaucracy to control the.
So having this magic ability to, yeah, do you want to be agile?
Yes, of course I do agile. Agile is exciting.
It's not boring old structure, it's I can be agile, I can do
what I like. It allows me to do all sorts of
amazing things. So there's a lot of traction,
There's a lot of people are impressed by it.
Some of the origins, as I pointed out, also go back to
production and manufacturing. And innovation management.
And there's a challenge around finding the balance between
product development and project management.
And is it a technical development that we are doing?
Can we get, can we do away with the project manager?
Can we? So the project management
community has found agile. Now they're very it seems like
an exciting development, and there are all kinds of
conversations around what replaces what does all this
waterfall V agile discussion everywhere, and which is
relatively meaningless because neither of them is a project
management approach. We're APM, the only chartered
membership organisation for the project profession.
When you become an AP member, you'll receive the resources and
support you need to make an impact, delivering better
projects with better outcomes. Plus you'll access exclusive
training and benefits to support your ongoing career development.
Find out how we can help you reach your potential by visiting
apm.org.uk, Because when projects succeed, society
benefits. Is the project management
profession using Agile? However, they define it as an
effective way to have fewer project failures.
Is it a useful tool? Interesting question.
We have fewer of the old failures, but our job will
introduce inevitably introduce its own kinds of failures.
I've been collecting failures for literally all of my career,
not just my own. I'm I'm less fussy nowadays.
I'm I'm happy to welcome anyone to the club. 10s of thousands of
failures have made it in and there are some major failures
and they're very different. What what Agile allows you to do
is you've got small, small teams doing things and this is what
the essence going back to 2001. This is the essence of that
meeting. This is what it was all about.
It was a small solution to a small problem of a small group
of people working together to do small things.
This is what our job enables you to do and this is what they were
talking about. So this is it liberates a small
team to do things far more creatively.
And at the time we were doing some research around Agile which
the view of determining whether it had, whether it had values.
We were looking at the quality of what the teams were producing
and we did a one year study where we we looked at multiple
projects and we looked at the outputs of teams in the banking
sector and we we measured the the output and the agile teams
produced far more code which means they produced far more
functionality. The obvious question would be
was that useful functionality? The answer would be to spend a
lot more time with your clients. So they did a lot more.
They delivered a lot more. But it introduced different
challenges. The fact that the software
community has moved on and is interested in dev OPS reflects
an interest in continuing operations.
It reflects a need to integrate other activities all the time to
to to continue to develop and to have backlogs that allow you to
bring in items from different areas and integrate them into
your deliverables. It means that agility.
So make the chef now. Agility allows you to do things
differently. Agility allows you to deal with
uncertainty. Agility allows you to see what
you can do. Action has been really exciting
during the Pandemic actually allows you to start.
You have a small team run with it, go with it.
Here's the ball run. So Agile allows you to enables
you to do all kinds of things. It's up to the project manager
to decide whether it is the most useful way of deploying off
using their team and the most useful way of doing things.
It's still the skills of the project managers.
I don't think the world is just to deliver pizza.
The role is to decide what you're hoping to deliver and to
say, well, we can do things together, we can do exciting
things and agile will allow us to experiment going back to
where we're starting, you can do your experiments.
You can see whether it makes sense.
During the pandemic, that was amazing.
You actually said to the team, so we need to start.
Rules are changing from This is a Friday afternoon announcement.
Rules are changing from Monday. Go run with it.
We don't really know what it means.
Let's go and explore the boundaries and it it liberates
teams. It enables them to do things but
it it's something that allows a small a small team to do things.
Somewhere along the line we realised, wait a minute, if one
team can be our gel, can we have lots of teams playing in the
same field being agile? So then we started talking about
scaling agile, and now we're talking about business agility.
And now we're talking about all kinds of other notions which are
far more complex. The problem is, if you allow one
team to, I'm not gonna say misbehave.
If you allow one team to have fun, you have to allow all of
them to have give everyone the ability to have fun together.
And that's where we come back to that good old balancing act
between creativity and structure and control.
One team having fun is exciting enough and noisy enough, but can
you let hundred teams, thousand teams do that?
So we are back to OK, now we need to introduce scaling
methods and approaches to to to getting multiple teams work
together. So we are beginning to put some
certain controls around that. If you ask the original people
who attended the the meeting and created the Agile Manifesto what
they think about it, They hate it.
Their view is that Agile was this liberating activity and
they don't like the many of them, don't like the scaling
approaches. There are various methods that
have been designed to do that and they feel that they are
adding a lot of structure, which you would need to do when you
have multiple teams playing and having fun together at the same
time. You want to control the noise,
you want to control the space, you want to control what's going
on. You need to manage it, so the
challenge for project managers is to think what it can achieve
through our journey. It goes back to the skills of
the project manager. We if we don't assume that all
the knowledge is there, it's really how do we deploy, how do
we create, how do we become pragmatic managers use their
professional judgement and decide how to deliver, how to
shape, how to deploy in meaningful ways.
So it's creating strategies that will allow the mix that will
allow the different activities that need to take place to
happen together at the same time.
Darren That's a. Fantastic way to end this
podcast. Thank you so much for your time.
Really, really interesting speaking to you about this.
So thank you again. It's a pleasure.
Thank you very much. Thanks again to.
Darren for joining us and to you for listening to the APM
Podcast. Don't forget to look out for
more episodes or to rate and review us wherever you get your
podcasts. We welcome you to get in touch
with your comments, feedback and suggestions by emailing us at
apmpodcast@thinkpublishing.co.uk. This podcast has been brought to
you by APM, The Chartered Body for the project profession.
For more information on APM, visit to apm.org.uk.