1
00:00:00,160 --> 00:00:02,880
The term context engineering has
become the biggest buzzword in 

2
00:00:02,880 --> 00:00:07,560
AI in the past 10 days, and it's
a word that if industry leaders 

3
00:00:07,560 --> 00:00:11,640
are to be trusted, you must now 
start to learn, with many people

4
00:00:11,640 --> 00:00:13,720
calling it the next big 
evolution from prompt 

5
00:00:13,720 --> 00:00:16,200
engineering. 
In very simple terms, prompt 

6
00:00:16,200 --> 00:00:19,960
engineering is what you ask an 
AI model to do for you, whereas 

7
00:00:19,960 --> 00:00:23,440
context engineering is what a 
model knows when you ask it 

8
00:00:23,440 --> 00:00:25,560
something. 
This shift reflects very hard 

9
00:00:25,560 --> 00:00:28,240
one lessons when building AI 
agent systems. 

10
00:00:28,840 --> 00:00:31,400
You can have something that has 
all the bells and whistles, but 

11
00:00:31,400 --> 00:00:34,240
if it doesn't give the right 
context, it can never deliver 

12
00:00:34,240 --> 00:00:36,720
good outcomes for people. 
In today's episode, we're going 

13
00:00:36,720 --> 00:00:39,640
to discuss the evolution from 
prompt engineering to context 

14
00:00:39,640 --> 00:00:42,560
engineering, and I'll breakdown 
what context means when it comes

15
00:00:42,560 --> 00:00:44,880
to AI. 
And finally, I'll actually give 

16
00:00:44,880 --> 00:00:48,120
a breakdown of some of the real 
things that you as a person are 

17
00:00:48,120 --> 00:00:51,320
going to start noticing as a 
result of improved context 

18
00:00:51,320 --> 00:00:53,160
engineering. 
This is in the Loop with Jack 

19
00:00:53,160 --> 00:00:54,840
Halton. 
I hope you enjoy the show. 

20
00:01:06,550 --> 00:01:08,270
So I'm going to assume most of 
you already know what prompt 

21
00:01:08,270 --> 00:01:10,990
engineering is, but just just 
for the sake of the episode, I'm

22
00:01:10,990 --> 00:01:14,350
going to spend 20 seconds on it.
Prompt engineering is a phrase 

23
00:01:14,350 --> 00:01:18,510
that gained mainstream adoption 
and traction generally after the

24
00:01:18,510 --> 00:01:21,470
launch of ChatGPT in about late 
2022. 

25
00:01:21,950 --> 00:01:24,430
And this is because millions of 
people are suddenly trying to 

26
00:01:24,430 --> 00:01:26,230
figure out how to talk to an AI 
system. 

27
00:01:27,320 --> 00:01:29,800
It was one of the very early job
roles that were advertised for 

28
00:01:29,800 --> 00:01:32,080
by big companies. 
Prompt engineering became a 

29
00:01:32,080 --> 00:01:35,720
skill set that people went into 
courses about generally became a

30
00:01:35,720 --> 00:01:37,960
really important skill. 
And engineering a prompt 

31
00:01:37,960 --> 00:01:42,120
basically meant using trial and 
error and intuition really to 

32
00:01:42,120 --> 00:01:44,640
find a set of I guess 
instructions and also the way 

33
00:01:44,640 --> 00:01:48,040
the instructions are worded to 
make the model do what you want 

34
00:01:48,040 --> 00:01:50,200
it to do. 
A really famous example of this 

35
00:01:50,200 --> 00:01:54,320
was the very famous prompt that 
literally just said, let's think

36
00:01:54,320 --> 00:01:56,040
step by step. 
And this led to train of 

37
00:01:56,040 --> 00:01:59,360
thought, which is a form of AI 
reasoning, which typically 

38
00:01:59,360 --> 00:02:02,840
improves the performance of how 
an AI model performs when you 

39
00:02:02,880 --> 00:02:05,480
just ask it a simple question 
because it's making itself a 

40
00:02:05,480 --> 00:02:07,000
plan and following it step by 
step. 

41
00:02:07,360 --> 00:02:10,800
However, as the use cases for AI
have become increasingly 

42
00:02:10,800 --> 00:02:16,200
advanced, so have the steps and 
the tasks that an agent must 

43
00:02:16,200 --> 00:02:18,800
follow to actually, I guess, 
deliver value today. 

44
00:02:18,920 --> 00:02:21,880
And as a result, there were very
clear limitations with prompt 

45
00:02:21,880 --> 00:02:25,120
engineering because early prompt
engineering is essentially the 

46
00:02:25,680 --> 00:02:28,560
A1 shot attempt. 
So meaning you write a single 

47
00:02:28,560 --> 00:02:31,360
prompt, which could maybe 
include a few examples or 

48
00:02:31,360 --> 00:02:33,760
instructions, and the model then
give an answer as a result. 

49
00:02:33,880 --> 00:02:36,680
But many tasks require dialogue 
with users. 

50
00:02:37,240 --> 00:02:39,560
It requires the agent to 
memorize things. 

51
00:02:39,560 --> 00:02:42,760
It requires the agent to use 
tools, so integration into 

52
00:02:42,760 --> 00:02:45,520
separate systems to capture data
from those systems. 

53
00:02:45,960 --> 00:02:49,560
And so, for example, asking a 
model to, say, answer this 

54
00:02:49,560 --> 00:02:52,800
question by searching the web 
and then write a summary can't 

55
00:02:52,800 --> 00:02:55,560
be done in a single one prompt 
attempt. 

56
00:02:55,760 --> 00:02:58,280
The model needs to take many 
different actions, like doing a 

57
00:02:58,280 --> 00:03:00,680
search and remembering what it 
found and then doing something 

58
00:03:00,680 --> 00:03:03,440
with that information. 
And many builders in the space, 

59
00:03:03,440 --> 00:03:06,080
including ourselves, do things 
called prompt chaining, which 

60
00:03:06,080 --> 00:03:08,760
essentially means chaining 
prompts together to give the 

61
00:03:08,760 --> 00:03:11,400
model more context to every 
single step. 

62
00:03:11,960 --> 00:03:14,720
So take a step and then follow 
another step and use the 

63
00:03:14,720 --> 00:03:16,160
information from that previous 
step. 

64
00:03:16,240 --> 00:03:19,440
And there was many different 
innovations that include RAG, so

65
00:03:19,440 --> 00:03:23,600
retrieval, augmented generation,
including what's called React, 

66
00:03:23,600 --> 00:03:28,080
So reasoning and action, which 
essentially are ways of getting 

67
00:03:28,080 --> 00:03:32,080
a model to prompt and capture 
subsequent information and use 

68
00:03:32,080 --> 00:03:34,160
that as context to provide a 
better answer. 

69
00:03:34,200 --> 00:03:37,520
The point being is that it's not
actually just the instruction 

70
00:03:37,520 --> 00:03:39,120
sent to a language model that 
matters. 

71
00:03:39,360 --> 00:03:42,640
It's actually the context the 
language model has and how it 

72
00:03:42,640 --> 00:03:45,240
carries context throughout many 
different steps. 

73
00:03:46,480 --> 00:03:48,480
And this is where context 
engineering really starts to 

74
00:03:48,480 --> 00:03:50,960
take center stage. 
Because context engineering is 

75
00:03:51,000 --> 00:03:53,840
all about deciding what a 
language model should know 

76
00:03:53,840 --> 00:03:55,760
before it speaks and answers a 
human. 

77
00:03:55,760 --> 00:03:59,600
Not just what you ask, but what 
data or tools or memories, 

78
00:03:59,600 --> 00:04:03,600
instructions, and history should
be fed into that system to guide

79
00:04:03,600 --> 00:04:05,560
its answer. 
And the term itself became 

80
00:04:05,800 --> 00:04:10,080
largely popularized by Shopify 
CEO very recently on Twitter. 

81
00:04:10,080 --> 00:04:13,320
He said, I prefer the name 
context engineering to prompt 

82
00:04:13,320 --> 00:04:15,000
engineering. 
It better describes the core 

83
00:04:15,000 --> 00:04:18,079
skill set, which is giving the 
model the context it needs to 

84
00:04:18,079 --> 00:04:21,120
plausibly solve the task. 
And there are other big names 

85
00:04:21,120 --> 00:04:24,800
out there like Andre Kapathy who
was one of the Co founders or 

86
00:04:24,800 --> 00:04:27,160
let's say founding team members 
of open AI. 

87
00:04:27,400 --> 00:04:31,280
And he's very famous and even 
coined the term vibe coding. 

88
00:04:31,480 --> 00:04:34,040
He described it more as an art 
and science of filling the 

89
00:04:34,040 --> 00:04:38,560
models context window just 
enough to make the next step in 

90
00:04:38,560 --> 00:04:42,000
its sequence work, but not too 
much and not too little. 

91
00:04:42,000 --> 00:04:43,800
It's just the right amount of 
information. 

92
00:04:43,880 --> 00:04:46,560
So you can almost consider 
context engineering like 

93
00:04:46,560 --> 00:04:48,920
orchestration. 
You're orchestrating an agent 

94
00:04:48,920 --> 00:04:52,000
over many subsequent steps and 
trying to control what 

95
00:04:52,000 --> 00:04:55,320
information and context it 
should have after each step and 

96
00:04:55,320 --> 00:04:57,960
how it should use that context 
in the subsequent one. 

97
00:04:58,120 --> 00:05:01,920
Let's go one level deeper and 
actually unpack what counts as 

98
00:05:01,920 --> 00:05:03,560
context when it comes to an AI 
model. 

99
00:05:04,160 --> 00:05:08,120
Because every piece of software 
built over a language model, 

100
00:05:08,120 --> 00:05:10,920
that's mindset, that's every 
provider is essentially this 

101
00:05:10,920 --> 00:05:15,120
fantastic piece of software that
is dictating the context and the

102
00:05:15,120 --> 00:05:18,240
instructions sent to a model and
how that information is surfaced

103
00:05:18,240 --> 00:05:20,720
to a user. 
And that is where all the hard 

104
00:05:20,720 --> 00:05:23,960
work comes from because it's 
flipping difficult to get it all

105
00:05:23,960 --> 00:05:26,440
to work. 
And you'll see why in a second. 

106
00:05:26,440 --> 00:05:29,200
There's many layers to context. 
So you've got the first piece of

107
00:05:29,200 --> 00:05:31,160
context, which is the immediate 
prompt. 

108
00:05:31,600 --> 00:05:36,240
So this is the direct question 
or instruction by a user, for 

109
00:05:36,240 --> 00:05:38,000
example, creating a marketing 
strategy. 

110
00:05:39,120 --> 00:05:41,000
And this is where prompt 
engineering has typically been 

111
00:05:41,000 --> 00:05:45,600
focused #2 you've got systems 
instructions, which essentially 

112
00:05:45,880 --> 00:05:50,440
a system message sent to the 
large language model to set a 

113
00:05:50,440 --> 00:05:52,440
behavior. 
So you are an expert travel 

114
00:05:52,440 --> 00:05:53,720
panel. 
The second layer is called 

115
00:05:53,720 --> 00:05:56,880
system instructions and 
essentially this is where you 

116
00:05:56,880 --> 00:06:00,000
send a system message to a large
language model such as you are 

117
00:06:00,000 --> 00:06:03,920
an expert travel planner. 
So this is the context that 

118
00:06:03,920 --> 00:06:06,840
influences say the tone, the 
scope, and this alone can be 

119
00:06:06,840 --> 00:06:08,800
quite detailed. 
We've got an entire agent 

120
00:06:08,800 --> 00:06:12,000
builder around these which 
dictates how to behave, not to 

121
00:06:12,000 --> 00:06:15,560
behave, guard rails, policy, 
personality, how it should 

122
00:06:15,560 --> 00:06:18,800
structure its responses, but 
that's all system messages. 

123
00:06:18,920 --> 00:06:23,160
Again, the software built on top
of these things a highly complex

124
00:06:23,160 --> 00:06:25,960
in advance just to make sure the
right thing is sent at the right

125
00:06:25,960 --> 00:06:27,560
time with the right amount of 
context. 

126
00:06:27,720 --> 00:06:30,960
So third piece of context is the
conversation history, which is 

127
00:06:30,960 --> 00:06:32,280
often called the short term 
memory. 

128
00:06:32,640 --> 00:06:35,800
So in a conversation which they 
ChatGPT, you want parts of that 

129
00:06:35,800 --> 00:06:39,680
chat to be included in every 
single prompt that goes to the 

130
00:06:39,680 --> 00:06:41,720
LLM. 
So when you ask a question, it 

131
00:06:41,720 --> 00:06:45,200
has the context of all previous 
conversations you've had in that

132
00:06:45,200 --> 00:06:50,520
thread again, so you might say 
as I was speaking about earlier,

133
00:06:50,760 --> 00:06:52,680
and therefore it knows what 
earlier means. 

134
00:06:52,680 --> 00:06:56,520
And that's because every time a 
message by a user is sent 

135
00:06:56,920 --> 00:07:00,400
ChatGPT is a piece of software 
also takes the message history, 

136
00:07:00,560 --> 00:07:03,000
condenses it down and sends it 
to the language model. 

137
00:07:03,200 --> 00:07:05,320
So it then has that context of 
the conversation. 

138
00:07:05,440 --> 00:07:09,280
The 4th type of context that's 
absolutely critical is long term

139
00:07:09,280 --> 00:07:12,600
memory or persistent memory. 
This can be a summary of 

140
00:07:12,600 --> 00:07:15,280
previous interactions. 
This could be stored facts about

141
00:07:15,280 --> 00:07:19,200
a user, such as my preferences 
or information about my profile 

142
00:07:19,400 --> 00:07:22,480
or even previous conversations 
between different threads. 

143
00:07:23,200 --> 00:07:25,560
So it's that context that 
persists across different 

144
00:07:25,560 --> 00:07:29,440
messages. 
So it knows that you work in a 

145
00:07:29,440 --> 00:07:32,400
certain job, you like response 
in a certain way, you hate 

146
00:07:32,400 --> 00:07:35,600
things that sound like this. 
We're actually as mindset 

147
00:07:35,600 --> 00:07:39,200
launching an entire long term 
memory layer next month, which 

148
00:07:39,240 --> 00:07:42,840
enables our customers to dictate
what an agent should remember. 

149
00:07:43,000 --> 00:07:44,960
Then you've got retrieving 
external information. 

150
00:07:45,280 --> 00:07:48,320
So this might be both RAG or 
integrations into third party 

151
00:07:48,320 --> 00:07:49,800
systems. 
So sometimes you'll ask 

152
00:07:49,800 --> 00:07:53,160
questions that will require 
context of external documents or

153
00:07:53,160 --> 00:07:56,280
code snippets or videos. 
And so the software that you'll 

154
00:07:56,280 --> 00:08:00,480
be interacting with will then 
call that information, bring it 

155
00:08:00,480 --> 00:08:03,960
into the prompt that is sent. 
So your message plus that 

156
00:08:03,960 --> 00:08:07,400
information from an external 
data source sent to the LLM, and

157
00:08:07,400 --> 00:08:09,440
then the LLM will use that to 
answer your question. 

158
00:08:09,600 --> 00:08:12,520
Another example could be a 
customer support bot that pulls 

159
00:08:12,520 --> 00:08:15,280
in user's account data whenever 
it answers a question about 

160
00:08:15,280 --> 00:08:16,720
billing. 
You've then got another layer 

161
00:08:16,720 --> 00:08:19,880
down, which is tool definitions.
So if the models can use tools 

162
00:08:19,880 --> 00:08:24,640
like say, making API calls or 
running calculations, to do 

163
00:08:24,640 --> 00:08:27,600
that, it needs to understand how
to actually call those tools or 

164
00:08:27,600 --> 00:08:30,480
interact with external tools, 
and it must use the information 

165
00:08:30,480 --> 00:08:33,679
it gets back to answer or 
support what it's doing. 

166
00:08:33,880 --> 00:08:36,360
And so whenever it uses one of 
those tools, let's say 

167
00:08:36,600 --> 00:08:39,520
summarizes all my WhatsApp 
conversations, you then want it 

168
00:08:39,520 --> 00:08:43,600
to be able to retain the output 
of that query into WhatsApp for 

169
00:08:43,600 --> 00:08:46,480
the next step in the process. 
So yeah, that's different layers

170
00:08:46,480 --> 00:08:48,600
of what context actually means 
in AI system. 

171
00:08:48,600 --> 00:08:51,840
Again, it just sounds like a 
human that requires context to 

172
00:08:51,840 --> 00:08:55,520
do good work, but it actually 
has very big implications on 

173
00:08:55,520 --> 00:08:57,280
what it means for a genetic 
systems. 

174
00:08:57,280 --> 00:08:59,280
And it's one of the big reasons 
we're launching a new 

175
00:08:59,280 --> 00:09:03,680
orchestration system that 
enables our customers to really 

176
00:09:03,680 --> 00:09:06,040
give models the right context at
the right time. 

177
00:09:06,240 --> 00:09:09,520
And I think in summary, and I'll
explain why in just a second, 

178
00:09:10,000 --> 00:09:13,240
simpler processes or sequential 
step by step processes are 

179
00:09:13,240 --> 00:09:15,280
better than multi agent systems 
right now. 

180
00:09:26,160 --> 00:09:28,360
We've talked about multi agent 
systems before in previous 

181
00:09:28,360 --> 00:09:31,720
episodes, but this is where you 
have teams of agents doing many 

182
00:09:31,720 --> 00:09:34,320
different things. 
And it's really relevant to this

183
00:09:34,320 --> 00:09:38,600
conversation because it's all 
about the concept of context 

184
00:09:38,600 --> 00:09:41,520
engineering and really 
highlights why context is so 

185
00:09:41,520 --> 00:09:43,720
important. 
So let me delve a level deeper 

186
00:09:44,040 --> 00:09:46,760
by explaining the problem of 
having teams of agents doing 

187
00:09:46,760 --> 00:09:49,280
many different things at once. 
There are many different use 

188
00:09:49,280 --> 00:09:53,560
case ideas that people often 
think, oh, let's get a team of 

189
00:09:53,560 --> 00:09:56,680
agents to do this. 
Let's get a team of agents also 

190
00:09:56,680 --> 00:10:00,360
working on tasks in parallel to 
each other and then breaking 

191
00:10:00,360 --> 00:10:03,200
down those tasks into subtasks 
and getting other agents to help

192
00:10:03,200 --> 00:10:06,040
with those subtasks. 
So the idea might be to have, 

193
00:10:06,040 --> 00:10:08,880
say, Agent A break a problem 
into small little parts, 

194
00:10:09,320 --> 00:10:13,640
delegate it to agent B&C, and 
then have a say combine all of 

195
00:10:13,640 --> 00:10:15,160
their information into one 
answer. 

196
00:10:15,600 --> 00:10:18,480
Because in theory it would just 
make the responses faster 

197
00:10:18,880 --> 00:10:21,920
because you have people working,
or so I say, agents working in 

198
00:10:21,920 --> 00:10:23,920
parallel. 
And those agents could be 

199
00:10:23,920 --> 00:10:27,040
specialized, breaking down 
tasks, for example, or 

200
00:10:27,160 --> 00:10:31,320
summarizing information. 
So in theory it makes sense, but

201
00:10:31,400 --> 00:10:33,640
often it will fail because of 
coordination. 

202
00:10:33,640 --> 00:10:37,000
And when we say coordination, 
what we actually mean is 

203
00:10:37,000 --> 00:10:41,120
context, contacts not being 
shared between different agents.

204
00:10:41,240 --> 00:10:44,160
Now, there was a fantastic paper
recently released by Cognition 

205
00:10:44,160 --> 00:10:47,760
Labs again, the one of the labs 
behind what's called the Devin 

206
00:10:47,760 --> 00:10:51,280
agent, one of the very famous 
coding agents before the new age

207
00:10:51,280 --> 00:10:55,760
coding apps emerged. 
And that paper was called Don't 

208
00:10:55,760 --> 00:10:58,960
build multi agent systems. 
And to show why they actually 

209
00:10:59,240 --> 00:11:02,080
tasked a set of agents to build 
a Flappy Bird clone. 

210
00:11:02,240 --> 00:11:06,040
To accomplish this. 
The I created sub agents want to

211
00:11:06,040 --> 00:11:08,760
create the background graphics 
and another one to create that 

212
00:11:08,760 --> 00:11:11,000
player bird, the thing that 
you're going to control. 

213
00:11:11,040 --> 00:11:14,120
Now without the full context, 
the first sub agent might 

214
00:11:14,120 --> 00:11:18,200
interpret the task a little bit 
different and generate a Mario 

215
00:11:18,200 --> 00:11:21,520
like background instead of a 
Flappy Bird pipes background. 

216
00:11:21,920 --> 00:11:23,640
Now they're very similar but 
slightly different. 

217
00:11:24,120 --> 00:11:26,720
But the second agent might then 
produce a bird that doesn't 

218
00:11:26,720 --> 00:11:29,320
therefore fit with the style of 
graphics. 

219
00:11:30,160 --> 00:11:33,920
So when these two agents merge 
the outputs to combine it into a

220
00:11:33,920 --> 00:11:37,440
single game, it obviously 
doesn't work. 

221
00:11:38,080 --> 00:11:41,600
It feels like separate products.
And the root cause of this is 

222
00:11:41,600 --> 00:11:44,920
that the sub agents lack the 
full context of the original 

223
00:11:44,920 --> 00:11:48,400
objective and the work of other 
agents going on at the same 

224
00:11:48,400 --> 00:11:50,000
time. 
Now the straightforward solution

225
00:11:50,000 --> 00:11:52,960
you might say would just to be 
give all these sub agents all 

226
00:11:52,960 --> 00:11:56,200
the information they need, like 
the information on the task, the

227
00:11:56,200 --> 00:11:59,600
description, any decisions of 
the agents have made so they can

228
00:11:59,600 --> 00:12:01,920
go and do a good job. 
In practice, that means when 

229
00:12:01,920 --> 00:12:04,720
agent A, say, delegate 
something, it should pass along 

230
00:12:04,720 --> 00:12:08,240
not just the sub task, but also 
all of the information, the 

231
00:12:08,240 --> 00:12:11,280
conversation history, everything
that's happened up until that 

232
00:12:11,280 --> 00:12:13,600
point, and all the decisions 
that led there. 

233
00:12:14,320 --> 00:12:17,920
However, even sharing the entire
context isn't enough sometimes 

234
00:12:18,240 --> 00:12:21,360
because the sub agents then 
might work with full knowledge 

235
00:12:21,360 --> 00:12:24,280
of say the main objective, yet 
still produce something that's 

236
00:12:24,320 --> 00:12:28,000
incompatible because they may 
make assumptions throughout that

237
00:12:28,000 --> 00:12:30,160
process that conflict with each 
other. 

238
00:12:30,480 --> 00:12:33,560
Like 1 deciding that the game 
should have a cartoon style and 

239
00:12:33,560 --> 00:12:36,360
the other a realistic style. 
Again, you'll notice that every 

240
00:12:36,360 --> 00:12:40,000
single decision or action by an 
agent will always have a knock 

241
00:12:40,000 --> 00:12:42,280
on effect. 
And so overall what Cognition 

242
00:12:42,280 --> 00:12:46,120
Labs have claimed and believed 
after much testing is that 

243
00:12:46,120 --> 00:12:50,040
single agent architectures, 
which means that one agent 

244
00:12:50,280 --> 00:12:54,440
follows sequentially a few steps
is far easier to get a fantastic

245
00:12:54,440 --> 00:12:57,720
output than multi agent systems.
And it's exactly as I said, why 

246
00:12:57,720 --> 00:13:01,440
mindset is enabling this exact 
use case making enabling our 

247
00:13:01,440 --> 00:13:05,960
customers to dictate all the 
different steps and separate 

248
00:13:05,960 --> 00:13:08,680
integrations and the long term 
memory that an agent should 

249
00:13:08,680 --> 00:13:11,720
carry throughout a conversation 
and essentially what it should 

250
00:13:11,720 --> 00:13:14,680
do with each part of its data 
and output at each step. 

251
00:13:14,680 --> 00:13:16,600
And they're not the only ones 
that reported this because 

252
00:13:16,600 --> 00:13:19,720
Anthropic did a lot of research 
into multi agent systems for 

253
00:13:19,720 --> 00:13:23,760
their research assistance and 
hire the exact same problem. 

254
00:13:24,720 --> 00:13:27,680
We've personally found that same
problem when trying to use multi

255
00:13:27,680 --> 00:13:29,520
agent systems in our own 
business. 

256
00:13:29,600 --> 00:13:32,640
So for all the excitement of 
multi agent systems with teams 

257
00:13:32,640 --> 00:13:35,840
of agents doing things and and 
even a single agent doing 

258
00:13:35,840 --> 00:13:39,240
something really powerful, it 
all comes down to how well the 

259
00:13:39,240 --> 00:13:44,400
software product on top of those
large language models can prompt

260
00:13:44,800 --> 00:13:47,240
and give it context every single
time it prompts. 

261
00:13:47,480 --> 00:13:49,840
So I think this is going to be a
growing area of innovation over 

262
00:13:49,840 --> 00:13:53,040
the next sort of 12 months. 
And obviously I believe 

263
00:13:53,040 --> 00:13:55,240
everything will be solved and 
multi agent systems will 

264
00:13:55,240 --> 00:13:59,760
probably be possible in I'd say 
12 months, but we can always 

265
00:13:59,760 --> 00:14:01,800
half it and everything happens 
faster than we thought. 

266
00:14:12,840 --> 00:14:15,800
So I guess the interesting 
question therefore, is as a 

267
00:14:15,800 --> 00:14:18,720
listener right now, how does 
this impacts you? 

268
00:14:18,720 --> 00:14:20,360
Like what should you know about 
this? 

269
00:14:20,360 --> 00:14:23,520
Why is this relevant to you? 
But really simply, I think good 

270
00:14:23,520 --> 00:14:26,800
context engineering, either as a
user using just something simple

271
00:14:26,800 --> 00:14:30,360
as ChatGPT, or using a product 
that's an AI product. 

272
00:14:30,640 --> 00:14:33,560
Good context engineering is 
where you're left with a sense 

273
00:14:33,560 --> 00:14:37,560
of that was actually helpful. 
It also massively influences the

274
00:14:37,560 --> 00:14:41,040
types of experiences and 
features and options as a user 

275
00:14:41,040 --> 00:14:42,680
that you're going to be given by
products. 

276
00:14:42,680 --> 00:14:45,040
So one of those are going to be 
personalization. 

277
00:14:45,280 --> 00:14:47,720
The system is going to 
increasingly be able to remember

278
00:14:47,720 --> 00:14:49,880
everything about you, your 
preferences, and previous 

279
00:14:49,880 --> 00:14:51,640
conversations. 
You're going to start to see 

280
00:14:51,640 --> 00:14:55,040
more user interface and user 
experience patterns emerge. 

281
00:14:55,560 --> 00:14:58,240
For example, as AI becomes 
integrated and say, apps, 

282
00:14:58,760 --> 00:15:02,320
designers are going to have to 
come up with ways to give 

283
00:15:02,320 --> 00:15:06,080
context to agents without users 
actually realizing it. 

284
00:15:06,080 --> 00:15:09,160
For example, many chat 
interfaces have an upload file 

285
00:15:09,160 --> 00:15:11,880
button, which enables you to 
upload a document. 

286
00:15:12,280 --> 00:15:13,960
From a user's perspective, it's 
straightforward. 

287
00:15:13,960 --> 00:15:17,040
I want to talk to a document. 
But actually what you're doing 

288
00:15:17,040 --> 00:15:20,680
is giving the agent or the AI 
language model context. 

289
00:15:21,400 --> 00:15:23,440
You're also going to have 
different modes of working. 

290
00:15:23,440 --> 00:15:25,880
So even if you go on the ChatGPT
app, you have the deep research 

291
00:15:25,880 --> 00:15:28,320
prompt. 
If you're a pro user, you have 

292
00:15:28,320 --> 00:15:30,640
the Google search, you now have 
study with me. 

293
00:15:30,760 --> 00:15:33,840
There's many different options. 
And what that does is 

294
00:15:33,840 --> 00:15:37,000
essentially give context to an 
agent that they built. 

295
00:15:37,000 --> 00:15:39,640
And it might just be a single 
agent architecture with a series

296
00:15:39,640 --> 00:15:42,120
of steps. 
It gives it context on how it 

297
00:15:42,120 --> 00:15:45,520
should behave in that situation.
Another big area is managing 

298
00:15:45,520 --> 00:15:47,960
context in multi tenant 
environments. 

299
00:15:48,200 --> 00:15:50,600
So many of our customers are 
technology providers, software 

300
00:15:50,600 --> 00:15:55,160
providers who have say thousands
of enterprise customers and each

301
00:15:55,160 --> 00:15:57,640
one of them have their own space
in these platforms. 

302
00:15:58,280 --> 00:16:01,880
How therefore does an agent or 
an AI retain conversations and 

303
00:16:01,880 --> 00:16:05,200
contacts between all these 
people and stay secure? 

304
00:16:05,360 --> 00:16:07,360
So it's an area we've been 
working really harder and we're 

305
00:16:07,360 --> 00:16:11,000
excited to bring some out for. 
But this is the area that's 

306
00:16:11,000 --> 00:16:14,960
really important because let's 
take a SAS platform in sales. 

307
00:16:14,960 --> 00:16:19,040
It might integrate with every 
single salesperson's contacts. 

308
00:16:19,040 --> 00:16:23,800
It's CRM, it's e-mail. 
So when a salesperson asks to 

309
00:16:23,800 --> 00:16:27,520
say, let's say draft an e-mail, 
it'll just use all of that data 

310
00:16:27,520 --> 00:16:31,440
seamlessly, very cleverly, 
magically and answer you. 

311
00:16:31,840 --> 00:16:36,080
But all of that magic actually 
had to be done quite securely. 

312
00:16:36,240 --> 00:16:38,480
I think the other one is going 
to be privacy and trust for you 

313
00:16:38,920 --> 00:16:43,160
because I would be worried about
what happens with all this data.

314
00:16:43,240 --> 00:16:44,960
They're capturing a lot of data 
on us now. 

315
00:16:45,240 --> 00:16:48,440
I personally use it as a life 
coach as ever. 

316
00:16:48,440 --> 00:16:50,760
I use it for almost any question
that I've got. 

317
00:16:50,920 --> 00:16:53,720
I will almost always go to 
ChatGPT or another AI system 

318
00:16:53,720 --> 00:17:06,650
first. 
So I guess let's just conclude, 

319
00:17:07,130 --> 00:17:10,450
early on we were obsessed with 
the model itself and how to talk

320
00:17:10,450 --> 00:17:13,480
to it with prompt engineering. 
And over time we've realized 

321
00:17:13,480 --> 00:17:16,680
that an AI agent isn't just the 
model, it's the whole system 

322
00:17:16,680 --> 00:17:20,480
that feeds it contacts and 
interpret its output and uses 

323
00:17:20,480 --> 00:17:23,400
that output to inform its next 
set of contacts. 

324
00:17:24,680 --> 00:17:27,560
So in a way, we're moving one 
level above in terms of 

325
00:17:27,560 --> 00:17:30,760
abstraction. 
How do I get the model to output

326
00:17:30,760 --> 00:17:33,920
X? 
Response is more about thinking 

327
00:17:34,080 --> 00:17:38,560
what information do I need to 
give it and guidance to get that

328
00:17:38,560 --> 00:17:41,480
best response. 
Anyway, I hope you enjoyed the 

329
00:17:41,480 --> 00:17:45,160
episode, I certainly did, and 
I'll see you next week.