1
00:00:00,040 --> 00:00:03,720
The absence of incidents doesn't
prove the effectiveness of 

2
00:00:03,720 --> 00:00:06,480
controls. 
Controls can degrade even while 

3
00:00:06,480 --> 00:00:09,840
an organisation is handing out 
awards for two years. 

4
00:00:09,960 --> 00:00:13,200
LTI Free Nancy Levinson 
describes the challenge. 

5
00:00:13,520 --> 00:00:16,720
There are always warning signs 
before a major accident, and 

6
00:00:16,920 --> 00:00:20,000
such weak signals are often 
perceived only as noise. 

7
00:00:20,000 --> 00:00:23,800
So how can we identify and 
measure those signals before the

8
00:00:23,800 --> 00:00:26,880
fire and brimstone? 
Everyone welcome to SAFE as I'm 

9
00:00:26,880 --> 00:00:30,080
Doctor Ben Hutchinson. 
Now the question I probably get 

10
00:00:30,080 --> 00:00:33,400
the most is what indicators 
should we be using? 

11
00:00:33,400 --> 00:00:37,320
I've covered dozens of studies 
on indicators in safety and 

12
00:00:37,320 --> 00:00:39,040
broader organisational 
indicators. 

13
00:00:39,600 --> 00:00:42,160
Check them out on my site. 
There's so much there to read. 

14
00:00:42,160 --> 00:00:44,960
Now today I'm going to try and 
sidestep that whole debate and 

15
00:00:45,160 --> 00:00:48,400
we're going to focus on one 
specific element of indicators, 

16
00:00:48,400 --> 00:00:52,080
focusing on the effectiveness of
our risk control system. 

17
00:00:52,080 --> 00:00:55,640
So there's extensive debate 
regarding indicator typologies, 

18
00:00:56,040 --> 00:01:00,080
lead verse lag, drive, verse 
monitoring, proactive verse 

19
00:01:00,080 --> 00:01:02,480
reactive, and more. 
Andrew Hopkins provides a more 

20
00:01:02,480 --> 00:01:04,720
focused thread. 
He notes that the primary 

21
00:01:04,720 --> 00:01:08,400
purpose of performance 
indicators must be to evaluate 

22
00:01:08,400 --> 00:01:11,960
the effectiveness of the 
specific controls upon which the

23
00:01:11,960 --> 00:01:15,160
risk control system relies. 
From this perspective, the 

24
00:01:15,160 --> 00:01:18,360
concern isn't the volume of 
audits or inspections completed,

25
00:01:18,360 --> 00:01:21,920
but whether the specific barrier
is designed to prevent a serious

26
00:01:21,920 --> 00:01:25,400
incident or unwanted event. 
A healthy step one. 

27
00:01:25,480 --> 00:01:30,360
The risk control system. 
The U KS HSG 254 guide focuses 

28
00:01:30,360 --> 00:01:33,240
on assurance and indicators for 
major hazards. 

29
00:01:33,240 --> 00:01:36,520
It assumes management systems 
are already in place and shifts 

30
00:01:36,520 --> 00:01:40,200
emphasis towards checking 
whether risk control systems are

31
00:01:40,200 --> 00:01:44,200
operating as intended. 
GC254's contribution is the 

32
00:01:44,200 --> 00:01:47,880
system of dual assurance. 
For every risk control system, 

33
00:01:48,120 --> 00:01:51,040
you require both a leading and a
lagging indicator. 

34
00:01:51,040 --> 00:01:54,400
The process involves 1. 
Identifying the risk control 

35
00:01:54,400 --> 00:01:56,640
systems and their required 
outcomes 2. 

36
00:01:56,640 --> 00:02:00,200
Setting lagging indicators for 
each system to monitor what can 

37
00:02:00,200 --> 00:02:03,640
go wrong, effectively monitoring
the failure of immediate causes 

38
00:02:03,640 --> 00:02:05,680
3. 
Identifying critical elements of

39
00:02:05,680 --> 00:02:09,600
each system, the specific 
components that must function 

40
00:02:09,680 --> 00:02:13,520
for the whole system to work 4. 
Setting leading indicators to 

41
00:02:13,520 --> 00:02:16,600
monitor the health of these 
critical elements Step 2 

42
00:02:16,600 --> 00:02:18,680
Defining performance and 
criticality. 

43
00:02:18,720 --> 00:02:22,160
Moving really into the heart of 
this logic, the ICM Ms critical 

44
00:02:22,160 --> 00:02:25,480
control management framework 
targets high consequence 

45
00:02:25,600 --> 00:02:28,160
material unwanted events. 
The goal is to define the 

46
00:02:28,160 --> 00:02:32,040
critical controls, objectives, 
performance requirements and how

47
00:02:32,040 --> 00:02:34,320
that performance is verified in 
practise. 

48
00:02:34,320 --> 00:02:37,360
Critical control is distinct 
from a standard or regular 

49
00:02:37,360 --> 00:02:39,160
control. 
It's crucial to preventing the 

50
00:02:39,160 --> 00:02:43,160
event and its absence or failure
would significantly increase 

51
00:02:43,160 --> 00:02:45,680
risk, despite other existing 
controls. 

52
00:02:45,680 --> 00:02:49,880
To ensure these work, the ICMM 
defines 3 specific performance 

53
00:02:49,880 --> 00:02:54,400
lenses. 1 Availability is the 
control in place and accessible 

54
00:02:54,440 --> 00:02:58,160
to reliability and the control 
performance function to the 

55
00:02:58,160 --> 00:03:01,920
required standard. 3 
Survivability will the control 

56
00:03:01,920 --> 00:03:03,960
function during the stresses of 
the event. 

57
00:03:04,000 --> 00:03:07,280
To bridge the gap between high 
level risk assessment and field 

58
00:03:07,280 --> 00:03:12,360
verification, both the ICMM and 
QGN 35 utilise a critical 

59
00:03:12,360 --> 00:03:14,800
control information summary. 
Here's a work example. 

60
00:03:15,280 --> 00:03:18,200
Methane ignition on a long wall.
So consider the material 

61
00:03:18,200 --> 00:03:19,920
unwanted event of a methane 
ignition. 

62
00:03:19,920 --> 00:03:22,880
Unidentified critical control is
the automatic methane power 

63
00:03:22,880 --> 00:03:24,480
trip. 
The objective is to 

64
00:03:24,480 --> 00:03:28,400
automatically isolate electrical
power to the long wall face when

65
00:03:28,400 --> 00:03:32,000
methane concentration exceed a 
certain threshold, preventing a 

66
00:03:32,000 --> 00:03:34,520
potential ignition source 
performance requirements. 

67
00:03:34,680 --> 00:03:37,840
Well, the trip must activate 
exactly that threshold and 

68
00:03:37,840 --> 00:03:40,640
complete the power isolation in 
less than two seconds or 

69
00:03:40,640 --> 00:03:43,880
whatever the criteria is. 
We have support activities and 

70
00:03:43,880 --> 00:03:46,640
these are the background tasks 
ensuring the control stays 

71
00:03:46,640 --> 00:03:49,720
healthy, such as regular sensor 
calibration and physical 

72
00:03:49,720 --> 00:03:52,400
protection of the sensor heads, 
and then the verification. 

73
00:03:52,440 --> 00:03:56,680
The indicators in part weekly in
situ gas testing, which is a 

74
00:03:56,680 --> 00:04:00,680
formal workplace monitoring and 
monthly software trip testing, a

75
00:04:00,680 --> 00:04:04,080
form of formal auditing, provide
the leading indicators of the 

76
00:04:04,080 --> 00:04:06,080
controls health. 
And there's probably a range of 

77
00:04:06,080 --> 00:04:08,840
other things you could do. 
And here's a potential shortcut.

78
00:04:09,040 --> 00:04:11,440
If you've already produced bow 
ties as part of your risk 

79
00:04:11,440 --> 00:04:15,000
management or critical control 
framework implementation, then 

80
00:04:15,000 --> 00:04:16,880
you've done most of the heavy 
lifting already. 

81
00:04:17,200 --> 00:04:20,959
Use your bow ties to guide your 
risk control system indicators. 

82
00:04:21,240 --> 00:04:24,400
Identify the ways to monitor and
verify the barriers or critical 

83
00:04:24,400 --> 00:04:27,680
controls and the support 
structures and the degradation 

84
00:04:27,680 --> 00:04:31,480
factors that either enable their
success or affect their 

85
00:04:31,480 --> 00:04:34,080
integrity. 
It doesn't need to be a gigantic

86
00:04:34,080 --> 00:04:36,640
list. 
Identify the few critical things

87
00:04:36,640 --> 00:04:40,480
that have to go right first 
time, every time, or just 

88
00:04:40,480 --> 00:04:43,640
otherwise critical and 
particularly susceptible to 

89
00:04:43,640 --> 00:04:45,920
reliability or erosion. 
Step three. 

90
00:04:45,920 --> 00:04:49,480
Condition verse compliance. 
This example illustrates Hopkins

91
00:04:49,480 --> 00:04:51,760
point. 
We must distinguish between 

92
00:04:51,760 --> 00:04:54,760
compliance and condition. 
Compliance could be like did we 

93
00:04:54,760 --> 00:04:57,880
finish the weekly gas testing 
and condition what was the 

94
00:04:57,880 --> 00:05:00,920
failure rate during those tests?
Indicators must report the 

95
00:05:01,040 --> 00:05:05,280
actual condition, the defect 
rate or trigger breaches, rather

96
00:05:05,280 --> 00:05:07,680
than just the activity of 
checking whether it was done or 

97
00:05:07,680 --> 00:05:10,000
not. 
For triggers and tarps, a 

98
00:05:10,000 --> 00:05:12,880
performance trigger is a point 
at which a control is considered

99
00:05:13,000 --> 00:05:15,280
not maintained. 
Trigger set the criteria for 

100
00:05:15,280 --> 00:05:18,720
what action must be taken, by 
whom and when. 

101
00:05:18,720 --> 00:05:23,120
So if that critical methane 
reading requires immediate 

102
00:05:23,120 --> 00:05:26,200
action, let's say it's 5% 
requires immediate action, but 

103
00:05:26,200 --> 00:05:30,320
the reading is 4.95%, what 
action should the operator take?

104
00:05:30,320 --> 00:05:32,240
This is where triggers and tarps
come into it. 

105
00:05:32,240 --> 00:05:36,160
So tarps are your if their 
mechanism if trigger level X, 

106
00:05:36,600 --> 00:05:40,920
action set Y, escalation Z. 
This is a real example where 

107
00:05:41,120 --> 00:05:44,760
let's say it was 5% and they 
measured it at 4.98%. 

108
00:05:44,760 --> 00:05:47,360
Some incidents have shown that 
people just didn't take action 

109
00:05:47,360 --> 00:05:50,160
because it's technically they 
only had to escalate the issue 

110
00:05:50,160 --> 00:05:51,480
if it was 5%. 
Let's say. 

111
00:05:51,560 --> 00:05:55,800
In summary, effective assurance 
requires moving past broad 

112
00:05:55,800 --> 00:05:58,600
organisational metrics to 
monitor the specific 

113
00:05:58,600 --> 00:06:02,320
performance, specifications, 
availability, reliability, and 

114
00:06:02,320 --> 00:06:06,800
survivability of the barriers we
rely on to prevent catastrophe. 

115
00:06:06,840 --> 00:06:10,320
You can still set broader 
measures and indicators, but 

116
00:06:10,320 --> 00:06:13,080
they shouldn't come at the 
expense of the effectiveness of 

117
00:06:13,080 --> 00:06:16,280
your risk control system and 
being laser focused on 

118
00:06:16,280 --> 00:06:20,120
understanding and monitoring the
effectiveness of your systems. 

119
00:06:20,120 --> 00:06:22,640
I highly recommend you check out
these sources that I've linked 

120
00:06:22,640 --> 00:06:25,840
today, including Nancy 
Levinson's work on indicators. 

121
00:06:25,840 --> 00:06:28,680
Maybe I'll cover that paper 
alone on another episode. 

122
00:06:29,160 --> 00:06:33,200
The short of it is identify the 
things that have to happen or 

123
00:06:33,200 --> 00:06:36,200
cannot happen, implement the 
controls or constraints and then

124
00:06:36,200 --> 00:06:39,360
design indicators to measure the
health of those constraints and 

125
00:06:39,360 --> 00:06:41,720
assumptions and controls. 
That's targeted. 

126
00:06:41,960 --> 00:06:44,600
It's focused on the things that 
are most important to your 

127
00:06:44,680 --> 00:06:47,640
organisation and its risk 
control systems and it doesn't 

128
00:06:47,640 --> 00:06:52,160
involve really lofty, pie in the
sky broad organisational 

129
00:06:52,160 --> 00:06:54,360
indicators. 
It's targeted on the things that

130
00:06:54,360 --> 00:06:57,440
have to happen or can't happen. 
That's it on safe as. 

131
00:06:57,480 --> 00:07:00,520
Thanks everyone. 
Please like and subscribe and 

132
00:07:00,520 --> 00:07:02,360
leave a comment directly on the 
video. 

133
00:07:02,480 --> 00:07:05,840
It really helps and link in show
notes to support the channel.