Measures

Targets on measures of targets on measures of things

giantknave Brain Farts , , , , ,

In this post I’m going to differentiate between:

  1. Measures of things
  2. Targets on (measures of things)
  3. Measures of (targets on (measures of things)); and
  4. Targets on (measures of (targets on (measures of things)))

Wow, that last one is hard to write, let alone say out loud! You might think that it’s a nonsense (which it is) but, sadly, it’s very common.

Note: I added the brackets to (hopefully) make really clear how each one builds on the last.

I’ll attempt to explain…

1. Measures of things:

Seems straight forward enough: I’m interested in better understanding a thing, so I’d like to measure it1.

Some examples…

A couple of personal ones:

  • What’s my (systolic) blood pressure level? or
  • How quickly do I ride my regular cycle route?

A couple of (deliberately) generic work ones:

  • how long does it take us to achieve a thing? or
  • how many things did we achieve over a given period?

Here’s a graph of a measure of a thing (in chronological order):

Nice, we can clearly see what’s going on. We achieved 13 things in week 1. Each thing took us anything between 2 and 36 days to achieve…and there’s lots of variation in-between.

It doesn’t surprise me that it varies2 – it would be weird if all 13 things took, say, exactly 19 days (unless this had been structurally designed into the system). There will likely be all sorts of reasons for the variation.

However, whilst I ‘get’ that there is (and always will be) variation, the graph allows us to think about the nature and degree of that variation: Does it vary more than we would expect/ can explain?3 Are there any repeating patterns? Unusual one-offs? (statistically relevant) Trends?

Such a review allows us to ask good questions, to investigate against and learn from.

“Every observation, numerical or otherwise, is subject to variation. Moreover, there is useful information in variation.” (Deming)

2. Targets on (measures of things):

Let’s say that we’ve been asked to achieve a certain (arbitrary4) target.

Here’s an arbitrary target of 30 days (the red line) set against our measure:

And here’s how we are doing against that target, with some visual ‘traffic lighting’ added:

Instance (X)12345678910111213
Target of 30 days met? (Yes/No)NYYNYYYYYYYNY

We’ve now turned a rich analogue signal into a dull digital ‘on/off’ switch.

If we only look at whether we met the target or not (red vs. green), then we can no longer see the detail that allowed us to ask the good questions.

  • We met ‘target’ for instances 2 and 3…but the measures for each were quite different
  • Conversely, we met ‘target’ for instances 5 all the way through to 11 and then ‘suddenly’ we didn’t…which would likely make us think to intensely question instance 12 (and yet not see, let alone ponder, the variation between 5 and 11).

The target is causing us to ask the wrong questions5, and miss asking the right ones.

3. Measures of (targets on (measures of things)):

But I’m a fan of measures! So, let’s show a measure over time of how we are doing against our target.

In week 1 we met our 30-day target for 10 out of our 13 instances, which is 77%. Sounds pretty good!

Here’s a table showing how many times we met target for each of the next five weeks:

Week12345
Things achieved1315141112
Number meeting 30-day target10141278
% meeting  30-day target77%93%86%64%67%

Let’s graph that:

It looks like we’ve created a useful graph, just like in point 1.

But we would be fooling ourselves – we are measuring the movement of the dumbed-down ‘yes/no’ digital switch, not the actual signal. The information has been stripped out.

For example: There might have been huge turbulence in our measure of things in, say, week 3 whilst there might have been very little variation in week 4 (with lots of things only just missing our arbitrary ‘target’)…we can’t see this but (if we want to understand) it would be important to know – we are blind but we think we can see.

4. Targets on (measures of (targets on (measures of things))):

And so, we get to the final iteration:

How about setting an arbitrary target on the proportion of things meeting our arbitrary target…such as achieving things in 30 days for 80% of the time (the red line)…

And here’s the table showing how we are doing against that target:

Week number:12345
80% Target on 30-day Target met?NYYNN

Which is a double-dumbing down!

We’ve now got absolutely no clue as to what is actually going on!!!

But (and this is much worse) we ‘think’ we are looking at important measures and (are asked to) conclude things from this.

The table (seemingly) tells us that we didn’t do well in week’s 1, 4 and 5, but we did in week’s 2 and 3…

The base data series used for this example:

In order to write this post, I used the Microsoft Excel random number generator function. I asked it to generate a set of (65) random numbers between 1 and 40 and then I broke these down into imaginary weeks. All the analysis above was on pure randomness.

Here’s what the individual values look like when graphed over time:

(Noting that instances 1 – 13 are as per the graph at point 1, albeit squashed together)

Some key points:

  • There is nothing special about any of the individual data points
  • The 30-day target has got nothing to do with the data
  • There is nothing special about any of the five (made up) weeks within
  • The 80% target on the 30-day target has got nothing to do with anything!

The point: Whilst I would want to throw away all the ‘targets’, ‘measures of target’ and ‘targets on measures of target’…I would like to understand the system and why it varies.

This is where our chance of improving the system is, NOT in the traditional measures.

Our reality:

You might be laughing at the above, and thinking how silly the journey is that I’ve taken you on…

…but, the ‘targets on (measures of (targets on (measures of things)))’ thing is real and all around us.

  • 80% of calls answered within 20 seconds
  • 95% of patients discharged from the Emergency department within 4 hours
  • 70% of files closed within a month
  • [look for and add your own]
  • …

Starting from a position of targets and working backwards:

If you’ve got a target and I take it away from you…

…but I still ask you “so tell me, how is [the thing] performing?” then what do you need to do to answer?

Well, you would now need to ponder how has the thing been performing – you would then need to look at a valid measure of a thing over time and ponder what this shows.

In a nutshell: If you’ve got a target, take it away BUT still ask yourself ”how are we doing?”

A likely challenge: “But it’s hard!”

Yes… if you peel back the layers of the ‘targets on targets’ onion so that you get back to the core of what’s actually going on, then you could be faced with lots of data.

I see the (incorrect) target approach as trying to simplify what is being looked at so that it looks easy to deal with. But, in making it look ‘easy to deal with’, we mustn’t destroy the value within the data.

“Everything should be made as simple as possible, but no simpler.” (attributed to Einstein)

The right approach, when faced with a great deal of data, would be to:

  • Look at it in ways that uncover the potential ‘secrets’ within (such as in a histogram, in a time-series plot); and
  • understand how to disaggregate the data, such that we can split it up into meaningful sub-groups. We can then:
    • compare sub-groups to consider if and how they differ; and
    • look at what’s happening within each sub-group (i.e. comparing apples with apples)

To close:

If you are involved in ‘data analysis’ for management, I don’t think your role should be about ‘providing the simple (often 1-page) picture that they’ve asked for’. I would expect that you would wish your profession to be along the lines of ‘how can I clearly show what’s happening and what this means?’

If you are a manager looking at measures: why would you want an (overly) simple picture so that you can review it quickly and then move on to making decisions? Wouldn’t you rather understand what is happening and why … so that good decisions can be made?

Footnotes

1. Measurement of things – a caution: We should be careful not to fall into the trap of thinking that everything is measurable or, if we aren’t measuring it, then it doesn’t matter.

There’s plenty of stuff that we know is really important even though we might not be measuring it.

2. Variation: If you’d like to understand this point, then please read some of my earlier posts, such as ‘The Spice of Life’ and ‘Falling into that trap’

As a simple example: If you took a regular reading of your resting heart rate, don’t you think it would be weird if you got, say, 67 beats per minute every single time? You’d think that you’d turned into some sort of android!

3. Expect/ can explain – clarification: this is NOT the same as ‘what we would like it to be’.

4. Arbitrary: When a numeric target is set, it is arbitrary as to which number was picked. Sure, it might have been picked with reference to something (such as 10% better than average, or the highest we’ve ever achieved, or….) but it’s arbitrary as to which ‘reference’ you choose.

5. Wrong questions: These wrong questions are then likely to cause us to jump to wrong conclusions and actions (also known as tampering). Such actions are likely to focus on individuals, rather than the system that they work within.

6. ‘Trigger’: The writing of this post was ‘triggered’ the other day when I reviewed a table of traffic-lighted (i.e. against a target) measures of targets on measures of things.

Inspector Clouseau

giantknave Brain Farts , , , ,

inspector-clouseauSo, a bloody good mate of mine manages a team that delivers an important public service – let’s call him Charlie. We have a weekly coffee after a challenging MAMIL 1 bike ride….and we thoroughly explore our (comedic) working lives.

And so to our most recent conversation – Charlie told me that his department is due an inspection (a public sector reality)…and how much he, ahem, ‘loves’ such things! 🙂

“Oh, and why’s that?” I say.

His response2 went something like this: “Well, they come in with a standardised checklist of stuff, perform interviews and site visits to tick off against a set of targets, and then issue a report with our score, and recommendations as to what we should be doing….it’s not exactly motivational!”

“Mmm” I say…”but presumably their intent – to understand how well things are going – is good?”

Here’s the essence of Charlie’s reply: “Yep, I ‘get’ the intent behind many of the items on their checklist, and I accept that knowing how we are doing is really important…but, rather than act like robotic examiners, I want them to:

  • understand us, and our reality (what we are having to deal with3)…and I want them to focus on what really matters in respect of our service, not hide behind narrow ‘currently trendy’ targets and initiatives set from above;
  • give us the chance to demonstrate:
    • how we are doing;
    • where we know we have room for improvement; and
    • what we are doing to get better; and finally 

  • assist us, by adding value (perhaps with useful references to what they’ve seen working elsewhere) rather than be seen as ‘taking up our time’. “

“Righto”, I said ”…that sounds excellent! I’ll write a post on that lot”…so here goes:

A ‘short but sharp’ generic critique of inspections:

inspectionInspection requires someone looking for something (whether positive or negative)…and this comes from a specification as to what they believe you should be doing and/or how you should be doing it.

The ‘compliance’ word fits here.

…and, as such, the inputs, behaviours and outcomes from inspections are rather predictable. You can expect some or all of the following:

  • people, often (usually?) from outside your service, spend time writing (often inflexible) specifications as to what you should (and should not) be doing;
  • people are employed, and trained, as inspectors of those specifications;
  • you and your team spend precious time preparing before each imminent inspection:
    • running preparatory meetings to guess what might happen;
    • window-dressing solely for the benefit of ‘the inspector’ e.g. making your work space look temporarily good, filling in (and perhaps even back dating) ‘paperwork’;
    • even performing ‘dummy runs’ (rehearsals!);
  • you and your team serenade the inspector around during their visit, with everyone on their best behaviour;
  • questions are asked, careful (guarded) answers are given, and an inspection report is issued;
  • your post-inspection time is then consumed rebutting (what you consider to be) poorly drawn recommendations and/or drafting action plans, stating what is going to be implemented to comply.

…and after it’s all over, a big sigh is let out, and you go back to how you were.

Wouldn’t it be great if, rather than playing the ‘inspection game show’, you truly welcomed someone (anyone) coming in to see what you actually do and, when they arrive, you carried on as normal because you are confident that:

  • you are operating in the way that you currently believe to be the best; and
  • you want them to see and understand this, and yet provide you with feedback that you can ponder, experiment with, and get even better at delivering against your purpose4.

…so how might you get to this wonderland?

A better way:

just-one-questionI’m a huge fan of a (deceptively) simple yet (potentially) revolutionary idea put forward by John Seddon:

“Instead of being measured on compliance, people should be assessed on whether they are able to show that they are working to understand and improve the work they do.

It is to shift from ‘extrinsic’ motivation (carrot and stick) to intrinsic motivation (pride), which is a far more powerful source of motivation.”

…and to the crux of what Seddon is suggesting:

“Inspection of performance should be concerned with asking only one question of managers:

‘What measures are you using to help you understand and improve the work?’ “

This, to me, is superb – the inspector doesn’t arrive with a detailed checklist and ‘cookie cutter’ answers to be complied with; and the manager (and his/her team) has to really think about that question!

To answer it, the team must become clear on:

  • the (true) purpose of the service that they provide;
  • what measures5 would tell them how they are doing against this purpose (i.e. their capability);
  • how they are doing against purpose (i.e. as well as knowing what to measure, they must be actively, and appropriately, measuring it for themselves);
  • what they are working on to improve, and how these are affecting the performance of their system; and
  • …what fresh ideas have arisen to experiment with.

You can see that this isn’t something that is simply ‘prepared in advance’ for a point-in-time inspection. It is an ongoing, and ever maturing, endeavour – a way of working.

It means that any inspector (or interested party) can arrive at any time and explore the above in use (as opposed to it being beautifully presented in ‘this years’ audit file ring-binder)

 “Are you saying that ‘specifications’ are wrong then?”

i-love-to-clarifyBefore ending this post, I’d like to clarify that:

  • No, I’m not saying that specifications are (necessarily) wrong; and
  • I’m also not saying that scientific know-how, generated from a great deal of experience over time should be ridiculed or ignored ‘just because it comes from somewhere else’.

Taking each in turn,

  • Specifications (e.g. the current best known way to perform a task) should be owned by the team that have to perform them…and these should be:
    • of adequate depth and breadth to enable anyone and everyone to professionally perform their roles;
    • suitably flexible to cater for the variety of demands placed upon them (requiring principled guidelines rather than concrete rules); and
    • ‘living’ i.e. continuously improved as new learning occurs6

  • If there is a central function, then their role should be to:

    • understand what is working ‘out there’; and
    • effectively share that information with everyone else (thus being of great value to managers)

…without dictating that a specific method should be ‘complied with’.

The point being that we should not think in terms of ‘best practise’…we should be continuously looking for, and experimenting with, better practise, suited to each scenario. This is to remove the (attempted) authority from the centre, and place it as the (necessary) responsibility where the actual work is performed, by the service (manager, and team) on the front line.

In this way, ‘the centre’ can shift itself from being seen as an interfering, bureaucratic and distant police force, to a much valued support service.

To Ponder:

If you are ‘being inspected’ then, yes, I ‘get’ that you currently need to ‘tick those boxes’…but how about thinking a little bit differently:

…how would you show those ‘inspecting you’ that you truly understand, and are improving, your system against its purpose?

You could seriously surprise them!

If you can really answer that one question, then you are likely to be operating a stable, yet continually improving service within a healthy environment, both for your team and those they serve.

Who knows – ‘the inspector’ might want to share what you are doing with everyone else 🙂

And, going back to the top – i.e. Charlie’s reply as to what he really wanted out of an inspection – I reckon he was ‘right on the money’!

A final comment…for all you private sector organisations out there:

Don’t think that this post doesn’t apply to you!

If you have centralised ‘Audit’, ‘Quality Assurance’ and/or ‘Business Performance’ teams (i.e. that are separate from the actual work), then virtually everything written above applies to your organisation.

Footnotes:

1. MAMIL: Middle aged men in lycra

2. To ‘Charlie’ – Please excuse the poetic licence that I have taken in writing the above…and I hope it may be of some use (woof woof 🙂 ).

3. What we are really dealing with: I picked the ‘Inspector Clouseau’ picture to allude to the possibility (probability?) that many an inspector, stuck with their heads in their audit checklist, hasn’t a clue about what is really going on within, and/or what really matters for, the service before them….and for some, this is still true AFTER the inspection has been completed 😦

I’m not trying to ‘shoot at’ inspectors – this is a role that you have (currently) been given. You might also like to ponder the above and thereby look to re-imagine your purpose. Doing so could dramatically improve your work satisfaction…and help improve the services that you support.

4. Purpose: not to be confused with the lottery of attempting to meet a numeric target.

5. A set of Measures that uncover how the system is performing, NOT one supposedly ‘all seeing’ KPI and an associated target.

6. Living: Years of (regularly futile) experience have proven to me that ‘learning aids’ (whether they be documents, diagrams, charts, pictures….) will only ‘live’ (i.e. improve) if they are regularly (i.e. necessarily) used by the workers to do the work. An earlier post (Déjà vu) fits here.

7. I think that a couple of previous posts are foundational and/or complimentary to this one:

The Principle of Mission: That clarification of intent, and allowing flexibility in how it is achieved, is far more important than waiting for, and slavishly carrying out ‘instructions’ from above.

Rolling, rolling, rolling: The huge, and game-changing difference between rolling out and rolling in change. One is static, the other is dynamic and purpose-seeking.

Slaughtering the ‘Sacred Cow’

giantknave Brain Farts , , , , , ,

moo-cowI’ve written enough posts now to ‘write a book’ 🙂 …so it’s about time I dealt with a seemingly sacred cow – the ‘Balanced Scorecard’.

Context

First, I’ll delve into a bit of history…

Robert Kaplan and David Norton performed a research project back in 1990 in respect of measuring organisational performance.

It was based on the premise that:

  • An organisation’s knowledge-based assets1 were becoming increasingly important;
  • The primary measurement system remained2 the financial accounting system; and
  • Executives and employees pay attention to what they measure and, therefore, were overly focused on the (short term) financials and insufficiently on the (longer term) intangible assets.

balanced-scorecardThe outcome of their research project was the concept of a Balanced Scorecard of measurements (and, of course, the accompanying Harvard Business School (HBS) management book).

This retained the organisation’s financial measures (as historic results) but added three additional perspectives:

  • Customer;
  • Internal Business Processes; and
  • Learning & Growth.

The last two were said to represent the lead indicators of future financial performance.

The Balanced Scorecard quickly gained traction in many corporations. This was helped by many a ‘big consultancy’ cashing in3 on the lucrative ‘implementation’ revenue stream.

Version 2.0

Over a decade later (2004) Kaplan and Norton then took things further by linking strategy formulation and execution to their measurement ideas and came up with the Strategy Map concept (and, you’ve guessed it…an accompanying HBS management book). I imagine that this was for two reasons:

1. They saw some improvements to/ holes in the original idea;

…and with my cynical hat sat jauntily on my head…

2. They now had an adoring following that would buy the sequel which, as ever, sets out:

– the big idea in detail;

– a set of carefully curated case studies; and

– instructions on how to implement ‘the big idea’ in (on?) your organisation

strategy-mapThe ‘Strategy Map’ turned the four quadrants of the balanced scorecard into a linear cause-effect view (see picture)

The idea went that the desired financial outcomes would be stated at the top, which would then be achieved by reverse engineering down the strategy map to the bottom.

Thus, through setting objectives from top down to bottom and using measures, targets and action plans (involving initiatives with business cases and budgets), the desired outcome could be achieved.

Wow, that all looks really cool – neat looking and oh-so-complete! Doesn’t it?

So why the ‘Sacred Cow’ reference?

Well, many (most?) organisations feverishly adopted the Balanced Scorecard/ Strategy Map tools and technique as if it were common sense. Indeed, some 20 years later, it has become ‘part of the management furniture’. Unquestioned…even unquestionable.

However, I believe that there are a number of serious problems within…so let’s consider whether that proverbial sacred cow deserves to be slaughtered…

There are two angles that I could come at it from:

  1. The thinking within the Balanced Scorecard/Strategy Map logic; and
  2. How organisations typically implement these ‘big ideas’.

It would be too easy to shoot at how organisations typically implement them (i.e. how they might have bastardised it4)…and you could easily accuse me of ‘cheap shots’, saying that these aren’t Kaplan and Norton’s fault. So, instead, I’ll critique the foundational logic using four headings.

Here goes…

1. Measurement:

The foundation of Kaplan and Norton’s logic is that we must have measures if we are to manage something…and this is regarded as conventional wisdom…but here’s a counter-quote from W. Edwards Deming to ponder:

“Of course visible figures are important but he that would run his company on visible figures alone will in time have neither company nor figures. The most important figures are unknown and unknowable but successful management must nevertheless take account of them.”

His point is that we seem to be obsessed with trying to measure the effect of a given change (usually to ‘claim it’ for some recognition or even reward), but that we cannot accurately do so…and it is a mistake to think that we can. Sure, we can likely determine whether a change is having a positive or negative effect on the system (and thereby try to amplify or dampen it) but we cannot isolate the change from everything else going on (internally or externally; occurring right now, previously or in the future)

Deming went on to provide some examples of ‘important but unknowable’:

  • The multiplying effect on sales that comes from a happy customer, and the opposite from an unhappy one;
  • The improvement of quality and productivity from teamwork (across the horizontal value stream and with suppliers);
  • The boost in quality and productivity all along a value stream from an improvement at any activity upstream;
  • The loss from the annual rating of people’s performance (the time taken by everyone to perform this process and, of far greater concern, the resulting de-motivation and relational damage caused)
  • …and so on

Deming famously wrote that “it is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth.”

feedback-cartoonExample: Can I manage how employees feel? Yes, by how I behave.

Should I become obsessed with measuring employee feeling through those dreaded culture surveys? No!!!!

…just continue to manage how people feel – by constantly and consistently applying simple philosophies such as the most excellent “Humanity above Bureaucracy” (Buurtzorg).

Leave the constant crappy ‘surveying of the obvious’ to those organisations that (still) don’t get it.

The balanced scorecard was derived because of the major limitations of purely financial measures. However, we should not assume that such a tool is a definitive answer for what we need to manage.

Indeed, it causes damaging behaviours – with management wearing blinkers when focusing on the scorecard “because we’ve tied all our management instruments into it and therefore that’s all that counts round here.”

The highly limited and ‘helicopter view’ scorecard becomes a major part of the ‘wrong management system’ problem.

2. Balance:

This word is used as if we need to balance our focus on the four different quadrants, playing one off against the others as if they are counterbalances to keep in check.

But this isn’t the case. If we did a little bit of, say, learning and growth (e.g. developing our people) and/or customer focus but then said “whoa…steady on, not too much…we need to balance the financials” then we aren’t understanding the nature of the system….and we certainly don’t ‘get’ cause and effect.

cause-and-effect

A metaphor for business to help explain the point:

Let’s suppose that you keep breaking out in a nasty skin rash.

You could pour ice cold water on it, apply a lotion or scratch it…until it bleeds (ouch).

These actions might appear to alleviate the effects…but they are also likely to make things worse…and none of them have considered (let alone dealt with) the cause!

If you continue to ignore the cause and just treat the (currently visible) effects, things could escalate…with new effects presenting…complicating any necessary treatments…causing long lasting or permanent damage…and even death.

If you want to get rid of the rash…and keep it that way (and perhaps even improve your skin complexion and wider health)…then you need to focus your attention on its cause:

  • are you reacting to something you are putting on your skin?
  • what about something you eat, drink or otherwise introduce into your body?
  • maybe it’s something else more complicated?

And once you’ve worked out the likely cause(s) then you need to do something about it.

You work on the cause (such as stop using that brand of sun cream or stop eating shell fish or…stop injecting heroin!!) whilst checking whether it is working by observing the effect (what the likes of Seddon and Johnson would refer to as ‘keeping the score’).

You don’t think “mmm, I’ll balance the cause and the effect”…because you understand the glaringly obvious definitions behind the words ‘cause’ and ‘effect’

“Cause: A person or thing that gives rise to an action, phenomenon, or condition

Effect: A change which is a result or consequence of an action or other cause.” (Oxford Dictionary)

Okay, back to that Balanced Scorecard/Strategy map thingy and a cause – effect journey:

  • mgmt-cause-and-effectSenior Management’s beliefs and behaviours determine (i.e. cause) the management system that they choose to put into effect and (often stubbornly) retain;
  • The management system creates (i.e. causes) much of the environment that the people work within (effect);
  • The work environment is the foundation of (i.e. causes) how people act and react whilst doing their jobs (e.g. whether they are engaged, innovative, intrinsically motivated…or not);
  • How people act influences (i.e. causes) how processes are operated and the nature, size and speed of their evolution (whether by continuous or breakthrough improvements);
  • How processes operate and improve creates (i.e. causes) the outcomes that customers experience…and tell other potential customers about (i.e. as advocates or detractors);
  • Customers (whether they buy from, and advocate for us or ignore, avoid and slag us off) determine (i.e. cause) whether we stay in business.

The bl00dy obvious point is that THE FINANCIALS ARE THE EFFECT! So why are we so focused on them, other than to keep the score5.

…or, in a short, snappy sentence: This isn’t something to be BALANCED!!!!!!!!

The ‘balanced’ word keeps people tied to a ‘manage by results’ mentality, rather than managing the causes of the results such that the results then look after themselves.

What winds me up even more than the balanced bit is….wait for it…applying % weightings on the four quadrants5….usually with the financials (yes, the effect) getting the lions share!

That’s like saying “We’ll focus 75% on scratching the rash but only 25% on taking fewer heroin injections”. Aaaargh!!!

Now, you might respond to me by saying you believe that Kaplan and Norton understood the problem with the ‘balanced’ word…which is why they, ahem, ‘refreshed’ their logic with their ‘Strategy Maps’ book.

The problem with this is that they didn’t attack the results thinking, they merely added to it and, as such, many (most?) organisations continue with balancing and weighting…and spectacularly missing the point.

3. Key Performance Indicators vs. Capability:

kpi-statusOkay – let’s suppose that senior management accept that measures aren’t everything and that we shouldn’t be balancing (let alone weighting) things – I hope that we can all agree that some “right measures, measured right” (Inspector Guilfoyle) are going to be very useful…

…and so to the next whopper problem – the “measured right” bit.

Nothing (that I have seen) within the Balanced Scorecard/ Strategy Map logic reflects on, let alone deals with, the hugely important subject of variation and the need to always visualise measures over time.

Management simply use a set of KPIs on a ‘scorecard’ and look at their red down/ green up arrows against last period and/or their traffic lights against budget.

This is to completely ignore the dynamics of a system, and whether such movements are predictable or not….and therefore whether any special attention should be paid to them.

The Balanced Scorecard/Strategy Map approach can therefore create a set of Executives exhibiting the ‘God complex’ (as in “I have the answer!”) whilst being “fooled by randomness” (Taleb) – blissfully ignorant of the capability of their value streams (or processes within) and doing much damage by tampering.

and last, but by no means least…

4. Strategy vs. Purpose:

The underlying assumptions within the Balanced Scorecard/Strategy Map thinking would appear to be the conventional ‘shareholder value’ view of the world.

(I’ve previously written a 5-part serialised post on what I think about this….so I won’t repeat this here)

We get fed a feast of:

In short: The core problem (for me) with Kaplan and Norton’s two books is that, not only do they retain the problematic traditional command and control management system, focused on delivering shareholder value – they use it as their foundation to build upon.

It’s therefore no wonder that organisations carry on as before (doing the same crappy stuff), whilst waving their supposedly game-changing ‘Strategy Map’ around a lot.

Have you got hold of that cow? Good…now where’s my ceremonial knife?

To end: ‘having a go’ at me because I’m being so negative

You might shout back “okay you cynic…what would you do instead?!”

Well, I’m not going to be able to answer that in a paragraph – even Kaplan and Norton took two (rather verbose) books…and more than a decade in-between…to present their logic – but I’d suggest that, if you are curious, the 130+ posts on this site would go some way to expressing what I (and I believe my giants) think.

…and if you want to start at measurement then you might want to look here first.

Footnotes:

1. Knowledge based assets: Kaplan and Norton list the following as examples of assets that aren’t measured and managed by financial measures: employee capabilities, databases, information systems, customer relationships, quality, responsive processes, innovative products and services.

2. Measurement system remaining financially based: H. Thomas Johnson’s book ‘Relevance Regained’ makes clear that it wasn’t always so. Financial measures used as operational measures (a bad idea) only came into being from the 1950s onwards. Johnson refers to the period 1950s – 1980s as the ‘Dark Age of Relevance Lost’ and ‘Management by Remote Control’. I would argue that many an organisation hasn’t exited this period.

3. Big consultancies ‘cashing in’: I can (sadly) write this because I have first hand evidence – I was there! 😦

4. Bastardising the Strategy Map includes organisations changing the order of the four elements!!!

5. Financials: There’s a HUGE difference between a) using financial measures to keep the score (which would be good governance) and b) attempting to use them to make operational decisions! Using financials to make operational decisions is to attempt to ‘make the tail wag the dog’.

Yes, accountants should keep the score, for cash flow monitoring and assisting with longer term investment decisions…but accountants should not be attempting ‘remote control management’ of operations.

6. Weighting the elements of the scorecard: See, for example, fig. 9.8 in ‘The Balanced Scorecard’ (1996) and the related commentary.

7. Diversity: I understand that the cow is a holy animal to some. Please don’t be offended by my use of an English phrase in expressing my thinking – no real cows were harmed in the writing of this post…and no harm is intended to those living now, or in the future 🙂

 

“Sir, Sir, Sir…have you marked it yet?!”

giantknave Brain Farts , , , , , ,

class with hands upSo my son had some school exams and this post was triggered from a conversation I had with him just afterwards:

I expect all of you can cast your minds back to school and if you’ve got teenagers then, like me, you will also be sharing their experiences.

Picture the following scenario:

  • You’ve studied for, let’s say, a maths exam1;
  • You’ve spent 2 long hours sat on an uncomfortable school chair, whilst being watched by the beady eyes of the maths teacher (who was actually asleep), and have just emerged from the exam hall;
  • You and your mates fall straight into discussing the trauma that you’ve just been through:

“What did you put for question 4?”

“Oh [beep], I hadn’t realised it was about that! I wrote about [something else that was completely irrelevant to the question]”

“Could you work out the pattern in that sequence of numbers?…’Fibonacci’ who?”

“What do you mean there were more questions over the page?!!!”

…and so on.

What you will notice is that they are all ‘switched on’ in the moment, whether they ‘enjoyed’ the exam or not. They really want to know what the answers were and how they did against them!

The after’math’ 🙂

So, next day, they have double-maths…whoopee!

The Students all plead together: “Sir, Sir, Sir…have you marked our exam yet?”

Teacher: “Whoa, hold your horses, I’ve barely sat down! I’ll do it as soon as I can.”

…and the students engage in yet more chatter about the exam but their memory of the exam is beginning to fade.

At the end of the week, they have maths again:

The majority of Students: “Sir, Sir, Sir…have you marked our exam yet?”

Teacher: “No, not yet, I’ll do it over the weekend.”

…much less chatter now. They have forgotten most of it.

So, now it’s the following week and maths:

A few keen Students: “Sir, have you marked our exam yet?”

Teacher: “Sorry, no, I’ve been writing reports so I haven’t got around to it yet. I’ll definitely do it by the end of this week.”

…the mood has changed. The content of the exam has been forgotten and so, instead, they fall back to merely wanting to know a score.

End of week 2 maths lesson:

One diligent Student: “Sir, have you marked our exam yet?”

Teacher: “Yes I have! I’ll read out the marks” and the marks are duly read out to the class, which brings out the whole spectrum of emotions (from feelings of elation to tears of despair, with a healthy dose of indifference in between).

That diligent Student again: “…but Sir, can I have my marked exam paper back?”

Teacher: “Erm, yes…I haven’t got them with me now…I’ll bring them in next week.”

What do we think about this?

We all know that by far the best thing to do for effective learning to take place is to mark this exam, get the marked papers back to the students and then go through the paper to explain and then discuss it question-by-question…and to do all of this As Soon As Possible.

(… and I know that this is what all good teachers will try to do)

We can see that:

  • There is a human desire for immediate and meaningful feedback, which quickly dissipates over time;
  • An overall score (the result), whilst potentially providing some useful indicative data, cannot help with learning – you can feel emotions from receiving a score but you can’t improve. Instead, you need to know about the method (or, in this exam scenario, each question);

“We don’t learn from our mistakes, we learn from thinking about our mistakes” (Ralph Tyler, Educator)

  • There is little point in just the teacher knowing the current capability of each of their students. Each student should be very clear on this for themselves.

So, to organisations:

The above might seem blindingly obvious and a world away from work but every day we all carry out actions and interactions within value-streams for the good of our customers…and the usual buzz phrase uttered at regular intervals is ‘we want to continuously improve!‘…but do we provide ourselves with what we need to do so?

Think of the richly varied units of customer demand that we* strive to satisfy as analogous to the maths exam:

  • (how) do we all know how we (really) did?
  • (how) do we find this out quickly?
  • (how) do we know what specifically went well and what didn’t?
  • …and thus, (how) can we learn where to experiment and how this went?!

(* where ‘we’ refers to the complete team along the horizontal value stream)

There’s not much point in senior managers receiving a report at the end of the month that provides them with activity measures against targets and some misleading up/down arrows or traffic light colouring. Very little learning is going to occur from this…and, worse, perhaps quite a bit of damage!

…and when I say learning, I hope you understand that I am referring to meaningful changes being made that improve the effectiveness of the value stream at the gemba.

The value-creating people ‘at the gemba’:

The people who need the (relevant) measures are the people who manage and perform the work with, and for, the customer.

If the people who do the work don’t know how they are truly doing from the customer’s point of view then they are no different from the students who don’t have their marked exam papers back.

hamster wheelThere should be no surprise if the workers are merely clocking in, turning the wheel, collecting their pay and going home again. It’s what people end up doing when they are kept in the dark….though they likely didn’t start out like this!

Senior Management may respond with “but we regularly hold meetings/ send out communications to share our financial results with them, and how they are doing against budget!”

  • This gives people the wrong message! If you lead with, and constantly point at, the financials, you are telling people that the purpose of the system is profit, and NOT your stated ‘customer centric’ purpose;
  • You can’t manage by financial results. This is an outcome – ‘read only’. You have to look at the causes of the results – the operational measures;

To repeat a hugely important John Seddon quote:

“Use operational measures to manage, and financial measures to keep the score”

I am championing what may be termed as ‘visual management’: being able to easily see and understand what is happening, in customer terms, where the work is done.

A whopping big caution

caution signHowever, ‘visual management’ should have a whopping big warning message plastered all over its box, that people would have to read before undoing the clasps and pushing back the lid…because visual management works for whatever you put up on the wall!

If you put up a visual display of how many calls are waiting or how long your current call has taken or a league table of how many sales each member of your team has made or….etc. etc. etc. people WILL see it and WILL react….and you won’t like the dysfunctional behaviours that they feel compelled to engage in!

So, rather than posting activity measures and people’s performance comparisons, what do the value creating people need to know? Well, put simply, they need to know how their system is operating over time, towards its purpose.

Here’s what John Seddon says about the operational measures that should be “integrated with the work: In other words they must be in the hands of the people who do the work. This is a prerequisite for the development of knowledge and, hence, improvement.

  • Flow: what is the capability of the system to handle demands in one-stop transactions? Where a customer demand needs to go through a flow, what is the capability of that flow, measured in customer terms? 

…in both cases we need to know the extent of variation – by revealing variation we invite questioning of its causes. By acting on2 the causes, we improve performance.”

A final thought: This blog has often said “don’t copy manufacturing because Service is different!“ But gemba walks through a well run ‘Lean thinking’ factory floor may very well assist your understanding of what is meant by good visual management. No, I’m not saying ‘copy what you see’…I’m suggesting that you might understand how a well run value stream has a physical place alongside the gemba where its participants gather and collaborate against a background of what they are currently achieving (their current condition) and what experiments they are working on to improve towards some future target condition.

To close – A shameless segue:

So I’ve been writing this blog for nearly 2 years…and I know many people read it…but I don’t get much feedback3.

If you have read, and accept the thinking within this post, you will understand that this limited feedback ensures that I am somewhat ‘in the dark’ as to how useful my writings are for you.

I do know that people see/ open my posts…but I don’t know too much more:

  • you might read the title or first few lines of a post, yawn, and go and do something else;
  • you might get half way through and not understand what on earth I am rambling on about;
  • you might read to the end and violently disagree with some or all of what I’ve written;

but…and here’s the punch line, how would I know? 🙂

Notes:

  1. It’s clearly a totally separate, and MUCH bigger question as to whether taking exams is good for learning – I’m aware that many educators think otherwise. The genesis of this post merely comes from my son’s exam reality. Just for clarity: I’m not a fan of the ‘top-down standards and constant testing’ movement.  
  2. Seddon writes ‘acting on’, NOT ‘removing’ the causes of variation. The aim is not to standardise demand in a service offering…because you will fail: the customer comes in ‘customer shaped’. The aim is to understand each customer’s nominal value and absorb it within your system as best you can…and continue to experiment with, and improve how you can do this.
  3. A big thanks to those of you that do provide me with feedback!….and I’m most definitely not criticising those that don’t comment – I’m just saying that I have a very limited view on how I am performing against my purpose…just like many (most?) people within their daily work lives.

 

What have the Romans ever done for us!!

giantknave Brain Farts , , , , , , ,

Biggus DicusFor those of you Python fans out there, I suspect the title of this post draws a smile of recollection from you. It draws out a big hearty grin from me.

For those of you who don’t know what I am writing about (and for those who do…but would like to relive the moment – go on, you know you want to!), here’s the famous clip from the Monty Python film ‘The Life of Brian’:

What have the Romans… (1 min. 25 secs)

This clip was triggered in my mind the other day when pondering how people collect and use data in reports (I had just seen one that offended my sensibilities!). I get frustrated when I point out a serious fault within a report and the response I get is “yes, but apart from that….”

Here’s my attempt at a Python-like response:

Leader (John Cleese): “Look at what this report is telling us!”

Minion 1: “…but we don’t have enough data to know what’s actually happening.”

John Cleese: “What?”

Minion 1: “We are only using a couple of data points to compare. This tells us virtually nothing and is likely to be highly misleading.”

John Cleese: “Oh. Yeah, yeah. We have only got this month vs. last month. Uh, that’s true. Yeah.”

Minion 2: “…and we’re using averages – we’ve got no idea as to the variation in what is happening.”

Side kick 1 (Eric Idle): “Oh, yeah, averages, John. Remember some of the mad decisions we’ve made in hindsight because of averages?”

John Cleese: “Yeah. All right. I’ll grant you that our lack of data over time and the use of averages makes our report a bit suspect.”

Minion 3: “…and, even if we did have enough data points and could see the variation, we don’t understand the difference between noise and a signal (common and special cause variation)”

John Cleese: “Well, yeah. Obviously we don’t want to be caught tampering. I mean, understanding the difference between common and special cause goes without saying doesn’t it? But apart from a lack of data, (miss)using averages and tampering – ”

Minion 4: “We often compare ‘apples with pears’: Lots of the things we ‘hold people to account for’, they have virtually no ability to influence.”

Minion 5: “Much of the data we use is unrepresentative and/or coerced out of people, which makes any data biased.”

Minions: “Huh? Heh? Huh… “

Minion 6: “And we are focusing on one KPI and not seeing the side effects that this is causing to other parts of the system.”

Minions: “Ohh…”

John Cleese: Yeah, yeah. All right. Fair enough.

Minion 7: “and we are using targets, which are arbitrary measures that have nothing to do with the system and cause dysfunctional ‘survival’ behaviours from our people.”

Minions: “Oh, yes. Yeah… “

Side Kick 2 (Michael Palin): “Yeah. Yeah, our targets cause some pretty mad behaviours, John, and it’s really hard to spot/ find this out because our people don’t like doing ‘bad stuff’ and, as such, don’t like to tell us about it. Huh.”

Minion 8: “Our reports are focused on people (and making judgements about them), rather than on the process that they have to work within.”

Eric Idle: “And our people are ‘in the dark’ about how the horizontal value stream they work within is actually performing, John.”

Michael Palin: “Yeah, they only know about their silo. Let’s face it. If our people knew how the horizontal flow was actually doing, they’d be far more engaged in their work, more collaborative (if we removed some of the management instruments that hinder this) and therefore far more able and willing to continually improve the overall value stream.”

Minions: “Heh, heh. Heh heh heh heh heh heh heh.”

John Cleese: “All right, but apart from a lack of data, (miss)use of averages, tampering, comparing apples with pears, biased data, focusing on one KPI, the use of arbitrary targets, reports focused on judging people, and our value workers being ‘in the dark’….Look at what this report is telling us!”

Minion 9: We’re using activity measures (about outputs), rather than seeing the system and its capability for our customers (about outcomes).

John Cleese: Oh. Seeing the capability of the system from the customers’ point of view? SHUT UP!

  • THE END –

In short, many (most?) organisations are terrible when it comes to measurement. They are stuck in a weird ‘conventional reporting’ world. Perhaps this is a blind spot in our human brains?

‘Statistics’ is a word that strikes fear into the hearts and minds of many of us. I’m happy to admit that I’m no expert. But I think we should have a healthy respect for data and how it should and should not be used. I’ve heard many a manager raise their voice to say that they have the data and so can ‘prove it!’…and then go on to make inferences that cannot (and should not) be justified.

(Personal view: I think that it is better to be mindful (and therefore cautious) of our level of competence rather than blissfully ignorant of our incompetence, charging on like a ‘Bull in a china shop.’)

Where to from here?:

I’ve previously written a few posts in respect of measurement. I’ve linked a number of them in the skit above or in the notes below. Perhaps have a (re)read if you’d like to further explore a point I’m attempting to make.

…and here’s a reminder of the brilliant Inspector Guilfoyle blog that is dedicated to measurement. He writes nice ‘stick child’ stories about the mad things we do, why they are mad…and what a better way looks like.

Some closing notes on some of the ‘reporting madness’ points made above:

Binary Comparisons: Here’s a really great explanation of the reasons why we shouldn’t use a couple of data points: Message from the skies

Averages: If you don’t understand the point about averages, then have a think about the following quote: “Beware of drowning in a river of average depth 1 metre.” (Quoted by John Bicheno in ‘The Lean Toolbox’)

Variation: Deming’s red bead experiment is an excellent way to understand and explore the point about variation that is inherent in everything. I’ve written about variation in (what happens to be my most read post to date): The Spice of Life

Tampering: This comes about from people not understanding the difference between common and special cause variation. I wrote a specific post about the effects of tampering on a process: Tampering

Biased data: There are loads of reasons why data collected might be biased. The use of extrinsic motivators (as in contingent monetary incentives) is a BIG one to consider and understand.

Targets: John Seddon  is the place to go if you want a deeper understanding of the huge point being made. His book ‘Freedom from Command and Control’ is superb. Also, see my post The trouble with targets.

Capability measures: I believe that this point can take a bit to understand BUT it is a huge point. I wrote Capability what? In an attempt to assist.

Capability what?

giantknave Brain Farts , , , , ,

tape_measureReaders of this blog will have likely come across a phrase that I often use but which you might not be too clear on what is meant – this phrase is Capability Measure.

(Note: I first came across the use of this specific phrase from reading the mind opening work of John Seddon).

I thought it worthwhile to devote a post to expand upon these two words and, hopefully, make them very clear.

Now, there are loads of words bandied around when it comes to the use of numbers: measures, metrics, KPIs, targets. Are they all the same or are they in fact different?

Let’s use the good old Oxford dictionary to gain some insights that might assist:

Measure:     “An indication of the degree, extent, or quality of something”

Metric:     “A system or standard of measurement”

KPI (Key performance indicator): “A quantifiable measure used to evaluate the success of an organization, employee, etc. in meeting objectives for performance.”

Target:     “An objective or result towards which efforts are directed”

So putting these together:

A measure quantifies something…but this of itself doesn’t make it useful. It depends on what you are measuring! In fact, there is a huge risk that something that is easily measureable unduly influences us:

“We tend to overvalue the things we can measure and undervalue the things we cannot.” (John Hayes)

A metric is the way that a measurement is performed – it’s operational definition. There’s not much point in taking two measurements of something if the method of doing so differs so much as to materially affect the results obtained.

KPIs are an attempt to get away from using lots of different measures and, instead, boil them down into a handful of (supposedly) ‘important ones’ because then that will make it sooo much easier to manage won’t it?…I hope your ‘Systems thinking’ alarm bells are ringing – if we want to understand what is really happening, we need to study the system. Any attempts at short-cutting this understanding, combined with the use of targets and extrinsic motivators is likely to lead to some highly dysfunctional behaviour, causing much damage and resulting in sub-optimal outcomes. The idea of ‘management by dashboard’ is deeply flawed.

Targets – well, where to start! The dictionary definition clearly shows that their use is an attempt at ‘managing by results’…which is a daft way to manage! We don’t need a target to measure…and we don’t need (and shouldn’t attempt) to use a target to improve! A target tells us nothing about the system; distorts our thinking; and steals our focus from where it should be.

So what are we measuring?

I hope I’ve usefully covered ‘measure’ and its related terms so let’s go back to the first word: Capability

To start, we need to be clear as to what system we are studying and what its purpose is from the customer’s point of view. Then we need to ask ourselves “so what would show us how capable we are of meeting this purpose (in customer terms)?”

Some important points:

  • Capability is always about meeting the customer’s purpose and should be separate from the method of doing so:
    • An activity measure (i.e. to do with method), such as “how many calls did I take today”, is NOT a capability measure. None of my customers care how many calls I took/made!;
    • Activity measures constrain method (tie us in to the current way of working i.e. “we make calls”) whilst capability measures liberate method and encourage experimentation (“what would happen to our capability if we…”).
  • The best people to explain what really matters to the customer are the front line process performers who help them with their needs (i.e. NOT managers who are remote from the gemba):
    • The process performers know what the customers actually want and whether they are satisfied or not
  • As a rule of thumb, the end-to-end process time from the customer’s point of view is almost always an essential capability measure BUT:
    • end-to-end is defined by the customer, not when we think we have finished;
    • Targets will distort the data that we collect and thereby lead to incorrect findings…so, if you really want to understand your system’s capability you need to remove the targets and related contingent rewards.
  • Other examples of likely capability measures of use are:
    • A system’s ‘one-stop capability’: the amount of demand that can be fully satisfied (as determined by the customer) in one-stop;
    • The accuracy and value created for the customer; and
    • The safety and well-being of your people whilst delivering to the customer

An example:

I am currently moving house. I have to switch the electricity provision from the previous occupants to me. I want this switch to happen as painlessly as possible, I only want to pay for my electricity usage (none of theirs) and I want the confidence to believe that this is the case.

So:

  • The system in question is the electricity switching process;
  • My purpose is to switch:
    • Easily (minimum effort on my part….easy to start the process, no need to chase up what is happening, and easy to know when it is complete)
    • On time (on the switching date/ time requested); and
    • Transparently (so that I trust the meter readings and their timings)
  • I don’t care how the electricity companies actually achieve this switching between themselves (the method, such as whether they use a SMART meter reading or a man comes to the house or…). I just care about the outcomes for me.

The electricity company should be deriving measures to determine how capable they are in achieving against my purpose. They are then free to experiment on method and see whether their capability improves.

I have deliberately used a generic example to make the point about the system in question, its purpose and therefore capability. You can apply this thinking to your work: what the customer actually wants/ needs and how you would know how you are doing against this.

Sense-check: Capability measures are method-agnostic. Think about putting your method inside a metaphorical ‘black box’. Your capability is about what goes into the black box as compared to what comes out and what has been achieved. You can then do ‘magic’ (I mean experiments!) as to what’s inside the black box and then objectively consider whether its capability has improved or not.

What does a capability measure look like, who should see it and why?

Okay, so let’s suppose we now have some useful capability measures. How should they be presented and to whom…and what are we hoping to achieve by this?

The first big point is that the measure should be shown over time*. We should not be making binary comparisons, and then overlaying variance analysis and ‘traffic lights’ to supposedly add meaning to this (ref: Simon Guilfoyle’s excellent blog ).

We want to see the variation that is inherent in the system (the spice of life) so that we can truly see what is happening.

* Note: A control chart is the name for the type of graph used to study how a metric changes over time. The data is plotted in time order. Lines are added for the average, upper and lower control limits – where these are worked out from the data…but don’t worry about ‘how’ – these statistics can be worked out by an appropriate computer application (e.g. Minitab) in the hands of someone ‘in the know’.

Here’s a control chart showing the time it takes me to cycle to or from work:

Cycle time control chart

The second big point is that these capability control charts should be in the hands of those who perform the work. There’s little point in them being hidden within some managerial report!

Here’s what Jeffrey Liker says about how Toyota use visual management:

Every metric that matters…is presented visually for everyone who is involved in meeting the goal [purpose] to see. A key reason…is that it clarifies expectations, determines accountability for all the parties involved and gives them the ability to track their progress and measure their self-development.

[Making these metrics highly visible] is not to control behaviour, as is common in many companies, but primarily to give employees a transparent and understandable way to measure their progress.

Put simply: if the people doing the work can see what is actually happening, they are then in a place to use their brains and think about why this is so, what they could experiment with and whether these changes improved things or not* ….and on and on.

* Looking back at my ‘cycling to work’ control chart: I made a change to my method at cycle ride number 15 and (with the caveat that I need more data to conclude) the control chart shows me whether my change in method made things better, worse or caused no improvement. I cannot tell this from a binary comparison with averages, up/down arrows and traffic lights.

It should by now be clear that a capability measure is about the system, and NOT about the supposed ‘performance’ of individual operators within.

To summarise:

In bringing the above together, John Seddon applies 3 tests to determine whether something is a good measure. These tests are:

  1. Does it relate to purpose? (i.e. what matters to the customer);
  2. Does it help in understanding and improving performance? (i.e. does it reveal how the work works? To do this, it must be a measure over time, showing the variation inherent within the system, and it must be devoid of targets);
  3. Is it integrated with the work? (i.e. in the hands of the people who do the work so that they can develop knowledge and hence improve).

If it passes these three tests then you truly have a useful Capability Measure!

As luck would have it: One of my favourite bloggers, ‘Think Purpose’, released a similar ‘measurement’ post just after I had written the above. It includes a couple of very useful pictures that should compliment my commentary. It’s called A managers guide to good and bad measures – you could print them out and put them on your wall 🙂

A clarification: I’m happy with the use of the word ‘target’ if it is combined with the word ‘condition’. A reminder that a target condition (per the work of Mike Rother) is a description of the desired future state (how a process should operate, intended normal pattern of operation). It is NOT a numeric activity target or deadline. I explain about this in my earlier post called…but why?

A breakthrough!…but is it all that it seems?

giantknave Brain Farts , , , , ,

The word Breakthrough breaking through glass to symbolize discovSo, over the last few days a number of people have sent me links to this recent business article on Stuff: Accenture ditches annual performance reviews. Thanks for that, you know who you are 🙂

In summary:

  • Accenture, one of the largest professional services organisations in the world has decided to radically change its people processes: getting rid of the annual performance review
  • They aren’t the first ‘big beast’ to do something like this:
    • Deloitte (THE biggest professional services organisation in the world) went public in a similar vein last March. An April 2015 HBR article called Reinventing performance management explains where they are going;
    • I understand that the likes of Microsoft, Expedia and Adobe dropped most or all of the performance review process a couple of years ago;
    • Our very own NZ organisation, Telecom (or is that Spark?!), appeared to be heading down a similar path back in 2013 , though this would appear to me to have been driven by cost rather than the science of psychology:

Telecom [will] stop using online forms to appraise staff performance, reverting to a “far lighter” system of one-on-ones and “adult-to-adult conversations” on regular four-to-eight week cycles, he [Simon Moutter, CEO] said.

The “forms and processes” associated with performance appraisal had impeded Telecom, he said.”When we hit ‘appraisal season’, the company nearly grinds to a halt with the bureaucracy.”

Caveat: Looking at this 2015 Spark site, I’m not sure whether they successfully ‘broke away’ from the past…the picture at the bottom looks remarkably familiar!

A reminder: I have written quite a bit on the subject of performance review. In particular see An exercise in futility.

Ironic

What I find highly ironic about professional services firms eulogising about their new found wisdom is that they have large ‘human capital’ consulting arms that have been selling their wares for decades (I know, I used to work alongside them)…and what have they been earning millions of $ on? Yep, advising on implementing supposedly highly researched incentives schemes and performance review programmes….you know, the ones that they have now decided aren’t so great.

Taking a look, for example, at Deloitte’s website, I can deduce that they see a huge opportunity in presenting themselves as (what professional service organisations love to call themselves) ‘thought leaders’ to sell their new-found performance management brilliance (the next Silver Bullet) to all the other organisations out there.

A Fudge?

I have read the Deloitte HBR article (referenced above) and I see their ‘answer’ as a likely fudge.

They talk a lot about the wasteful time and effort expended in the current annual appraisal system. They talk about it not actually deriving valid results (being hugely biased by who is making the judgement). Yet their answer (when boiled down to its essence) is to merely make it simpler – a sort of ‘reboot’. It would appear that they are still asking questions about a person to rate them, which will determine a reward.

You could point to their strap line of “Replace ‘rank and yank’ with coaching and development” and, yes, I can get behind that BUT:

  • they haven’t once talked about the system and its monumental effect on what a person can (or cannot) achieve; and
  • they appear to be clinging to the idea of motivating an individual’s performance through contingent rewards, and judging them accordingly.

I can see that the games people understandably play will simply mutate, yet remain.

“Tell me how you will measure me and I will [show] you how I will behave” (Goldratt).

Going back to Alfie Kohn’s work:

  • First you need to remove contingent rewards;
  • Second, you need to re-evaluate the performance review process (change from judgement to feedback);and
  • Then you can create the conditions for authentic motivation.

A reminder of why judgement and rewards do not belong anywhere near helping people develop:

“If your parent or teacher or manager is sitting in judgement of what you do, and if that judgement will determine whether good things or bad things happen to you, this cannot help but warp your relationship with that person.

You will not be working collaboratively in order to learn or grow; you will be trying to get him or her to approve of what you are doing so that you can get the goodies.

A powerful inducement has been created to conceal problems, to present yourself as infinitely competent, and to spend your energies trying to impress (or flatter) the person with power.” (Kohn)

“Mind the gap”

Many an organisation might read about* what the likes of Accenture are doing and conclude that, clearly, they need to copy them.

But a reminder of the dangers of copying: Yes, look at what others are doing and, yes, be curious as to why…BUT you need to work it out for yourselves – you need to ‘get’ why it is the right thing to do and then adapt it accordingly. Otherwise you can expect one great big mess.

(* A particular quote from the Accenture article which I found of interest: “Employees that do best in performance management systems tend to be the employees that are the most narcissistic and self-promoting” We should be seriously questioning if this is actually what we want.)

“Nothing to see here”

Whilst a part of me is very pleased to see the big beasts ‘coming out’ (more or less) against the performance review process:

  • I’m unmoved (being polite) by their commissioning/ invoking of seemingly new and brilliant research that arrived at their ‘new insights’.

Why? Well, there’s nothing new here. Go back to Alfie Kohn’s brilliant book ‘Punished by Rewards’ to see the body of research from many decades ago. Go back to Deming’s 4 day lectures that he gave to thousands between 1981 and 1993 (that’s more than 30 years ago!!):

Deming’s Deadly disease number 3: Evaluation of performance, merit rating, or annual review

“In practise, annual ratings are a disease, annihilating long term planning, demolishing teamwork, nourishing rivalry and politics, leaving people bitter, crushed, bruised, battered, desolate, despondent, unfit for work for weeks after receipt of rating, unable to comprehend why they are inferior…sending companies down the tube.”

…go back even further to what Deming and the Japanese were doing from the 1950s.

During this time, the majority of large corporations have been pushing in, and constantly justifying, the exact opposite of where they have arrived at now.

Now, to be clear, I think it is really great that there appears to be a movement against the ridiculous performance review process BUT:

  • I’m not convinced that they fully ‘get it’ in respect of human psychology; and
  • I think it is disingenuous, arrogant (or maybe ignorant) of any organisation that does not (outwardly) recognise that what they have just ‘discovered’ has been there, loud and clear, in front of their eyes all the time.

Falling into that trap!

giantknave Brain Farts , , , , , ,

BearTrap_01.jpga203455b-ef09-4c5a-be36-5fe7351fd23fLargeSo I cycle to work when I can. It’s approx. 22km from one side of the city of Christchurch to the other, right through the middle. I (usually) enjoy it immensely.

I upgraded my cycle computer a few months back to a fancy new GPS one. It’s got this cool screen that tells me lots of information. I can also download each cycle ride’s data onto my computer to analyse later – perfect for a geek!

I’d been using this computer for a few weeks and was starting to get an understanding of my average km/ hour for each trip. Sometimes I could get it into the 27 km/hour range, other times I could only manage 23 km/hour….and before all you MAMILs* out there guffaw thinking “how slow?!” the average speed takes into account stopping at lots of junctions (traffic lights, roundabouts, road works) and crossing busy traffic….not that I’m saying that I am fast.

(* MAMIL: middle-aged men in Lycra…and I must confess to being a member of)

Further, the computer tells me my average km/ hour at the bottom of the screen as I cycle. I can see it going up and down throughout the journey. If I stop at a junction I can see it plunge rapidly whilst I am sat idle and then if I cycle on a long flat straight I see it slowly rise back up again.

Now, I fell into the trap! I began to get distracted by the average km/ hour reading as I cycled. Without consciously thinking about it I set myself a target of completing each ride at an average of 25 km/hour or higher. The fact that I was doing this only dawned on me the other day after I performed a dodgy manoeuvre in my obsession to keep my average above the target… my sudden sense of mortality put me into a reflective mood.

Here’s what I reflected on:

What had I become?

  • I had started to look at the average km/ hour reading all the time and obsess about it dropping below the magical 25;
  • If I was stopped at traffic lights, I watched the average km/hour reading sink before my eyes. I was then in a mad hurry to get off so as to get that average back up again as quickly as possible:
    • which meant that I was trashing my legs and blowing them out too early in the ride…
    • which put me at huge risk of pulling a muscle/ injuring a joint whilst piling on the pressure to get back above that target…
    • which meant that I didn’t cycle the next day because I couldn’t!
  • If I was ‘on the cusp’ of the target and coming up to a junction then I was starting to do dangerous things in order to keep going, like running orange lights or crossing lanes in between cars;
  • Even more bizarrely, I had unconsciously started to cheat!
    • I had changed my behaviour to only turning on the computer after I had got going from my house because the few seconds getting up to speed from rest might count!
    • If my average was 25.0 as I got to work I would turn the computer off before I came to a rest…so that it couldn’t drop down to 24.9 as I slowed…because that would have meant failure!
  • Conversely, if I was well above the target (let’s say because I had a good tail wind or I had been really lucky with the light sequences), then I was pretty happy and relaxed…no need to push since I was ahead. I could have gone faster.

Reading the above, you may think me to be a bit of a nutter! Did I really do those things? Well, yes, I did…and I can honestly say that the 25 km/ hour target that I had set myself was to blame.

Now for those of you who have been on one of my courses or who have read the majority of posts on this blog, you will likely have a good laugh at me – “he bangs on about the evils of targets all the time!”

So, getting away from my target:

What is my actual purpose in cycling to work?

On reflection I came up with the following:

“To safely cycle to work so that I get fitter and I use the car less.”

Three things come out of this purpose:

  • Fitter: I wanted to get better at it
  • Use the car less: I needed to be able to cycle consistently, day after day
  • Safely: there’s no point in getting killed or badly injured cycling!

This clarity of purpose has helped me drastically change the way I cycle!

Thinking about variation within the system:

In this case, the system is the cycling to/from work and the conditions I encounter in doing so. One cycle ride is a single ‘unit of production’. I should be thinking about the capability of the system (about how all units go through it), not about a single unit.

Here’s a control chart showing the variation in the last 20 of my rides:

bike control chart

The control chart enables you to visualise what a table of data can’t…that my riding is stable. It shows the meaninglessness of the target and variance analysis.

The following are the more obvious causes of variation:

  • Head wind or tail wind (makes a huge difference!);
  • Wet or dry weather;
  • Heavy or light traffic;
  • Whether the traffic light sequences are with me or not;
  • …and so on

Some special causes might be:

  • An accident;
  • Road works

…although if you live in Christchurch you will know that there’s nothing special about road works !!!

You can see that it would be bizarre for me to achieve the same average km/ hour every day. Going back to that target: Some days it will be impossible for me to hit the 25, other days it will be really easy…and that’s without me changing anything about my riding.

Note: Looking back at the control chart, you might think that you detect an improvement from around ride 15 onwards. In fact, it was at around ride 15 that I had my ‘I’ve fallen into the target trap’ epiphany….so, yes, there could be something in that. However, you should also know that whilst ride 20 looks fantastic (it was), I had a massive Nor’ Wester wind behind me literally pushing me along.

What can I experiment with to increase my capability?

The first thing I need to do is STOP LOOKING at the average km/hour as I cycle. Then I can consider what I can change about my cycling for every ride.

Since ride 15 I’ve begun to experiment with:

  • Junctions: looking well ahead and adjusting my pace to time myself through them so that I reduce the need to slow down/ stop
  • Pedalling: trying to pedal more smoothly and efficiently
  • Crossing lanes: improving my balance when looking behind me so that I can safely do this whilst retaining my speed with the traffic
  • ….and so on.

Each of these changes will lead to small improvements on every ride, no matter what the conditions are. It’s not about the current unit, it’s about every unit.

Results?

Well, it’s too early to draw valid conclusions (I need more data), and it’s a never ending journey of improvement BUT I can say that I am cycling more often (because I didn’t wreck my legs the day before) and I’m having far less ‘that car’s a bit too close for comfort’ moments.

So what’s the point?

Targets cause dysfunctional behaviour. As Simon Guilfoyle makes clear:

“As a ‘method’ [Setting a target] is rubbish because it disregards the capability of the system and natural variation. … It assumes that the system knows the target is there and will respond to it (it doesn’t and it won’t!) It ignores the fact that the greatest opportunity for improving performance lies in making systemic adjustments rather than berating, comparing, or otherwise trying to ‘motivate’ the workers to achieve the target.”

‘No targets’ doesn’t mean ‘no measurement’! In fact, it’s quite the reverse. It means actually studying meaningful measures (i.e. against purpose) over time (via a control chart), understanding the capability of the system and therefore truly understanding whether it is improving, staying the same or getting worse.

Do you have targets? So what dysfunctional behaviours do they cause, moving us away from purpose?

The Spice of Life

giantknave Brain Farts , , , ,

spices-442726_640Variety is the spice of life. If everything were the same it would be rather boring. Happily, there is natural variety in everything.

Let me use an example to explain:

I was thinking about this as I was walking the dog the other day. I use the same route, along the beach each day, and it takes me roughly the same time – about 30 minutes.

If I actually timed myself each and every day (and didn’t let this fact change my behaviour) then I would find that the walk might take me on average 30 minutes but it could range anywhere between, say, 26 and 37 minutes.

I think you would agree with me that it would be somewhat bizarre if it always took me, say, exactly 29 minutes and 41 seconds to walk the dog – that would just be weird!

You understand that there are all sorts of reasons as to why the time would vary slightly, such as:

  • how I am feeling (was it a late night last night?);
  • what the weather is doing, whether the tide is up or down, and even perhaps what season it is;
  • who I meet on the way and their desired level of interaction (i.e. they have some juicy gossip vs. they are in a hurry);
  • what the dog is interested in sniffing…which (I presume) depends on what other dogs have been passed recently;
  • if the dog needs to ‘down load’ or not and, if so, how long this will take today!
  • …and so on.

There are, likely, many thousands of little reasons that would cause variation. None of these have anything special about them – they are just the variables that exist within that process, the majority of which I have little or no control over.

Now, I might have timed myself as taking 30 mins. and 20 seconds yesterday, but taken only 29 mins. and 12 seconds today. Is this better? Have I improved? Against what purpose?

Here’s 3 weeks of imaginary dog walking data in a control chart:

Untitled

A few things to note:

  • You can now easily see the variation within and that it took between 26 and 37 minutes and, on average, 30 mins. Understanding of this variation is hidden until you visualise it;
  • The red lines refer to upper and lower control limits: they are mathematically worked out from the data…you don’t need to worry about how but they signify the range within the data. The important bit is that all of the times sit within these two red lines and this shows that my dog walking is ‘in control’ (stable) and therefore the time range that it will take tomorrow can be predicted with a high degree of confidence!*
  • If a particular walk had taken a time that sits outside of the two red lines, then I can say with a high degree of confidence that something ‘special’ happened – perhaps the dog had a limp, or I met up with a long lost friend or…..
  • Any movement within the two red lines is likely to just be noise and, as such, I shouldn’t be surprised about it at all. Anything outside of the red lines is what we would call a signal, in that it is likely that something quite different occurred.

* This is actually quite profound. It’s worth considering that I cannot predict if I just have a binary comparison (two pieces of data). Knowing that it took 30 mins 20 secs. yesterday and 29 mins 12 secs. today is what is referred to as driving by looking in the rear view mirror. It doesn’t help me look forward.

Back to the world of work

The above example can equally be applied to all our processes at work…yet we ignore this reality. In fact, worse than ignoring it, we act like this isn’t so! We seem to love making binary comparisons (e.g. this week vs. last week), deriving a supposed reason for the difference and then:

  • congratulating people for ‘improvements’; or
  • chastising people for ‘slipping backwards’ whilst coming up with supposed solutions to do something about it (which is in actual fact merely tampering)

So, hopefully you are happy with my walking the dog scenario….here’s a work-related example:

  • Bob, Jim and Jane have each been tasked with handling incoming calls*. They have each been given a daily target of handling 80 calls a day as a motivator!

(* you can substitute any sort of activity here instead of handling calls: such as sell something, make something, perform something….)

  • In reality there is so much about a call that the ‘call agent’ cannot control. Using Professor Frances Frei’s 5 types of service demand variation, we can see the following:
    • Arrival variability: when/ whether calls come in. If no calls are coming in at a point in time, the call agent can’t handle one!
    • Request variability: what the customer is asking for. This could be simple or complex to properly handle
    • Capability variability: how much the customer understands. Are they knowledgeable about their need or do they need a great deal explaining?
    • Effort variability: how much help the customer wants. Are they happy to do things for themselves, or do they want the call agent to do it all for them?
    • Subjective preference variability: different customers have different opinions on things e.g. are they happy just to accept the price or are they price sensitive and want the call agent to break it down into all its parts and explain the rationale for each?

Now, the above could cause a huge difference in call length and hence how many calls can be handled…but there’s not a great deal about the above that Bob, Jim and Jane can do much about – and nor should they try to!. It is pure chance (a lottery) as to which calls they are asked to handle.

As a result, we can expect natural variation as to the number of calls they can handle in a given day. If we were to plot it on a control chart we might see something very similar to the dog walking control chart….something like this:

Control chart 2

We can see that:

  • the process appears to be under control and that, assuming we don’t change the system, the predictable range of calls that a call agent can handle in a day is between 60 and 100;
  • it would be daft to congratulate, say, Bob one day for achieving 95 and then chastise him the next for ‘only’ achieving 77…yet this is what we usually do!

Targets are worse than useless

Let’s go back to that (motivational?!) target of 80 calls a day. From the diagram we can see that:

  • if I set the target at 60 or below then the call agents can almost guarantee that they will achieve it every day;
  • conversely, if I set the target at 100 or above, they will virtually never be able to achieve it;
  • finally, if I set the target anywhere between 60 or 100, it becomes a daily lottery as to whether they will achieve it or not.

….but, without this knowledge, we think that targets are doing important things.

What they actually do is cause our process performers to do things which go against the purpose of the system. I’ve written about the things people understandably do in an earlier post titled The trouble with targets.

What should we actually want?

We shouldn’t be pressuring our call agents (or any of our process performers) to achieve a target for each individual unit (or for an average of a group of units). We should be considering how we can change the system itself (e.g. the process) so that we shift and/or tighten the range of what it can achieve.

So, hopefully you now have an understanding of:

  • variation: that it is a natural occurrence…which we would do well to understand;
  • binary comparisons and that these can’t help us predict;
  • targets and why they are worse than useless; and
  • system, and why we should be trying to improve its capability (i.e. for all units going through it), rather than trying to force individual units through it quicker.

Once we understand the variation within our system we now have a useful measure (NOT target) to consider what our system is capable of, why this variation exists and whether any changes we make are in fact improvements.

Going back to Purpose

You might say to me “but Steve, you could set a target for your dog walks, say 30 mins, and you could do things to make it!”

I would say that, yes, I could and it would change my behaviours…but the crucial point is this: What is the purpose of the dog walk?

  • It isn’t to get it done in a certain time
  • It’s about me and the dog getting what we need out of it!

The same comparison can be said for a customer call: Our purpose should be to properly and fully assist that particular customer, not meet a target. We should expect much failure demand and rework to be created from behaviours caused by targets.

Do you understand the variation within your processes? Do you rely on binary comparisons and judge people accordingly? Do you understand the behaviours that your targets cause?

DUMB

giantknave Brain Farts , , , ,

smartWe are all taught at an early age in our careers (i.e. ‘Management for dummies’) that we should cascade down S.M.A.R.T objectives. You will come across it as an idea that is so deeply rooted that it has been co-opted as ‘common sense’.

Sounds so good, it must be right, right?

Let’s just remind ourselves what SMART stands for:

  • Specific
  • Measurable
  • Achievable
  • Realistic
  • Time bound

Let’s then also remind ourselves about the definition of a system (taken from my earlier ‘Harmony or cacophony’ post):

“A system is a network of interdependent components that work together to try to accomplish the aim [purpose] of the system.” (W. Edwards Deming)

The cascaded objectives technique (known as Management by Objectives, or M.B.O) is used by ‘Command-and-control’ organisations in the mistaken belief that, if we all achieve our cascaded personal objectives, these will then all roll up to achieve the overall goal (whatever that actually is).

This misunderstands:

  • the over-riding need for all the parts (components) of a system to fit together; and
  • the damage caused by attempting to optimise the components…because this will harm the whole system.

A simple illustrative example (taken from Peter Scholtes’ superb book called ‘The Leaders Handbook’):

Let’s say that we run a delivery company – our system. Fred, Amy and Dave are our drivers – our people components. If we provide them each with SMART personal objectives cascaded down (and offer performance-based pay), we might assume that they will all be ‘motivated’ to achieve them and therefore, taken together, the purpose of the whole will be achieved. Sounds great – I’ll have some of that!

…but what should we expect?

  • Each driver might compete with the others to get the best, most reliable, largest-capacity truck;
  • Each driver might compete for the easiest delivery assignments;
  • Drivers might engage in ‘creative accounting’: such as trying to get one delivery counted as two; or unloading a delivery somewhere nearby where it can be made after hours so that they can go back to the warehouse to get more jobs;
  • If we have created a competition out of it (say, the getting of a desirable award) then we can expect to see little driver co-operation, more resentment and perhaps even subtle sabotage.

The above shows that the sum of the outcomes will not add up to what we intended for the whole system…and, in fact, will have caused much unmeasured (and likely immeasurable) damage!

This is a good point to bring out Eli Goldratt’s classic quote:

“Tell me how you will measure me and I will tell [show*] you how I will behave.”

* I prefer to use the word ‘show’ since most people won’t tell you! They know their actions aren’t good for the overall system (they aren’t stupid) and so don’t like telling you what daft practices the management system has ended up creating.

A critique of S.M.A.R.T:

“SMART doesn’t tell us how to determine what to measure, and it assumes knowledge – otherwise how do we know what is ‘achievable’ and ‘realistic’? It is only likely to promote the use of arbitrary measures that will sub-optimise the system.” (John Seddon)

If an individual (or ‘team’) is given a truly SMART objective then, by definition, it would have to have been set so that they could achieve it on their own….otherwise it would be unrealistic.

Therefore any interdependencies it has with the rest of the organisational system would have to have been removed…which, clearly, given the definition of a system means one of the following:

  • if all interdependencies had been successfully removed…then meeting the resultant SMART objective will be:
    • a very insignificant (and very possibly meaningless) achievement for the system; and/or
    • sub-optimal to the system (i.e. work against the good of the whole)

OR

  • if (most likely) it was in fact not possible to truly remove the interdependencies…despite what delicate and time consuming ‘word smith-ing’ was arrived at…then:
    • it will be a lottery (not really under the person’s control) as to whether it can be achieved; and/or
    • it will ‘clash’ with other components (and their supposedly SMART objectives) within the system

So where did the post title ‘D.U.M.B’ come from? Here’s a thought provoking quote from John Seddon:

“We should not allow a plausible acronym to fool us into believing that there is, within it, a reliable method.”

Consider     

  • SMART: Specific, Measurable, Achievable, Realistic, Time-bound

With

  • DUMB: Distorting, Undermining, Management-inspired, Blocking improvement

Does the fact that the acronym and its components ‘match’ make it any more worthy?

Cascaded personal objectives will either be ineffective, detrimental to the whole system or a lottery (outside of the person’s control) as to whether they can be achieved.

We need to move away from cascaded personal objectives and, instead:

  • see each horizontal value stream as a system, with customers and a related purpose;
  • provide those working within these systems with visible measures of the capability of the system as against its purpose; and
  • desist from attempting to judge individuals ‘performance’ and thereby allow and foster collaboration and a group desire to improve the system as a whole.