Any of you reading this who have been on my Systems Thinking course will have had the fun of being involved in Dr Deming’s famous red bead experiment.

This post is about Dr Deming’s other (not quite so famous but equally important) Funnel experiment. The experiment teaches about the harm caused by ‘Management by Results’ (MBR) …where this occurs through tampering.

Here’s the experiment set up:

We have a flat horizontal surface (let’s say a table) with a piece of paper placed on top of it. We also have a kitchen funnel (like we would use to decant a liquid from one bottle to another), and a marble that we will drop through the funnel towards the paper below.

Let’s assume that the funnel is held upright in some sort of stand, say 50cm above the piece of paper.

Now we mark a cross in the middle of the piece of paper.

Goal: to hit the cross with the marble by dropping it through the funnel.

Round 1: We position the funnel over the cross and then drop the marble through the funnel 50 times. For each marble drop, we mark the spot on the paper where it hits.

We are likely to see something like this on the paper (looking from above):

Remember, we simply dropped 50 marbles without attempting to make any changes in-between drops and the paper shows the system to be stable. However, note that there is variation as to where the marble landed. It continually landed near the cross (with probably a few direct hits) but there was variation.

Round 2: So this time, we think that by adjusting the position of the funnel between each marble drop, we can ensure that the marble hits the cross on the next drop!

So we drop the 1^st marble, note where it lands as compared to the cross and then move the funnel to compensate for this error. i.e.

• if the marble landed 1 cm to the left (west) of the cross, we move the funnel 1cm to the right (east) of its current position….because this ‘fine tuning’ will make the next drop hit the cross, right?;
• if this 2nd marble lands 2cm below (south of) its new position, then we move the funnel 2cm north from where it is currently positioned;
• ….and so on, iteratively moving the funnel

So, what happens after we use this approach with our 50 marbles, iteratively adjusting the funnel’s position after each drop?

Well, it’s somewhat disappointing!

Our attempts at compensation have made the variation increase drastically (experiments show by approx. 40%). We’ve made things much worse.

Clearly our ’round 2′ method of compensating didn’t work as we wished. Is there another way of compensating and therefore getting better at hitting the cross?

Round 3: The new idea is to do the opposite of the last idea! This time, we will move the funnel directly over where the last marble landed and keep doing this for the 50 drops.

Oh dear, the marble is moving away from the cross and will eventually move off the table and (as Deming put it) all the way “off to the Milky Way”.

Perhaps using the last marble drop as a guide isn’t a good idea!

Conclusions: So which method was best?

• Round 1 was clearly the best.
• Round’s 2 and 3 are examples of tampering (though in different ways). They show the effects of tweaking a process based on the ongoing results of that process…it will simply increase the variation and the chances of failures.

So, what should we do? To actually improve a process requires an understanding of the sources of the variation…and then the performance of controlled experiments to identify process improvements.

For our Funnel system we could experiment with:

• lowering the funnel;
• decreasing the size of the funnel hole;
• strengthening the stand holding the funnel to make it more stable;
• …performing the process in a vacuum 🙂

All of these proposed improvements involve changing the system rather than merely tampering with it based on previous results.

So what?

Now all of the above looks like good fun…I’m already thinking about borrowing a funnel…but it seems an awful long way from our working lives. So let’s explain why in fact it’s not…

Taking the production/ selling of something, let’s say a sandwich shop as an example:

• you sold 10 sandwiches on Monday, so you make 10 for Tuesday..
• you only sold 2 sandwiches on Tuesday, so you throw 8 in the bin (not fresh anymore) and you only make 2 sandwiches for Wednesday….
• you have 6 customers on Wednesday, so you sell the 2 sandwiches you made, have 4 disappointed would-be-customers but make 6 sandwiches for Thursday…
• …and so on. You can expect to have major stock problems and a lot of unhappy customers!
• it would be much better to make a set number of sandwiches each day, collect information about demand variation over a sensible period of time and then adjust your system accordingly.

The sandwiches are a very simple example of any process. What about taking a call centre as an example:

• There will be a great deal of natural variation in customer calls throughout a day (with a number of causes, categorised and explained in this earlier ‘Spice of Life’ post)…so the number of ‘calls waiting’ will constantly fluctuate, though likely between predictable bounds. No surprises there.
• …let’s assume that Bob’s job is to constantly monitor the current ‘calls waiting’…
• …it gets to a point where Bob thinks the number of calls waiting is high…so he sends out an urgent request for everyone to drop what else they are doing and get on the phones…and they all rush to do so…
• ….so the number of ‘calls waiting’ now drops really low and even disappears…excellent. Now Bob sends out a screen pop-up message along the lines of “panic over, people who missed out on their breaks can go now”…
• ….so the number of ‘calls waiting’ now jumps up again…and the up-and-down cycle continues.
• Bob has a really stressful job looking at the numbers and continually tampering (using the ’round 2′ method) in a hopeless attempt to exactly match supply to demand.
• A better way would be to increase our understanding of the system by studying demand (its types and frequencies) and amending its design based on what we learned. There might be:
  • loads of failure demand in there (which is a waste of capacity); and
  • frequency patterns within the different types of value demand

Clarification: Many of you working in contact centres may say “but Steve, of course we analyse incoming calls and look for patterns!” I would note that:

• whilst you can, and should, look for predictable patterns in the data, I doubt that you can tell me how many calls you will get in the next 15 minutes and how long they will take. There will be variation and this is outside your control….does this make you tamper?
• Nearly all contact centres simply see incoming calls as ‘work to do’ with an ‘average handling time’. Hardly any will analyse this demand. Can you tell me what types of value and failure demand you get, and their frequencies…and what conclusions you draw from this?

I’m not picking on contact centres – I simply use it as an example that we should all be able to understand. Tampering happens all over the place.

In general, managers often look at the results of the last hour/ day/ week/ month* and attempt to make adjustments accordingly, whether by congratulating or berating staff, moving people between queues, changing work quotas, knee-jerk reacting to defects and so on.

(* where the unit of time will depend on the nature of the transactional service being provided)

In fact, praising someone one week for a perceived outstanding result (likely against the lottery of a target that they had very little control over meeting) and then giving them a ‘talking to’ the next because their result was considered poor is tampering.

The point is to understand the system and the reasons for variation. Then (and only then) you can make meaningful changes instead of merely tampering.

Note: The ‘Round 3’ type of tampering is not as common as the ‘Round 2’ type…but it does happen. Consider the following cases of using the last example to inform the next:

• Using the last board cut as a pattern for the next; or
• Train the trainer: Mary trains John who trains Bob who trains Jess.

Both of these examples show that any variation from purpose will be retained and amplified as it is passed on, like a chain of whispers.

Credit: I’ve seen the funnel experiment performed a few times now but, rather than taking the laborious time to recreate it, I have borrowed the 3 marble drop pattern pictures used above from this website.

Note: For those aficionados amongst you, this post represents a slightly simplified version of Deming’s full funnel experiment. There is yet another tampering rule (which I have left out for the sake of brevity) …but, just so you know, it also doesn’t work. You can read all about the funnel experiment in Chapter 9 of Deming’s book ‘The New Economics’.