Thursday, June 9, 2011

Overconfidence Bias

The other day, a friend and I are out throwing cricket balls to one another.

I believe that when practising a skill, you should set the difficulty of the thing you're practising to the point where you fail one time in ten. Then you're getting positive emotional reinforcement from your successes, while constantly practising things that are still challenging, and pushing yourself periodically. If you're practising a skill that is in fact dangerous, then this also gives you a reasonable chance of avoiding injury, if the usual failure case is to not get injured.

I felt that the throws Joe was giving me were getting below the one in ten level, i.e. I seemed to be catching most of them routinely without having to try too hard, and I was about to ask him to make them harder.

Instead, we started counting successes and failures.

Over the next twenty or so throws, the total number of catches was 14, and I dropped the ball seven times. i.e. the true rate was about one in three.

Both Joe and I thought that this was due to getting freaked out by suddenly making success/failure a thing that was noticed, or possibly getting distracted by the effort of counting. So we carried on. Neither of us thought that the throws had got more difficult because of the counting.

Over the next twenty throws, the proportion of catches rose, until after about seventy-five, the proportion was 60 to 15. I'd completely stopped worrying about counting, and I'm pretty sure that the effort of remembering the two numbers wasn't interfering.

I think we were observing learning actually happening, as the rate changed from one in three to one in four.

We ended the session there, since we were both bored, but I really can't now believe that I'm capable of catching nine out of ten similar throws.

And so I wondered for a bit what it is that makes a person (I'm assuming I'm typical!) who can do something one time in three, and has done it lots recently, and presumably failed one third of the time, believe sincerely that they are likely to succeed nine times out of ten.

A little research brings up the term Overconfidence Bias, which seems somewhat related, where people back their own judgements to be right 'with 90% confidence', and then get it right about 40% of the time.

Afterwards the thought occurred to me that I must have been miscalibrating my 9/10 rule all my life.

I used it to learn to ski and to learn to row, and at the moment I'm trying to use it to learn to catch cricket balls, and I imagine in various other contexts where I was learning to do some physical skill.

And then it occurred to me that I had no idea why I believed that that was the right ratio anyway.
I just seem to have come up with it once, tried it, found that it worked, and then adopted it as an article of belief without ever trying any other way of learning physical skills.

That also sounds like overconfidence, but actually it's called the Congruence Bias.
If you have a theory, you test it by doing what it says to do and seeing if it works.

And in fact if that's all you do, that's a terrible way of testing a theory. You should be looking into the dark and asking if, when you try something else, do you still get good results.

But it's hard to do this.

I'm now thinking 'I should try catching really hard throws'.
But I don't want to. I know what will happen. I'll get my fingers broken and my hands hurt and I'll teach myself to fear the ball even more than I already do.

I'm thinking 'I should try to catch really easy throws'.
But I don't want to. It will be very boring and I'll learn nothing from it.

It is difficult, this 'rationality'. Once you start looking at what you believe and why, and what you do and why, you find all sorts of odd things going on.

No comments:

Post a Comment