A fascinating mathematical problem

Posted by: Minky on 18 April 2004

I've just finished reading "The curious incident of the dog in the night-time" by Mark Haddon. The book has a brief passage on something called "The Monty Hall" problem. Marilyn vos Savant, the smartest person in the world by virtue of her Guinness record IQ, was asked the following question (my words) :

In a game show there are three doors. Behind two of them are goats. Behind the third is a car. You are a contestant trying to win the car. You choose a door and the game show host opens one of the two remaining doors to reveal a goat. Which door is most likely to have the car behind it (should you stick or switch) ?

If you have never encountered this problem before, have a think about it.

When Marilyn vos Savant gave her answer she was quite soundly chastised by some very heavy-hitting brainiacs, and this is at least as fascinating as the problem itself, because (as I have also found) this baby seems to have an almost supernatural ability to turn otherwise mild-mannered people into red-eyed monsters Mad
Posted on: 18 April 2004 by long-time-dead
OK - It's getting late and I am totally baffled by this problem.

3 doors - you've a 1 in 3 chance of winning if you stick. Whatever happens to any other door is totally unimportant.

Host eliminates one door and then offers a switch. The host MUST know where the car is to open a losing door in order to offer the switch. If he opened the door to reveal the car then game over and your chance was only ever 1 in 3.

Surely your chance is now 1 in 2 regardless if you switch or not as you are not getting the option of chhosing the opened door.

The opened door and the offering of a new question must surely negate the original question.

It's a 50:50 chance - win or lose - car or goat. It's surely that simple.
Posted on: 18 April 2004 by Minky
Blythe,

It was very thought provoking. I must admit that I read it with a sense of impending doom because at any moment it could easily have drifted into "about a boy" style mawkishness. Thank god that Haddon never allowed this to happen (although the last page jarred a bit in this respect).

It's interesting that Oliver Sacks was quoted on the front cover because I got some of the same visceral response reading this book as his "Awakenings". Behind things that most of us dismiss as "easy" lie extremely delicate and complicated mechanisms. I'm not sure that dwelling on this in real-time is constructive but it could help with things like tolerance and compassion.

Disturbingly, I felt myself identifying with Christopher on some level. I thought his analysis of R&R was spot on.

What did you think ?
Posted on: 18 April 2004 by matthewr
By an odd coincidence I was reading some stuff this last week which uses basic probabilities and simple statistics not a million miles away from those at work in this thread to work out the rather elegant solution to a seemingly intractable problem. I should warn you this is a pretty tough problem, but if you fancy yourself a bit handy at this sort fo thing then read on.

Mulder and Scully are each sent hundreds of thousands of uniquely numbered steel boxes. Each box has three doors numbered 1, 2 and 3.

With the boxes comes a letter which explains that the boxes contain a machine which, when you open one of the doors, will randomly flash a light either red or blue. Once you have opened the box and the flash has been emitted the box will be useless and no longer function. There is no way to determine before opening the door which colour the box will flash and the box bursts into flames after the flash which prevents any post-test investigation into how it works. Finally, the letter states, the boxes come in pairs, and if Mulder and Scully both open the same door on boxes with the same number then they will both see the same colour.

Intrigued by this Mulder phones Scully and they start opening boxes. Sure enough each time they open the same door on boxes with the same numbers the machine inside flashes the same colour for both Mulder and Scully.

"This is amazing", says Mulder. "I'm here in New York and you're in LA and yet these boxes are communicating with each other using an advanced alien technology".

"Pah! Why would you say that?" asks Scully.

"Becuase the flash colour is randomly determined when you open the door and yet our boxes always flash the same colour when we open the same door on boxes with the same number. So when I open my box it must somehow tell your box what colour to flash for the door I just opened so that it gets the same colour as mine".

"Don't be stupid, " says Scully, "obviously the flashes are not random but pre-determined. Each pair of boxes is just programmed to flash a certain colour when a certain door is opened."

"So inside my number 2,754 box there might be a program that says "1=red, 2=red, 3=blue" and your box 2,754 just has the same instructions. Give each pair of boxes a random 'program' and it looks like they are randomly flashing red or blue but still agree."

"Yes but...", starts Mulder.

"No buts", says Scully, "given that the boxes self destruct after we open the door so we cannot work out how they work, there is no way you can prove its truly random and not pre-programmed. And since them being programmed is a much more likely explanation than some bizarre, unknown form of instantaneous communication over 1000s of miles I think we have to believe the former".

Mulder is initially stumped and concedes defeat. But after thinking about it for a while he phones Scully back and announces that he can prove that the boxes are not pre-programmed.

"Ok," says Scully, "I'm listening".

"What we need to do is each open a lot of boxes but each of us randomly determine which door to open. We record the flashes for the door we opened on each box and compare our results".

So they each open their half of (say) 10,000 pairs of boxes randomly and independently deciding which door to open. Then Scully crunches the combined numbers in her iMac and analyses the results -- and as it turns out that Mulder and Scully saw the same colour in 50% of cases.

So who is right, Mulder or Scully?

Matthew
Posted on: 18 April 2004 by Steve Toy
quote:
"NOW....if part way through the game, just after Monty has opened the door to reveal a goat, an alien drops in, unaware of the rules and what has just happened.....well, he has two doors to choose from, one with a goat and one with a car. So HIS chances of choosing the car at this point ARE 50/50"

NO! It can never be 50/50. The process of choosing left or right, 1 or 2, can only be 50/50 if no process of elimination occurs.


For the alien no process of elimination has occured - unless the alien is then told which door had been originally chosen by the contestant prior to the host having opened one of the other two doors.

Thus for an uninformed alien, the odds are at 50/50; for the contestant who switches, the odds are 2/3 in his favour.



Regards,

Steve.
Posted on: 18 April 2004 by Steve Toy
Three guys are in a prison cell condemned to death at dawn. However, one of them is to be spared, and the decision as to which one it will be has already been taken by the guard. The three prisoners are aware that one of them is to be spared but they don't know which.

One of the three prisoners is awake while the other two sleep. He asks the guard to whisper the name one of the prisoners who will die at dawn other than himself. (The two sleeping prisoners obviously, for whatever reason don't care whether they live or die, which is the only explanation I can give for their being able to sleep through their few remaining hours before their fate becomes known to them.)

The guard duly obliges and the anxious prisoner awake looks upon the sleeping inmate to be condemned to die with certainty and mistakenly thinks that his own chances of survival have just risen from 1/3 to 50/50.

Meanwhile another guard overhears the name of one of the three condemned prisoners mentioned. He asks the first guard why he mentioned the name of one of the prisoners, and receives the answer that one of the prisoners asked who would die other than himself, but he doesn't ask which one of the three condemed uttered the question.

With the knowledge of what the question was but not which of the three prisoners asked it, he has a 50/50 chance of of guessing the survivor.

The prisoner with the knowledge of the question asked and that it was himself who asked the question, he still only has a 1/3 chance of survival.

To me, this isn't mathematics or probability, but pure logic.

Regards,

Steve.

[This message was edited by Steven Toy on Mon 19 April 2004 at 3:41.]
Posted on: 18 April 2004 by Minky
quote:
Originally posted by Matthew Robinson:
So who is right, Mulder or Scully?



I'll have a stab.

If it were truly random, each door would have a 1:2 chance of yielding a red. The chance of two people getting the same colour simultaneously is 1:4. This, together with the fact that the same doors on the same numbered boxes always yeild the same colour tends to say that the boxes ARE individually programmed.

If the boxes are individually programmed there would be 9 possible combinations of doors that could be opened :

Scully/Mulder

1/1
1/2
1/3
2/1
2/2
2/3
3/1
3/2
3/3

Three of these combinations (1/1,2/2,3/3) give a 1:1 chance of getting the same colour. The other 6 give a 1:4 probability, so the probability of getting the same colour at random from pre-programmed boxes would be (6/9*1/4) + (3/9*1) = 18/36 or 1:2.

So I'll put me money on Scully. Now PLEASE tell me I'm wrong, and why.
Posted on: 19 April 2004 by blythe
quote:
Originally posted by Minky:
Blythe,

It was very thought provoking. I must admit that I read it with a sense of impending doom because at any moment it could easily have drifted into "about a boy" style mawkishness. Thank god that Haddon never allowed this to happen (although the last page jarred a bit in this respect).

It's interesting that Oliver Sacks was quoted on the front cover because I got some of the same visceral response reading this book as his "Awakenings". Behind things that most of us dismiss as "easy" lie extremely delicate and complicated mechanisms. I'm not sure that dwelling on this in real-time is constructive but it could help with things like tolerance and compassion.

Disturbingly, I felt myself identifying with Christopher on some level. I thought his analysis of R&R was spot on.

What did you think ?

Like you, I found it very thought provoking.
At times I felt slightly uncomfortable when I found myself laughing; was I laughing at someone's misfortune? I also found myself smiling at some of the apparent logic and of course the times such as "the policeman put his hand on my shoulder, I don't like anyone touching me, so I hit him" I couldn't help myself and howled with laughter, having read his feelings earlier and got to know the character to some extent.
I felt angry when I learned that the father had lied to Christopher, but laughed at the logic of Christopher and the way he handled it.
Certainly a book I'll never forget.

Computers are supposed to work on 1's and 0's - in other words "Yes" or "No" - why does mine frequently say "Maybe"?......
Posted on: 19 April 2004 by matthewr
Minky,

"So I'll put me money on Scully. Now PLEASE tell me I'm wrong, and why"

You are most of the way there in that you are trying to solve the problem in the correct way: but you were 10 yards out on the wing with only a 10st flat footed Englishman to beat you then stepped out of bounds!

There are nine cases 1,1; 1,2; 1,3, etc. Thinking about what happens in the case of a box pair with a single "program" (say, 1=blue, 2=blue, 3=red) would be a good place to start.

BTW Einstein couldn't work this out (or at least failed to realise it) and the bloke who did was eventually awarded umpteen Physics medals and a Nobel nomination.

Matthew
Posted on: 19 April 2004 by Minky
OK, maybe I was wrong to mix programmed with random.

I'm not sure if every box has to have the ability to flash either red OR blue depending on which door is opened.

If every box does have the ability to flash either red or blue (no programmed BBB or RRR) we can cross the door matrix with the colour matrix. Assume that every program has one of one colour and two of the other (RRB, RBB, BBR, RBR, BRB BRR). Take RRB :

1(R)/1(R)
2(R)/1(R)
3(B)/1(R)
1(R)/2(R)
2(R)/2(R)
3(B)/2(R)
1(R)/3(B)
2(R)/3(B)
3(B)/3(B)

Gives 5/9 - pretty close to 50% - and all colour combinations should give the same result.

Bugger. I've just remembered that there were 10,000 iterations Frown

[This message was edited by Minky on Mon 19 April 2004 at 10:55.]
Posted on: 19 April 2004 by matthewr
Minky,

You were *so* close.

"Bugger. I've just remembered that there were 10,000 iterations"

Exactly. If you do a lot of iterations 5/9 becomes significiant and the fact that Scully found she agreed with Mulder 50% of the time means that the machines cannot be programmed and must be truly random.

I.e. For a given program there are generally 5 of the nine cases where Mulder and Scully would see the same colours. e.g. for BBR we have 1,1; 2,2; 3,3; 1,2 and 2,1. In addition there are programs such as RRR and BBB which always give the same colour and so increase the discrepency.

This idea comes from Quantum Physics and is a statistical test called Spin Inequalities discovered by John Bell. It's a technique that allows us to prove that certain complementary particles can "communicate" instantaneously over arbitary distances -- a process Einstien refused to beleive and called "spooky action at a distance".

Matthew
Posted on: 19 April 2004 by matthewr
PR,

You appear to be talking bollocks.

Matthew
Posted on: 19 April 2004 by matthewr
PR,

I don't understand any of it. It reads like utter bollocks to me. I assume its some kind of subtle joke but I must confess it has completely passed me by.

"Ask you friend in the financial markets whether his choice in similar investments is based on mathematical logic (50/50) without time value being a component?"

He would think I have gone mad and revise his previous impression of me as a moderately intelligent person.

Matthew
Posted on: 19 April 2004 by matthewr
Er, I actually gave my answer in replying to Minky.

Mulder is right because they only agreed 50% of the time when opening doors randomly. If it were pre-programmed one would expect to find more than 50% agreement as explained previously.

Matthew
Posted on: 19 April 2004 by matthewr
PR,

I have no idea what you are on about but my gut instincts are definiitely saying bollocks. FWIW the Mulder/Scully thing has been proved in "real life experiements" although I have no idea if there was some kind of "Er, like, have you considered TIME" footnote somewhere along the way.

If I mentioned "Cox Ross Binomial fair value models" (whatever that might be) my friend would probably laugh and accuse me of trying to impress girls by using complicated sounding phrases I didn't understand.

Matthew
Posted on: 19 April 2004 by JeremyD
...Returning to the original problem:

The argument that the contestant should switch seems to be based on the assumption that the host's opening a door does not change the probability of of the car being behind the contestant's door. The easiest way of seeing that this is not the case, IMHO, is by listing the contingencies:

(c = car, g = goat)

contestant's door.. c... g... g
host's door........... g... c... g
remaining door...... g... g... c

Beforehand, each of the outcomes represented by a column is equally likely (with probability 1/3).

However, we know the host does not choose the car, so the middle column is excluded. This leaves us with two equally likely outcomes (with probability 1/2).

Hence, the contestant switching doors makes no difference to their likelihood of being right.

At least, as far as I can see... Wink

[This message was edited by JeremyD on Mon 19 April 2004 at 19:17.]
Posted on: 19 April 2004 by matthewr
PR

"Because you don't understand what I'm saying doesn't make it bollocks"

No, but the poorly written and confusingly expressed nature of your posts made it seem like it might be bollocks.

"one would would expect (based on your 50% chance), that 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 would exist in equal numbers when calculating a million decimal places"

I don't think anyone would expect them to be exactly equal -- rather one would expect them to be something close to equal depending on exactly how random Pi actually is. Whether this is random enough would depend on the tolerances required for a particular example.

"It is more than probably that Minky's real-life answer in the Mulder and Scully case would have been correct if the number of matching colours exceeded 50%. Which is more than possible through random selection?"

I am a little confused again so I shall re-iterate what I believe to be the correct answer:

-- If Scully is correct and the boxes are pre-programmed you would expect a result of around 5555 matches out of 10,000 iterations.

-- If Mulder is correct and the boxes are truly random then you should expect around 5000 iterations.

The difference between 5555 and 5000 over 10,000 runs I take to be statistically significant. If it isn't then replace 10,000 with 100,000.

Does that answer you question?

"As you mentioned 50% was their finding (unlikely in real-life) you are faced with a language test rather than a mathematical one"

This actual experiment does give an answer of 50% (within experimental and statistical error -- so 50.01%, 90.98%, etc). It looks like Maths to me.

Matthew
Posted on: 19 April 2004 by jayd
Ok, I'll play.

Seems to me the arguments favoring the switch assume something - namely, that "goat = goat", i.e., that goat is one choice. Suppose it's brown goat/grey goat, or billy goat/nanny goat:

Intitial choice | Revealed
Billy | Nanny ->>> Switch wins
Nanny | Billy ->>> Switch wins
Car | Nanny ->>> Stick wins
Car | Billy ->>> Stick wins

And, we're back to 50/50.

Saying that the like items are "degenerate" (in the strictly mathematical sense) works for quantum mechanics; we can't say "this electron" or "that electron", just "electron". Not sure it applies to goats, though... I mean, I'm sure they know the difference.Wink

Suppose it all depends on how ( or how well) the problem is defined.
Posted on: 19 April 2004 by JeremyD
The goats in my post above are degenerate but still give the same 50/50 outcome because there are two goats in each of the three possible outcomes. Using two distinguishable goats simply gives two versions of each of these outcomes. So the distinguishability of the goats is irrelevant, AFAICT.
Posted on: 19 April 2004 by jayd
quote:
Originally posted by JeremyD:
The goats in my post above are degenerate but still give the same 50/50 outcome because there are two goats in each of the three possible outcomes. Using two distinguishable goats simply gives two versions of each of these outcomes. So the distinguishability of the goats is irrelevant, AFAICT.


Yes, I agree; one out of two is as good as two out of four. Smile

The flaw in the switcher gambit (which you and I both address) is that they assume the goats are not identical if I choose goat initially, leading to two "switch wins" scenarios (i.e., this goat or that goat), whereas when I pick car initially, the gambit assumes goat=goat, yielding only a single "stick wins" scenario (see John Channing's explanation below):
quote:
Stick | Switch
Goat | Goat Car ->>> Switch wins
Car | Goat Goat ->>> Stick wins
Goat | Car Goat ->>> Switch wins



As outlined above, the middle line should rightly have two possible values, representing "reveal goat 1" and "reveal goat 2". Only one of these is shown, therefore, the switch seems to win.

In other words, if lines 1 and 3 in John's scenario are distinguishable, we must have two distinguishable versions of line #2.

Seems like.
Posted on: 19 April 2004 by Markus S
quote:
Originally posted by Matthew Robinson:
A proof for those who insist on seeing numbers rather than relying on one's intuition:

1. Pick a door (A). Chance of being correct is 1/3

2. Chances of it being one of the other two doors (B, C) is 2/3.

[I assume we all agree to this point]

3. Host eliminates door B which, since it never has the car, has a car probablity of 0.

[This is the bit people get wrong and assume this has a probability of 1/3 -- omitting the fact that the host knows how to pick the one without the car]

4. Probability of it being behind door C is 2/3 - 0 = 2/3. Door A's probability remains unchanged at 1/3.

Matthew


Wrong. It's not allowed to change the point in time for which you calculate the probability.

1. Pick a door (A). The game show host opens another door to reveal the goat. So, in the light of the information you now have, the real probability of you picking the car with your first pick was 1/2, and not 1/3.

2. Whether you stick or switch, you still have a probability of 1/2.
Posted on: 19 April 2004 by Rasher
Markus is right.
You are looking at the probability of a single solution from a variable position. You can only equate the outcome as a probability from a fixed position - so the probability changes as the situation progresses.
Posted on: 19 April 2004 by Markus S
To put it another way, when the rules state in advance that one of the three doors will be eliminated by the host, and that this door does not have the car behind it, the real choice only ever is between two doors. The third is just decorum, there to amuse the audience and confuse the candidate and learned heads on internet fora.
Posted on: 19 April 2004 by matthewr
It seems to me that all the people who struggle with this simple problem have the benefit of some higher education in Maths or Statistics. People who scraped their A-level like myself don't have nearly the same problem Wink

So, answer me this. If you pick door A and I pick door B *and* door C I win 2/3 of the time yes?

If the host wrote down on a piece of paper which one of my doors that definitely has a goat and I still get to open my two doors I win 2/3 of the time. Yes?

If the host whispers in my ear which one to open first I, with this extra information, will open one door and have a 2/3 success compared to the person who picked door A.

How does the host whispering in my ear affect your chances of winning from a choice you made earlier?

Matthew

PS I think I might write a program to demonstrate the right answer.
Posted on: 19 April 2004 by JeremyD
Matthew,
having just read your answer more carefully I see you've assumed that the host deliberately open a door that he knows has a goat behind it. This is not stated in the question, so I don't understand why you have assumed it.
Posted on: 19 April 2004 by Paul Ranson
If you stick you will win in 1/3 of attempts.
If you switch you will lose in 1/3 of attempts.

It doesn't need to be more complicated than that, does it?

Paul