"Computer hands" questioned - and software choices
#1
Posted 2012-November-08, 16:21
Today, I stopped at our club for a few minutes and was quickly approached by one of our better players who was very unhappy due to the hands having far more voids, singletons, and long suits than normal - and he is aware that normal shuffling tends to give you flatter hands, but insists the hands played so far are clearly beyond what he's seen when using hand records at ACBL tournaments.
Is there a good way to reduce the fears of the club players who are thinking the computer program generating the random hands has gone completely mad? (I'm thinking of taking all the PBN files in the first month and doing a statistical analysis - but I'd like other suggestions on how to convince suspicious players nothing "funny" is going on.)
And what are the best computer programs for generating hands that are as close to purely random as possible? (Note that being usable by game directors who have very little computer experience is a plus.)
Bud
#2
Posted 2012-November-08, 17:12
If you really care about this, inspect the underlying code and see how it was implemented.
If you decide to try to verify hand records, I'd recommend the following:
Ask the player who is complaining what the "worst" feature of the randomly dealt hands.
Are there too many voids...
Are there too many 7+ card suits...
Chose one specific feature
At this point in time, you can run hypothesis test <on a new sample> and see whether the actual is deviating too far from the expected...
#3
Posted 2012-November-08, 18:35
I believe that the USA currently hold only the World Championship For People Who Still Bid Like Your Auntie Gladys - dburn
dunno how to play 4 card majors - JLOGIC
True but I know Standard American and what better reason could I have for playing Precision? - Hideous Hog
Bidding is an estimation of probabilities SJ Simon
#4
Posted 2012-November-09, 03:28
hrothgar, on 2012-November-08, 17:12, said:
Are there too many voids...
Are there too many 7+ card suits...
Chose one specific feature
At this point in time, you can run hypothesis test and see whether the actual is deviating too far from the expected...
Well if he is complaining about too many voids, most likely there really were too many voids in his limited sample.
The alleged problem could be that
1) The software makes too many voids in general. What you need to do, after having heard his complaint, is to generate say 100000 new hands and see if the abnormality can be reproduced.
2) The software made too many voids on this specific occation. You would have to make a very conservative test, then, since it is post-hoc. A p-value of 0.01 is not interesting. If you get 0.00001 but it can't be reproduced then I suppose you could try to find a bug in the code. Obviously this will be a lot more difficult than a statistical analysis.
#5
Posted 2012-November-09, 04:27
BudH, on 2012-November-08, 16:21, said:
No. Whatever you do or say, they'll never believe you.
#6
Posted 2012-November-09, 08:10
gnasher, on 2012-November-09, 04:27, said:
So lie. Next week just produce an abnormally flat set and you can say that look it was just chance. You shouldn't have tolook through more than ten or twenty sets to find a set with no voids and no seven card suits probably.
#7
Posted 2012-November-09, 11:35
There has always been this perception that computer generated hands are somehow different than hand dealt hands
and what is it there is not a thourough shuffling of the hands that accounts for it.
#8
Posted 2012-November-09, 15:34
hrothgar, on 2012-November-08, 17:12, said:
If you really care about this, inspect the underlying code and see how it was implemented.
If you decide to try to verify hand records, I'd recommend the following:
Ask the player who is complaining what the "worst" feature of the randomly dealt hands.
Are there too many voids...
Are there too many 7+ card suits...
Chose one specific feature
At this point in time, you can run hypothesis test <on a new sample> and see whether the actual is deviating too far from the expected...
Agree and disagree. Testing is not that hard.
The "too many voids test" is easy, as is the "too few singles test".
Average, about one of five boards have one or more void, one of five have no void of single.
If you frequently have more than 25% of the boards with one or more void (or more than 25% of the boards without any single or void)
your program is probably faulty.
Here are my stats for "super unbal boards" (P1V) and "super balanced boards" (PNS):
For 32 Boards :
P1V : prob. of exacty N boards with one or more void
PNS : prob. of exactly N boards without a single or void
N P1V TP1V PNS TPNS
0 0.16% 0.16% 0.06% 0.06%
1 1.11% 1.27% 0.50% 0.56%
2 3.86% 5.13% 2.04% 2.60%
3 8.65% 13.78% 5.31% 7.91%
4 14.05% 27.82% 10.03% 17.94%
5 17.63% 45.45% 14.63% 32.57%
6 17.77% 63.22% 17.16% 49.73%
7 14.79% 78.01% 16.61% 66.35%
8 10.35% 88.37% 13.53% 79.88%
9 6.19% 94.55% 9.40% 89.28%
10 3.19% 97.74% 5.64% 94.91%
11 1.43% 99.17% 2.94% 97.85%
12 0.56% 99.73% 1.34% 99.19%
13 0.19% 99.92% 0.54% 99.73%
14 0.06% 99.98% 0.19% 99.92%
15 0.02% 100.00% 0.06% 99.98%
16 0.00% 100.00% 0.02% 99.99%
PS: As someone else wrote, they wont believe it.
yvan
#9
Posted 2012-November-09, 16:04
bidule5, on 2012-November-09, 15:34, said:
First of all, you aren't actually doing a hypothesis test. (I don't see any mention of confidence intervals or the like)
Second, you are only testing one possible feature. What makes this hard is that there's an extremely wide number of ways in which a Dealer program can be flawed.
If you really want to test this, you need to do something equivalent to testing the properties of the random number generator which is a pain in the butt (even using tools like dieharder)
#10
Posted 2012-November-09, 16:46
I'd suggest taking the first month of deals and counting things like voids, singletons, and 7-card suits. Compare this to the expected number of each (and provide a confidence interval). Pin the results on the club bulletin board. Repeat after 2 months and 3 months.
It won't convince everyone, but some players will find it interesting. And, some players will recognize that their perception doesn't quite match reality.
Tim
#11
Posted 2012-November-09, 18:18
hrothgar, on 2012-November-09, 16:04, said:
Second, you are only testing one possible feature. What makes this hard is that there's an extremely wide number of ways in which a Dealer program can be flawed.
If you really want to test this, you need to do something equivalent to testing the properties of the random number generator which is a pain in the butt (even using tools like dieharder)
Agree, but I was just trying to show how easy flawed dealer can be detected.
yvan
#12
Posted 2012-November-09, 22:57
TimG, on 2012-November-09, 16:46, said:
I'd suggest taking the first month of deals and counting things like voids, singletons, and 7-card suits. Compare this to the expected number of each (and provide a confidence interval). Pin the results on the club bulletin board. Repeat after 2 months and 3 months.
It won't convince everyone, but some players will find it interesting. And, some players will recognize that their perception doesn't quite match reality.
Tim
Tim, that is what I plan on doing at the end of this month........
#13
Posted 2012-November-09, 22:58
Any other good dealing programs out there, especially if they are easy to use to make random hands?
#14
Posted 2012-November-11, 15:09
I would like to take a group of PBN files (for example, one month of them used at my local club), and do a detailed (not just balanced hand, 7+ card suits, number of singletons/doubletons) analysis of all the hands.
What is the best method (and software) to use to accomplish this?
And if needed, how do you most easily combine a few dozen PBN files into a single large PBN file?
[I note that Dealmaster Pro gives you a pretty good analysis, but it appears you can access that only if you just then generated the hands using Dealmaster Pro.]
#15
Posted 2012-November-11, 16:09
BudH, on 2012-November-11, 15:09, said:
I would like to take a group of PBN files (for example, one month of them used at my local club), and do a detailed (not just balanced hand, 7+ card suits, number of singletons/doubletons) analysis of all the hands.
What is the best method (and software) to use to accomplish this?
Necessary caveat: I think that you are on a fool's errand.
1. The sheer amount of time and effort necessary to do properly this is enormous
2. Doing this right requires a fair amount of programming skills to generate the sample sizes necessary
3. Regardless of the results that you come up with, the critics will still complain
With this said and done, here is how I would proceed:
Step 1: Map your bridge deals to a unique 96 bit number. See http://www.rpbridge.net/7z68.htm for an explanation
Step 2: Test the result setting of hands for randomness using "dieharder" or another equivalent test. http://www.phy.duke....l/dieharder.php
As an alternative, you could conduct a more standard hypothesis test using R or MATLAB. However, this requires formally specifying the hypothesis that you plan to test which can be rather difficult. (The reason that I originally suggested a small number of simple tests was to avoid trying to describe a more complex test case)
#16
Posted 2012-November-11, 19:09
hrothgar, on 2012-November-11, 16:09, said:
1. The sheer amount of time and effort necessary to do properly this is enormous
2. Doing this right requires a fair amount of programming skills to generate the sample sizes necessary
3. Regardless of the results that you come up with, the critics will still complain
With this said and done, here is how I would proceed:
Step 1: Map your bridge deals to a unique 96 bit number. See http://www.rpbridge.net/7z68.htm for an explanation
Step 2: Test the result setting of hands for randomness using "dieharder" or another equivalent test. http://www.phy.duke....l/dieharder.php
As an alternative, you could conduct a more standard hypothesis test using R or MATLAB. However, this requires formally specifying the hypothesis that you plan to test which can be rather difficult. (The reason that I originally suggested a small number of simple tests was to avoid trying to describe a more complex test case)
OK, perhaps not quite THAT detailed! But I do appreciate the information.
#17
Posted 2012-November-11, 19:27
BudH, on 2012-November-11, 19:09, said:
No worries.
On a more practical basis, I really do recommend engaging with the individual who is claiming that the hands are biased in some way.
Have this player specify what is "wrong" about with the past set of deals. Agree on three, or four, or five specific examples:
There are too many voids
There are too many seven + card suits
There are too many 5431 patterns
Document this all in advance.
At this point in time, you can use this specific set of complaints to design a set of hypothesis tests that you can perform on a new set of deals. These tests won't require all that large a sample size and you can probably run them all by hand.
#18
Posted 2012-November-11, 19:54