Join Bridge Winners
Simulations: dealing with deal

Introduction to Simulations

While in Toronto at the last NABC I stumbled upon the simlabs series of blog posts by John Torrey. I quite enjoyed them and it motivated me to revisit Thomas Andrews' deal program, which had been updated a few times since I had last used it (I last did stuff with deal 3.08 and this article was using deal 3.19). I wanted to explore a few of the same questions that John had raised about invite or game with a 6 card major opposite a strong nt, but also show how to use some of deal's statistical functionality. I'll do that by going through some of the same questions and investigations that John did, but exercising the deal program more and including the published scripts in the post.

First Hand

The initial constraints of the problem is W opens a strong 1nt and E wants to know what to do with:

SA 9 7 6 5 2HQ 6 4D4 2C6 4

Our goal is to investigate how often double dummy spades will make 10+ tricks (game is good), 9 tricks (we can afford to invite), or 8 or fewer (inviting costs) given West has a strong 1nt. The theory being if it is something like 45/45/10 then inviting is good (since even if we aren't perfect at knowing when to accept, we'll likely get better results by inviting). If it is something like 55/10/35 we might like to just blast game. If it is something like 10/45/45 we'd probably just want to sign off in 2. When John ran this hand, using the very simple 1nt constraint that West is balanced and between 15 and 17 points, he got:

10+ 14/48 = 29%

9 15/48 = 31%

8- 18/48 = 38%

(observantreaders will realize that this only adds up to 47/48 hands, I don't know if one hand got thrown out for being unrealistic or if something was miscounted or misreported)

I'll share the deal script I used to test this in a moment, but first to explain there are three main functions that are used by deal. The most important one is main. This is where all the restrictions on hands take place (like the W must have balanced 15/17). This method gets called over and over again with a candidate deal that you can either reject (doesn't count against the number of deals) or accept (a valid deal). The next one is write_deal. This gets called once for each valid deal. If this is missing, deal will just write the deal like normal. But you can hijack this function to do some statistical analysis, as well as to turn off the printing of hands. The last one is deal_finished. This method will be called only once after all of the hands have been dealt.

Here is the deal script:

east is "A97652 Q64 42 64"
proc write_deal {} {
global hands
global sp10p
global sp9
global sp8m
incr hands
set sptr [tricks west spades]
sp add $sptr
if {$sptr >= 10} {
incr sp10p
} elseif {$sptr <= 8} {
incr sp8m
} else {
incr sp9
}

# formatter::write_deal
# puts stdout "West makes $sptr in spades"
# puts stdout "================================="
}
deal_finished {
global hands
global handstry
global sp
global sp10p
global sp9
global sp8m

puts stdout "We tried $handstry hands to select our $hands hands"
puts stdout "There are [sp average] tricks on average in spades"
puts "There were 10+ tricks $sp10p times, 9 tricks $sp9 times, and 8 or fewer $sp8m times."
puts stdout "That works out to game [expr 100*double($sp10p)/$hands]% of the time, safe invite [expr 100*double($sp9)/$hands]% of the time, and 2 only [expr 100*double($sp8m)/$hands]% of the time"
}
set handstry 0
set hands 0
sdev sp
set sp10p 0
set sp9 0
set sp8m 0
main {
global handstry
incr handstry
reject if {![balanced west] || [hcp west] < 15 || [hcp west] > 17 }
accept
}

Note that I set the east hand at the top of the script to be the exact hand we want. I also have a number of global variables to track the various statistics I want to track. handstry is how many candidate hands did deal evaluate (counting the ones that get rejected). hands is how many successful hands were selected (gets used a lot to calculate percentages). The three variables sp10p, sp9, sp8m track how often we take 10+ tricks in spades, 9 tricks in spades, and 8- tricks in spades. The sp is a special statistical variable that deal provides and it tracks the number of tricks in spades so we can come up with an easy average. Each time through the deal we ask deal's double dummy solver how many tricks can west take with spades as trump ("[tricks west spades]") and use the answer from that to modify these counting variables. You may observe that I have commented out (the lines starting with '#') some print outs that show the hand (calling formatter::write_deal prints the hand) and my statistics, which I generally do for early short runs debugging my scripts. In deal_finished I calculate and output all the statistics I care about. Note if you want percentages (or other things with division) that you have to use the double() function or else you get integer math which leads to everything truncating to 0.

To run a deal script and capture the output you just need to do:

deal -i scriptFile.txt 100 > output.txt

I generally keep my script files and output files in different directories and name the output files with the script name and the number of deals. So for this one I did:

deal -i my/scripts/6majoroppstr1nta.txt 10000 > myoutput6majoroppstr1ntaD10000.txt

Note that the path symbols in the -i switch need to be the opposite direction to what they are on windows (which is what the post > were).

Results

Rather than the 48 runs that John was doing, I tried running 10,000 matching hands. I have a very old computer at home (nearly 10 years old), but 10,000 hands of this script took only around 20 minutes. The vast majority of this time is likely in the double dummy solving, but more complicated scripts and more complicated can take much, much longer (I've left a number of scripts running for most of the work day, and I've run a script over the weekend before). So how did my run compare to the 29%/31%/38% that John got?

We tried 168089 hands to select our 10000 hands
There are 9.0368 tricks on average in spades
There were 10+ tricks 3121 times, 9 tricks 4204 times, and 8 or fewer 2675 times.
That works out to game 31.21% of the time, safe invite 42.04% of the time, and 2 only 26.75% of the time

Hmm, seems like we nearly agree on game, but the invite is much more likely on my run than it had been on his. In fact if you were pretty good about deciding when to accept or not a 31/42/27 might well be a good invite.

Are 48 (or 10,000) deals enough?

Well 48 runs probably was not enough if 31 and 42 are so far apart and 27 and 38 are so far apart (if the numbers are only accurate +/- 11% probably hard to tell what to do). But is 10,000 enough?

One way to help tell is to run more tests. John included results for a second run of 48 and he got 11/48 make 4+, 22/48 make 3, and 15/48 make 2- which gives 23%/46%/31% which is also much closer in the 9 tricks to my results, but now the 4+ and 2- are off.

I ran 10 additional 10,000 runs to observe how my results of the same script changed due to nothing but the randomness of the sample.

Results

We tried 167815 hands to select our 10000 hands
There are 9.047 tricks on average in spades
There were 10+ tricks 3115 times, 9 tricks 4259 times, and 8 or fewer 2626 times.
That works out to game 31.15% of the time, safe invite 42.59% of the time, and 2 only 26.26% of the time
We tried 168667 hands to select our 10000 hands
There are 9.0474 tricks on average in spades
There were 10+ tricks 3122 times, 9 tricks 4245 times, and 8 or fewer 2633 times.
That works out to game 31.22% of the time, safe invite 42.45% of the time, and 2 only 26.33% of the time
We tried 166885 hands to select our 10000 hands
There are 9.0282 tricks on average in spades
There were 10+ tricks 3036 times, 9 tricks 4291 times, and 8 or fewer 2673 times.
That works out to game 30.36% of the time, safe invite 42.91% of the time, and 2 only 26.73% of the time
We tried 165679 hands to select our 10000 hands
There are 9.0471 tricks on average in spades
There were 10+ tricks 3107 times, 9 tricks 4285 times, and 8 or fewer 2608 times.
That works out to game 31.07% of the time, safe invite 42.85% of the time, and 2 only 26.08% of the time
We tried 167112 hands to select our 10000 hands
There are 9.0467 tricks on average in spades
There were 10+ tricks 3113 times, 9 tricks 4283 times, and 8 or fewer 2604 times.
That works out to game 31.13% of the time, safe invite 42.83% of the time, and 2 only 26.04% of the time
We tried 168272 hands to select our 10000 hands
There are 9.0505 tricks on average in spades
There were 10+ tricks 3179 times, 9 tricks 4209 times, and 8 or fewer 2612 times.
That works out to game 31.79% of the time, safe invite 42.09% of the time, and 2 only 26.12% of the time
We tried 166134 hands to select our 10000 hands
There are 9.0341 tricks on average in spades
There were 10+ tricks 3097 times, 9 tricks 4237 times, and 8 or fewer 2666 times.
That works out to game 30.97% of the time, safe invite 42.37% of the time, and 2 only 26.66% of the time
We tried 167180 hands to select our 10000 hands
There are 9.0457 tricks on average in spades
There were 10+ tricks 3146 times, 9 tricks 4233 times, and 8 or fewer 2621 times.
That works out to game 31.46% of the time, safe invite 42.33% of the time, and 2 only 26.21% of the time
We tried 168923 hands to select our 10000 hands
There are 9.0271 tricks on average in spades
There were 10+ tricks 3050 times, 9 tricks 4271 times, and 8 or fewer 2679 times.
That works out to game 30.5% of the time, safe invite 42.71% of the time, and 2 only 26.79% of the time
We tried 167058 hands to select our 10000 hands
There are 9.0459 tricks on average in spades
There were 10+ tricks 3135 times, 9 tricks 4235 times, and 8 or fewer 2630 times.
That works out to game 31.35% of the time, safe invite 42.35% of the time, and 2 only 26.3% of the time


As we can see the results are pretty consistent across all the runs. The number of tricks on average ranged from 9.0505 to 9.0271, the number of times 10+ made ranged from 31.79% to 30.36%, the number of times 9 tricks made ranged from 42.91% to 42.04% (the lower bound being in our original run), the number of times 8- tricks made ranged from 26.79% to 26.04%. So from this sample it seems like we are accurate to within a percent or so when we run 10,000 runs of this script. A rough rule of thumb for these simulations is you need at least 4 times as many samples to get 2 times as accurate an estimate. So 10,000 is 208 times bigger then 48 so should be around 14 times as accurate.

We probably don't need any more samples, as once you start getting down to a single percent the other errors are probably much bigger than your sample errors. The other errors in this case would be:

1. How well did we constrain the West hand?

2. How well did we constrain the opponents hands (wouldn't they haveinterfered with some of them)?

3. How accurate is double dummy play anyways? Maybe some of the double dummy 10 tricks go down, or some of the double dummy 9 tricks usually makes on normal leads and defense.

We can deal with 1 and 2 by using more advanced scripts (a little slower to run), and deal with 3 somewhat by doing single dummy analysis instead of double dummy analysis (much slower to run). I'll do at least 1 and 2 in a future post.

Other hands

In John's linked post he looked at other hands as well with 48 hand runs. I'll revisit them with scripts that are identical to the one I've quoted in this post, except the east is "" line is changed to the hand included here.

SQ 9 7 6 5 2HA 6 4D4 2C6 4

John found 21 10+, 20 9, and 7 8-. That is 44%/42%/15%. I found:

We tried 159838 hands to select our 10000 hands
There are 9.1954 tricks on average in spades
There were 10+ tricks 3618 times, 9 tricks 4388 times, and 8 or fewer 1994 times.
That works out to game 36.18% of the time, safe invite 43.88% of the time, and 2 only 19.94% of the time

SQ 9 7 6 5 2H8 6 4DA 2C6 4

John found 13 10+, 23 9 and 12 8-. That is 27%/48%/25%. I found:

We tried 158351 hands to select our 10000 hands
There are 9.1134 tricks on average in spades
There were 10+ tricks 3380 times, 9 tricks 4181 times, and 8 or fewer 2439 times.
That works out to game 33.8% of the time, safe invite 41.81% of the time, and 2 only 24.39% of the time

SK 9 7 6 5 2HK 6 4D4 2C6 4

John found 18 10+, 19 9 and 11 8-. That is 38%/40%/23%. I found:

We tried 161102 hands to select our 10000 hands
There are 9.1885 tricks on average in spades
There were 10+ tricks 3809 times, 9 tricks 4047 times, and 8 or fewer 2144 times.
That works out to game 38.09% of the time, safe invite 40.47% of the time, and 2 only 21.44% of the time

SA 9 7 6 5 2HJ 6 4DJ 2CJ 4

John found 12 10+, 19 9, and 17 8-. That is 25%/40%/35%. I found:

We tried 180805 hands to select our 10000 hands
There are 9.036 tricks on average in spades
There were 10+ tricks 2953 times, 9 tricks 4420 times, and 8 or fewer 2627 times.
That works out to game 29.53% of the time, safe invite 44.2% of the time, and 2 only 26.27% of the time

SA 9 7 6 5 2HQ 6 4D6 4 2C4

John found 23 10+, 12 9, and 13 8-. That is 48%/25%/27%. I found:

We tried 168223 hands to select our 10000 hands
There are 9.2152 tricks on average in spades
There were 10+ tricks 4064 times, 9 tricks 3600 times, and 8 or fewer 2336 times.
That works out to game 40.64% of the time, safe invite 36.0% of the time, and 2 only 23.36% of the time

SA 9 7 6 5 2HJ 6 4DJ 4 2C4

John found 16 10+, 24 9, and 8 8-. That is 33%/50%/17%. I found:

We tried 170364 hands to select our 10000 hands
There are 9.1685 tricks on average in spades
There were 10+ tricks 3765 times, 9 tricks 3893 times, and 8 or fewer 2342 times.
That works out to game 37.65% of the time, safe invite 38.93% of the time, and 2 only 23.42% of the time

SJ 9 7 6 5 2HA 6 4DJ 4 2C4

John found 22 10+, 18 9, and 8 8-. That is 46%/38%/17%. I found:

We tried 156679 hands to select our 10000 hands
There are 9.3103 tricks on average in spades
There were 10+ tricks 4242 times, 9 tricks 3885 times, and 8 or fewer 1873 times.
That works out to game 42.42% of the time, safe invite 38.85% of the time, and 2 only 18.73% of the time

SJ 9 7 6 5 2HA 6 4D9 4 2C4

John found 13 10+, 23 9, and 12 8-. That is 27%/48%/25%. I found:

We tried 147994 hands to select our 10000 hands
There are 9.0422 tricks on average in spades
There were 10+ tricks 3167 times, 9 tricks 4043 times, and 8 or fewer 2790 times.
That works out to game 31.67% of the time, safe invite 40.43% of the time, and 2 only 27.9% of the time

It is kind of amazing to me that changing the lowly diamond J to a 9 drops more than 10% off the number of times game can be made!

6 card major 6 HCP, invite or not based on shortness?

In the end John concludes that to invite or not (when you have a 6M and 6 HCP) should be largely based on if you have shortness or not (although A and K are better than Q and J, especially if the controls are outside your suit). He runs two last tests where the East hand is random, with a 6 card spade suit and 6 HCP andvariesif they have shortness (singleton or void).

In the run where there is no shortness: 10+ tricks is made 13 times, 23 made 9, and 12 made 8- tricks. Which is 27%/48%/25%. But in the run where there is shortness: 27 made 10+ tricks, 15 made 9 tricks, and only 6 made 8- tricks. Which is 56%/31%/13%.

When I ran the no shortness test I had no line with East has and a new main function, the rest unchanged. The new main function was:

main {
global handstry
incr handstry
reject if {![balanced west] || [hcp west] < 15 || [hcp west] > 17 }
reject if {[hcp east] != 6 || [spades east] != 6 || [clubs east] < 2 || [diamonds east] < 2 || [hearts east] < 2}
accept
}

For the hand with shortness I again had no East has (or at least I did after I debugged why my run was taking forever!) and a new main function, the rest unchanged. The new main function was:

main {
global handstry
incr handstry
reject if {![balanced west] || [hcp west] < 15 || [hcp west] > 17 }
reject if {[hcp east] != 6 || [spades east] != 6 || ([clubs east] > 1 && [diamonds east] > 1 && [hearts east] > 1)}
accept
}

These scripts took just under 30 minutes to run 10,000 test runs of and my results were for no shortness:

We tried 180448707 hands to select our 10000 hands
There are 9.0348 tricks on average in spades
There were 10+ tricks 3068 times, 9 tricks 4233 times, and 8 or fewer 2699 times.
That works out to game 30.68% of the time, safe invite 42.33% of the time, and 2 only 26.99% of the time

For shortness:

We tried 97693109 hands to select our 10000 hands
There are 9.5806 tricks on average in spades
There were 10+ tricks 5438 times, 9 tricks 3185 times, and 8 or fewer 1377 times.
That works out to game 54.38% of the time, safe invite 31.85% of the time, and 2 only 13.77% of the time
Note that the number of hands tried is much, much higher when we are randomly dealing the East hands but then subjecting it to a narrow condition like 6 spades and 6 hcp.
So my shortness numbers are very similar to John's, but my no shortness are slightly more favorable with less 9 tricks exactly, but more games. But indeed shortness is really important, and 6 HCP with a 6 card major and a singleton or void is going to make game quite a lot opposite this definition of a 15-17 major.

Conclusions

Simulations are indeed interesting to look at different bridge questions. The deal program by Thomas Andrews is free and highly useful (and includes statistical analysis and double dummy analysis too). John's questions about 6 HCP 6 spade hands had some interesting findings for how much in/out mattered and how much shortness mattered. But to get accurate answers with a simulation like this you need many more hands than 10 or 48 or 100. Something like 10,000 gives a lot better answer, and on modern computers it is trivial to run this many runs (even on less modern computers it isn't too bad).
In a future post I'll explore what happens if you put constraints on the opponents hands and also include a more realistic 1nt opening style (at least for me) that allows some upgrades and semi-balanced shapes.

9 Comments
Getting Comments... loading...
.

Bottom Home Top