I’ve been using Thomas Andrews’ Deal hand generator for some time now.  It has taught me things about bridge that I would not have learned without it.  As I’ve gained experience with the program and its output options, I’ve developed methods that I find useful and practical.  Bridgewinners is an ideal place to share simulation methods and results, and it is logical to start this process with methods; it seems backwards to put results out there without explaining and evaluating the process that produced them.  This first piece has two main purposes.   First, if my methods are of value I can share them with others who might want to do their own investigations.  Second, I can address some of the questions about whether my approach to simulation leads to reliable conclusions.

Along the way there are real bridge hands and some small surprises.

The Method

(If you just want the “bridge part” you can skip to the second copy ot the bridge hand.  I don’t blame you.  Skip-to-the-hands can work later as well.)

Most of my tools are available for free download on the Internet.  The exception is the program, Dealmaster Pro, which I had been using to generate hands and hand records for club duplicate play.  It makes well-formatted output and includes double dummy analysis; I use both these features in my simulation approach.

The first essential, though, is the Andrews Deal program, which can be seen and downloaded at http://bridge.thomasoandrews.com/deal/.  To use the program after downloading and installing it, you need to make two files (I use Notepad to do this; the process on a non-Windows computer may be different):

1. A text file with rules defining the deals the dealer will accept or reject.  The rules are specified in a programming language peculiar to the Deal program.  (Deal works by randomly creating zillions of random deals, and then rejecting almost all of them.)
2. A “batch” file that invokes the Deal program and passes information to it (like the name of the text file above) and specifies the format and destination of its output.  The information can include the exact cards to be held by one of the hands.

An example  will clarify the above.  I wanted to evaluate
♠ A 9 7 6 5 2 ♥ Q 6 4 ♦ 4 2 ♣ 6 4
as an invitation in response to a 15-17 notrump.  My “rules” file, 1NTWest.txt, looks like:

main {
 reject if {![balanced west] || [hcp west] < 15 || [hcp west] > 17 }
 accept
}

It says we reject deals where West is unbalanced or has less than 15 or more than 17 hcp, and accept the rest.  We could have refined the specifications in many ways – for example, to require that West have a certain number of spades, or to put restrictions on the North or South hands.

The batch file, which I called AQ22.bat (The “bat” extension is required but the file name is whatever you make it) is:

deal 48 -E "A97652 Q64 42 64" -i 1ntWest.txt -i pbn > C:dmproAQ22.pbn

This tells Deal to create 48 hands, with East as specified, to include the 1ntWest.txt specification above and a Deal-supplied format specification called pbn, and to put the resulting output in a file called AQ22.pbn in the C dmpro directory, which is where Dealmaster Pro looks for input. 
When the file is created, I open the Dealmaster Pro program, import the hands, and print them with the double-dummy analysis supplied with Dealmaster Pro.  They print 24 deals to a page.  (This explains why I make 48 hands in the first place: I get a manageable two pages per run.)  I use a print-to-PDF utility (PDF995) to make electronic files out of the “printout.”  This saves paper and allows me to share the output with others, and make hard copy later if I need it..

The next step is to make some bridge-sense out of the printout.  The first stage – and frequently the last – is to count the number of results in the relevant categories.  The example hand is

♠ A 9 7 6 5 2 ♥ Q 6 4 ♦ 4 2 ♣ 6 4
(Welcome back, nonsimwonks!  A brief review: we want to know if the hand makes a good game invitation after a 15-17 1NT.  We made 48 simulated deals.)  At double dummy, 14 hands made four or more, 15 made three, and 18 made two or less.  (If you’re curious, none made more in notrump than in spades, and  one made exactly three in both spades and notrump.)  If the case for inviting looked good on these raw numbers, I might look further to see how many of the companion hands would accept the invitation: probably some of the games would not be bid, and some of the hands that made three would bid the game and go down.  The key first-glance numbers of interest are how many would make game (the upside) and how many would go down trying for it (the downside).

Does the Method Produce Reliable Results?

When I first started using the Deal program, I simply let it print out the two companion hands, and then did my own single-dummy analysis of what they could make.  I soon gave this up as hopeless: This hand had no chance against the expected lead (but is cold against others); that one was ninety percent;  some required several minutes of analysis – which then resulted in a percentage not too far from 50-50.  How was I to aggregate all these laboriously calculated percentages?  It didn’t matter, really, because I didn’t have the time to analyze enough hands.

The method I now use relies on the power of numbers.  Suppose you are shopping for an outdoor thermometer at a home-supplies store.  You want one that is accurate.  There are several of them on display; they all show different temperatures, and you have no way to know what the actual temperature is.   If you figure the rough average of the temperatures they display, you will do very well by picking one that displays that average temperature.   Transposing to bridge, one hand proves almost nothing, but in aggregate they tell us a useful story.

If we accept the principle that there is value in numbers, there are still several questions:

1. Are 48 deals enough?  Maybe the next 48 would be totally different.
2. Is the simulation biased by selection rules?  Several of the deals in the above example have opposing hands that would have overcalled or otherwise interfered, before our responder had the option of inviting or not.  Maybe those are the hands where bad breaks doom our otherwise sensible contract, and eliminating them would alter the odds.  Similarly – when Deal says a hand is “balanced,” it has no five-card major or six card minor, and is not 5-4-2-2.  All these shapes occasionally are opened 1NT in the real world. I’ve even heard rumors of the occasional singleton.
3. Is double dummy analysis even useful?  We’ve all seen “hopeless” contracts that become cold once the opening lead is made, or ones where we know it’s cold double dummy, but there is so much to guess that our practical chances are less than even.  Can we really create solid conclusions by aggregating questionable pieces?  The real question is whether, on average, double dummy results are biased towards more declarer tricks or towards fewer declarer tricks than would be taken in real life, or neither.

Rather than attempt theoretical or “trust me, I’ve done this a lot” answers to these questions,  I’ll continue working with the example hand above, and related hands.  One way to test the consistency of the simulation is simply to do another run with the same input.  On a second run, we see 11 hands making 4♠ (14 the first time), 22 making three (15), and 15 less than three (18).  The numbers are different but the message is the same: inviting with this hand has limited upside and significant downside.
Case Studies

Let’s play with the hand in small ways and look at the simulation results.  I encourage you to guess at what the simulation will say, based on your evaluation of the hands.  For starters, I’ll swap the ace and the queen, giving

♠ Q 9 7 6 5 2 ♥ A 6 4 ♦ 4 2 ♣ 6 4
The results here merit a small “wow!”   Now 21 hands make four, 20 make three, and only 7 make less.   I expected an improvement, but got much more than I expected.  Maybe I have learned something useful.  I might re-do a run when a result seems to be an outlier; in this case I’ll try a very similar hand that might be slightly worse:

♠ Q 9 7 6 5 2 ♥ 8 6 4 ♦ A 2 ♣ 6 4

Now there are 13 games, and 12 that do not make three.   Quite a bit worse than the previous, which is beginning to look like an outlier: I re-run the prior case just to see if there’s a significant difference.  The number of games made is down to 17 and the number of sets in three is 8, still a low number.  The numbers differ but the message is consistent, and I have to think that the position of the ace makes a real difference, even between Axx and Ax.

Some fast passes at other variations:

♠ K 9 7 6 5 2 ♥ K 6 4 ♦ 4 2 ♣ 6 4

18 games, 11 hands down in three.   Down the middle.

♠ A 9 7 6 5 2 ♥ J 6 4 ♦ J 2 ♣ J 4

12 games, 17 down in three, as we’d expect.
♠ A 9 7 6 5 2 ♥ Q 6 4 ♦ 6 4 2 ♣ 4

23 games making, 13 down in three, and I’m confident that swapping the honors would produce quite a bit more improvement.  This is good enough that I’ll look to see how many games will actually be bid and how these will do.  This starts out with discarding the deals where the opposition would surely compete before our responder has to decide whether to invite or not.  There are 16 of these; of the remaining hands 14 would bid and make game, none would bid it and go down, and 8 would be down in three.  (My accept-or-not decisions here were admittedly fast and imprecise.  For a serious study I would take more time with this.  It’s unlikely that two analysts would agree on every hand in an exercise like this, but also unlikely that their disagreements would alter the overall conclusions.)  Two hands that I thought would bid game and fail are in the 16 where the opponents compete.  The expectation with this kind of follow-up is that things will get worse, because some of the games that make will not be bid and some that are bid will fail: the downside will grow and the upside can’t.  In this case things stayed good because the thrown-out hands had several of the negative cases.

♠ A 9 7 6 5 2 ♥ J 6 4 ♦ J 4 2 ♣ 4

16 games, 8 down in three.  24 hands where partner may accept and go set when we should have stopped in three; this makes inviting unattractive even when we are twice as likely to make four as to fail in three.  Most of us would not invite in real life.

♠ J 9 7 6 5 2 ♥ A 6 4 ♦ J 4 2 ♣ 4

22 games, 8 down in three.  Just checking.

♠ J 9 7 6 5 2 ♥ A 6 4 ♦ 9 4 2 ♣ 4

13 games, 12 down in three.  More checking.

Many simulations do not specify the exact cards for any hand.  This usually goes with more restrictions on what deals will be accepted.  The runs above suggest that a singleton in a 6-point hand with a 6-card major can be a decisive advantage.  I ran two simulations: both had random 1NT openers like the ones in the above sims, and both had random responding hands with 6 spades and 6 hcp.  In the first set no responding hand had a singleton or void: in the familiar stats, 13 hands made game and 12 made two or less.  In the second set all the responding hands had a singleton or void: now 27 made game and 6 made two or less.  These numbers will not surprise you if you have been paying attention – and probably even if you have not.

Net Net

1. Are 48 deals enough?  If you’re like me, you are beginning to see these results as part of a fabric of related possibilities.   There can be runs of 48 where the results cluster to one side or the other, but these “stick out” and invite further investigation.  Each related result is supporting evidence for the validity – or at least, the consistency – of the whole fabric, and the fabric is persuasive evidence that a particular result is to be credited.
2. Is the simulation biased by selection rules?  I haven’t really addressed this yet.  Certainly any published simulation result should also publish its selection rules.  One reason I’ve been looking at maybe-invitational hands is that I want to do simulations in which I exclude responding hands that would invite on their own.   I feel confident that if I exclude hands with a six-card major and six hcp and a singleton or void, I will have made a good practical choice for this category of hand.  For what it’s worth, my experience says that seemingly small differences in selection rules can tip the balance in the result, when the issue is close in the first place.
3. Is double dummy analysis even useful?  I looked at the above hands (6 hcp maybe-invitations with 6 spades) because they represented a borderline case.  The results shed some interesting light on just where that border is.  You could say that the simulations confirmed that these hands are on the edge, but I would also say that the simulations are validated by the fact that they conform to our general knowledge about these hands.  If you look at the deals in detail, you see excellent contracts down on diabolical breaks and horrible ones that make because God put the opposing cards just right.  Some would make because of unfortunate defense, and some would go down because the declarer guesses wrong.  But just as in real life, better contracts win in the long run.  I’ve done sims where I “wanted” (because of a theory I was enamored of) good results, and ones where I “wanted” the opposite.  Simulations pleased me no more in one case than the other.

I have saved (as PDF files) all the deals I reported on in this study, and I can send the files to readers curious about them.  Email johnctorrey@aol.com. 
My next simulation exercise will look at another category of possible invitations.