Testing a Hypothesis
(Page of 3)

Determining whether or not a pair is likely to be signaling from only the hands is a difficult process. An absolute determination can never be made, since a pair might have simply hit a lucky streak. But with the proper procedure one can determine the likelihood of an honest pair achieving the results by chance, and from this information one can then make a judgment call about the pair in question.

Let's suppose we form the hypothesis that the partner of the opening leader is signaling for an opening lead. What now? Finding a bunch of hands where an apparently outlandish opening lead struck gold might be an indicator that something is going on, but that is all. If one selected the right hands from a pair's history, it would be easy to select hands where the pair made a non-obvious opening lead which hit partner's suit. Every pair will have such hands. But an honest pair will also have plenty of hands where they make a non-obvious opening lead which strikes mud. In order to do a meaningful study, it is necessary to examine all the hands over a given match, tournament, or whatever data base you choose to examine where the pair was on opening lead.

Now that we have gathered all the hands over the period we are examining, then what? Most of the hands aren't going to matter. The partner of the opening leader might not have a desire for a particular lead, so he won't be giving a signal. The opening leader might have a normal lead of the suit which would be signaled for anyway. The opening leader might have a hand where he would lead a different suit even if he got a signal.

The only hands which matter are hands which fit the following conditions:

The partner of the opening leader would probably be signaling for suit A.

The opening leader would not find suit A an attractive choice with no signal.

After getting a signal, suit A now becomes the likely lead of choice.

If a hand fits these conditions, it is a relevant hand. For this hand, the opening lead choice may be significant. If the leader chooses suit A, the signaled for suit, that is an indication of guilt. If he chooses another suit, that is an indication of innocence.

For example, consider this hand which involved the German doctors. The bidding all by the opponents was: 1(strong)-1(0-7);1-3;4. The partner of the opening leader held: 10853 J3 AQ43 J85. This is a hand on which it seems likely that he would signal for a diamond lead. The opening leader held: 9742 96 KJ76 Q103. Any lead could be right, but a diamond lead isn't particularly attractive. With the help of a signal for a diamond lead, the diamond lead becomes very attractive. Thus, this hand is clearly a relevant hand.

It is important to understand that if the player does find the diamond lead that is not a proof of guilt. While the diamond lead isn't particularly attractive one could see how an honest player might stumble onto the lead. Also, failure to lead a diamond is not a proof of innocence. Partner might not have signaled for a diamond lead. The leader might have his own ideas. Or the signal simply might have been missed. One hand doesn't prove anything. Only by examining all the hands can we come up with some meaningful conclusions.

Now we have all the relevant hands. On some of them the pair will have made the "guilty" lead, and on others they will have made the "innocent" lead. The percentage of "guilty" leads is the critical factor. There is a mathematical formula which can determine the probabality of an honest pair hitting that percentage or higher of "guilty" leads by chance. We can then make a judgment call about the pair in question. That judgment call will depend upon our previous feelings about the pair. Suppose the chance that an honest pair would produce these results or better is 1 in 1000. If we were previously convinced the pair was honest we would probably shrug it off. Thousand to one shots do happen, and probably this was just by chance. But if we were previously suspecting the pair, then these results would be a strong confirmation (not a proof) that our suspicions were correct.

Before Boye opened his website or made his posting on Bridgewinners that his team was vacating previously won titles with Fisher-Schwartz, he had contacted me privately asking me to examine the hands from the Spingold. He gave me two hypotheses. One was that they were signaling club length. The other was that they were signaling presence or absence of a heart honor.

I tested both of these hypotheses in the manner I previously described. I found that the hypothesis about club length was clearly inaccurate. The hypothesis about heart strength might be accurate. I would have to look at that more closely as there were a lot of bridge judgment decisions involving this knowledge which makes it very difficult to determine if the hypothesis is correct or not.

I also tested the hypothesis that they were signaling for opening leads. This is an obvious hypothesis to test, as the lead to be signaled for and the action taken are very clearly defined. My conclusion was that there was not an indication that they were signaling for opening leads, at least not in the Spingold. This is a subjective opinion, of course, and others may disagree with me. Keep in mind that I was looking only at the Spingold hands. I would guess that if I had done the same test over their hands from the 2014 European championship that I would come to a different conclusion.

Boye posted that he had come to his conclusions the day after the Spingold quarter-final match by examining the vu-graph results from the other table (he didn't play any of the quarters against Fisher-Schwartz that match). When I read that, I decided to take a very close look at that match to try to determine what Boye had seen.

I noticed that there appeared to be an unusually high number of defensive leads after trick 1 problems for Fisher-Schwartz. It also appeared that on several of these they had made a questionable choice, and that choice was almost always the lead partner would have signaled for.

From this I formed the following hypothesis: Fisher-Schwartz were signaling for defensive shifts after the opening lead. This seemed to me a logical hypothesis. Defensive shifts are very important information. Also, since the screen would be up it would be relatively easy to give a visual signal which would not appear obvious if an observer didn't know exactly what to look for.

The next step was to collect the data. I chose to examine only hands from the Spingold. From the vugraph archives, I found that they were on vugraph one quarter from the round of 32, one quarter from the round of 16, 3 quarters from the quarter-finals, 3 quarters from the semi-finals, and 2 quarters from the finals. I chose to exclude the last quarter of the semi-finals, as they were up 50 IMPs going into that quarter so they might not bother signaling anything. This left me 9 quarters, 135 deals to work with. Of these deals, I found 45 where Fisher-Schwartz were on defense and were on lead early in the hand. Some of these involved trivial plays, some more difficult decisions. Since completeness is important here, I looked at all of them.

Now I had to decide which are the relevant hands where a signal has a good chance of swinging the decision. I could have done this based only on my own personal judgment, but my judgment might be biased and cause some hands to be considered relevant which shouldn't be and other hands to be left out when they should be relevant. In order to get a more meaningful decision, I polled several experts on the decisions. I didn't bother with the decision of what suit would be signaled for, as this was virtually always quite clear and I felt I could trust my judgment on that. So, the questions I asked them for each of the 45 hands was:

1) What would you play under normal circumstances.

2) What would you play if partner signaled for a specified suit.

Now the question is how to use the poll to determine which are the relevant deals. I chose the following criteria: If the vote for the signaled for suit increased by 40% or more when receiving the signal vs. when no signal is received, then the hand is a relevant hand. Otherwise I didn't use the hand. A high vote for the signaled for suit with no information meant that this was the normal play anyway. A low vote for the signaled for suit upon receiving the signal meant that the person on lead knew to override the signal. Only the hands where there was a low vote for the signaled for suit with no information but a high vote for the signaled for suit after receiving the signal were considered relevant hands. My 40% figure was an arbitrary guess. I had to draw the line somewhere. If I had chosen a slightly different number it wouldn't have much effect on the final result.

Using the above surveys and criterion, I found that of the 45 deals there were 26 which were relevant -- the signal has a good chance to swing the choice of which suit to shift to. Note, and this is very important, this choice of hands has nothing to do with the play which was actually found at the table. The survey is simply for the purpose of determining which hands are relevant in a totally objective manner.

Now I looked at the plays actually made. On 15 of these hands, the signaled for suit was chosen. On 11 of these hands, some other choice was made.

In order for a hand to be relevant, the signaled for suit has to be a relatively unpopular choice to begin with. From the survey, the average percentage for the signaled for suit with no information came to 23.3%.

To get a feel for what this means, suppose you flip a coin twice. What is the chance that it will come up heads on both flips? Clearly that will happen 25% of the time if the coin is honest.

Consider the following experiment: We are going to make 2 flips of the coin, and we will do this for 26 trials. If the coin comes up heads on both flips, we consider that trial a win. If it comes up tails on one or both of the flips, we consider that trial a loss.

If the coin is an unbiased coin, what is your gut feel as to how likely it is that there will be 15 or more wins? It is difficult to estimate, but you can sense that it has to be a very unlikely occurrence. In your mind you would be seriously questioning whether the coin was really an unbiased coin. Of course it could happen just by chance, but you would probably think that it wasn't just chance.

This coin-flipping experiment is roughly the same as the Fisher-Schwartz results on these hands. They got 15 out of 26 wins, when the probability of a win for any given trial was less than 25%. It could happen by chance, of course, for an honest pair. But it is an extremely strong indicator that the pair is not honest.

I gave the problem to a mathematician friend of mine. He told me that if an individual trial has a probability of success of 23.3%, then the chances of getting 15 or more successes in 26 trials is approximately 1 in 6000. That means that the chance that an honest pair would have had the same or better success rate that Fisher-Schwartz had in hitting the signaled for suit on the relevant hands is about 1 in 6000. This does not prove that Fisher-Schwartz were signaling. It only says that if they weren't signaling they would have had about 1 chance in 6000 of getting these results. What the study does show is that there is a very strong indication that Fisher-Schwartz were signaling after trick 1 in the Spingold, and that if one wanted to prove this by examining videos and figuring out what they were doing this would be a good thing to look for.

The breaking of the code and the evidence from the video is far more important than this analysis. What is here is just corroborating evidence. I started the analysis before their code was broken, thinking it might be important to understand what they are doing. While not overly important for this particular case, I believe the methodology used for analyzing the hands in suspected signaling cases is the proper methodology and can be of value in the future.

Here are the actual hands which were found relevant.

Round of 32, 3rd quarter

http://www.bridgebase.com/tools/handviewer.html?bbo=y&linurl=http://www.bridgebase.com/tools/vugraph_linfetch.php?id=40353

Round of 16, 2nd quarter

http://www.bridgebase.com/tools/handviewer.html?bbo=y&linurl=http://www.bridgebase.com/tools/vugraph_linfetch.php?id=40363

Quarter-finals, 2nd quarter

http://www.bridgebase.com/tools/handviewer.html?bbo=y&linurl=http://www.bridgebase.com/tools/vugraph_linfetch.php?id=40388

Quarter-finals, 3rd quarter

http://www.bridgebase.com/tools/handviewer.html?bbo=y&linurl=http://www.bridgebase.com/tools/vugraph_linfetch.php?id=40395

Quarter-finals, 4th quarter

http://www.bridgebase.com/tools/handviewer.html?bbo=y&linurl=http://www.bridgebase.com/tools/vugraph_linfetch.php?id=40400

Semi-finals, 2nd quarter

http://www.bridgebase.com/tools/handviewer.html?bbo=y&linurl=http://www.bridgebase.com/tools/vugraph_linfetch.php?id=40419

Semi-finals, 3rd quarter

http://www.bridgebase.com/tools/handviewer.html?bbo=y&linurl=http://www.bridgebase.com/tools/vugraph_linfetch.php?id=40423

Finals, 3rd quarter

http://www.bridgebase.com/tools/handviewer.html?bbo=y&linurl=http://www.bridgebase.com/tools/vugraph_linfetch.php?id=40439

Finals, 4th quarter

http://www.bridgebase.com/tools/handviewer.html?bbo=y&linurl=http://www.bridgebase.com/tools/vugraph_linfetch.php?id=40440