What information should we exchange? Game theory and machine learning
#1
Posted 2020-December-25, 05:22
So we have two agents, each of which is privy to 52 bits of information (OK, should probably undergo some dimension reduction but that's a minor detail). The captain has to make a decision between 10 possible contracts (game or partscore, and which strain). Subject to the restriction that relayee can transmit a maximum of (say) 10 bits of information, reflecting the amount of information that can be transmitted below 3NT.
This is of course oversimplified (the decision between game and partscore will have to be made before all the information is transmitted), and obviously we also need to know if we should aim at slam, but that decision can often be made after we have bypassed 3NT. So I hope it makes sense to start with the above.
So we have this network:
(relayee hand) ---> Layer1 ---> (10 bits)
(captain hand)+(10 bits) ----> Layer2 ---> (contract choice)
(contract choice)+(relayee hand)+(captain hand) ----> (DD analysis) ----> reward
It is not a typical application of neural network kits such as tensorflow or pytorch, but it would surprise me if it wasn't possible to be build such a model.
The tools which I usually would use for this kind of hierarchical models (Jags or Template Model Builder) might also work but they can't optimize an arbitrary reward function so I would have to model it as a classification problem or something. Probably a minor problem.
Regardless of the tool chosen, a bigger problem seems to be the restriction of 10 bit, i.e. the transmission layer is 10 binary states. It would be much easier if the transmitted information was in the form of continuous scale variables, but bridge bidding is a digital protocol.
Any thoughts?
#2
Posted 2020-December-25, 06:35
First: Why do you think that the Captain only has 10 bits of information about RR's hand?
If I am playing relay, I normally have complete information about shape and a decent idea about strength before I am deciding on strain and I typically do so below the level of 3N.
Next: There are a lot of paths than might be taken to get to this decision point. In some of them the captain is going to ask about shape. In other's the captain is going to ask about stoppers. I suspect that you're going to need to train separate neural networks to cover different cases.
As a practical example, I the auction starts with, a strong club opening following by a positive response and then a relay break you're living in a completely different world than a strong club opening following by a relaying out shape.
#3
Posted 2020-December-25, 14:11
helene_t, on 2020-December-25, 05:22, said:
Very sad reflection on the world. The way things are going the whole world's knowledge will be as limited and restrictive as most algorithms
All because the people with money who dont understand difficult stuff, yet get to make important decisions, are easily conned by the latest whizbang stuff. You need to say you have a stupid algorithm to do anything these days
What we have done with our neural network algorithm is replicate what the human brain with its 100 billion(??) neurons and assocuated connections with a couple of layers and a few blobs with lines connecting them. And whats more we have managed to get rid of that annoying individual variance thing with one single boring little network
.. and that is my summary of 2020
But to be more serious about them and all the kinds of technology and analysis ruling our world. It is very frightening how much variance is being thrown away with all these commercial algorithms being sold to all and sundry as being able to determine if I'm a good credit risk or even soon whether I should be tied down and medicated for the rest of my life in a psych ward
And none of this is any disresepct at all to those who actually know what they are doing and the limitations and appropriate use. Sadly they are usually the small minority
But if anyone tried to use one to classify/cluster/whatever anybody in any important decisions over anaything I would ask them for a detailed explanation of their decision process
PS SOrry for the regular tirade about the world. I'm interested in relay systems and how they work
#4
Posted 2020-December-25, 14:51
thepossum, on 2020-December-25, 14:11, said:
But if anyone tried to use one to classify/cluster/whatever anybody in any important decisions over anaything I would ask them for a detailed explanation of their decision process
Why? You know as well as I do that the decision making processes that these black box algorithms very rarely translate well into something that a human can explain.
I don't actually need to know "how" a neural network or a decision tree is making a decision. I merely need to know how accurate the system is, the extent to which it is protecting against over fitting, and the space over which I can trust it not to go haywire.
Neural networks and the like have their place in life.
They are able to analyze enormous amounts of data in very short amounts of time.
Properly used, they are incredibly valuable.
However, asking folks for a detailed explanation of how they are making decisions is not how you validate whether the folks using / training the algorithms can be trusted to use them effectively.
#5
Posted 2020-December-25, 15:09
Your point about different asking bids is important, in principle it could be dealt with by adding one extra line to my diagram I think but I want to start with something simple. And I think optimizing the relay sequence first is relevant even if the endgoal is different.
Then there's the issues of being prepared for intervention and being able to stop in a partscore with less than 10 bits and sometimes with captaincy reversed. Maybe that could be dealt with when it comes to optimizing the coding of the information. Because having addressed the first objective we haven't said anything about coding so we still have a lot of flexibility.
#6
Posted 2020-December-25, 15:20
hrothgar, on 2020-December-25, 14:51, said:
I don't actually need to know "how" a neural network or a decision tree is making a decision. I merely need to know how accurate the system is, the extent to which it is protecting against over fitting, and the space over which I can trust it not to go haywire.
Neural networks and the like have their place in life.
They are able to analyze enormous amounts of data in very short amounts of time.
Properly used, they are incredibly valuable.
However, asking folks for a detailed explanation of how they are making decisions is not how you validate whether the folks using / training the algorithms can be trusted to use them effectively.
I agree and understand the nature of most decisions/parts of a task those algorithms tend to be used for. I know they have their specific uses.
As usual my concerns about the over-selling of analytics and algorithms over every area of our lives moved away from the detailed to discussion about broader issues. Sorry
I've deleted all my irrelevant stuff here
Still editing my genuine response. Sorry its boxing day here and recovering from the effects of too much Christmas spirit
#7
Posted 2020-December-25, 15:57
helene_t, on 2020-December-25, 15:09, said:
Why don't we start with the simplest possible scenario and then add on some additional complexity.
Let's assume that we're done a standard relay auction and revealed shape with some bid (3!D if reponder has a 5-4-3-1, 3!H if responder has a 6-4-2-0, ...). Furthermore, all that we know is that responder has a minimum of "foo" slam points
At least 6 slam points if responder has nine or fewer cards in their two longest suits
At least 5 slam points if responder has 10 or more cards in their two longest shits
To start with, I think that we're going to want to encode all hand shapes that are more frequent than a 7-3-3-0
So, we're going to need to encode 12 different 4-4-3-2 patterns, 12 different 5-3-3-2's, 24 different 5-4-3-1's.
All told, that's going to be 288 different hand types or, a little less than nine bits worth of space.
The choices for the relay captain are placing the contract in either 3N, 4M, or 5m
There is no notion of slam
(Note: I realize that most people prefer natural bidding. However, my belief is that it's best to start with the simplest possible scenario. If this works, great. We can add complexity. If it doesn't work, well at least debugging will be easier. To me, auction termination playing relay is about as simple as it gets)
#8
Posted 2020-December-25, 16:33
hrothgar, on 2020-December-25, 15:57, said:
Yeah I get your point but it kinda feels unsatisfactory to start with such specific assumptions (that slam points and shape, rather than (say) suit quality) is going to be transmitted.
Another example of a simplified scenario could be "what information do we need to transmit after partner's t/o double of a 4♠ opening". But maybe that doesn't lend itself well to a one-captain model.
#9
Posted 2020-December-25, 16:36
helene_t, on 2020-December-25, 16:33, said:
Once again
I would start with an extremely simple example.
See whether you can get this to work well
If so, then start loosening your assumptions
#10
Posted 2020-December-25, 16:41
hrothgar, on 2020-December-25, 16:36, said:
I would start with an extremely simple example.
See whether you can get this to work well
If so, then start loosening your assumptions
Yeah sorry I edited my response because at first I thought your proposal was about focusing on the coding after 3NT. I realize that it's still about information exchange.
Actually I think it's a good idea, with 52 features I will have some computational problem that distract from the more basic things.
I also thought about the decision whether to bypass 1NT.
But probably your idea is better as a starting point as it is more well-defined.
#11
Posted 2020-December-25, 17:04
Or also thought of some kind of zigzag algorithm that would alternate between optimizing layer 1 for a given solution to layer 2 and vice versa, but that's probably a bad idea.
#12
Posted 2020-December-25, 17:19
Most of the stuff that I deal with can bet solved well with boosted / bagged decision trees so I stick with those. I played around a little with NN's a couple years back, just to convince myself that they works and they they were a good approximation for simple regression models.
#13
Posted 2020-December-25, 18:02
hrothgar, on 2020-December-25, 17:19, said:
ha-ha, yes often simple regression models do just as well. It has happened so often that someone asked me for help with some machine learning tool, and my first question is usually "have you tried with a glm?". And they never have, and quite often the glm gives better predictions.
#14
Posted 2020-December-25, 20:19
Essentially most are all trying to model the same thing using similar approaches so you wouldnt expect much difference. Also they have all been selected or surved according to how they perform etc. Its obvious
Its fun for a while dropping different classifiers etc in and out of models, playing with parameters ad infinitum until you realise the cost benefits of that dont add up
People spend so much time stressing over getting an extra percent or fraction percent of explained variance. Maybe everyone is heading for overfits
Disclaimer. Not an expert professional opinion BTW
Edit and much of what I say may be a misrepresentation of stuff I either learned or worked out or heard from various people over the years etc
What amazes me is that so much is based on training and testing from the same limited sets of data, and validation against other models using much of the same data
It feels as if the whole world is currently on a similar trajectory to a converging algorithm development. Or maybe it's just my life, trapped inside one. Maybe everything is regressing to the mean, or converging to some terrible state. Who knows
#15
Posted 2020-December-26, 17:16
Ideally, I would like to use the total points (or IMPs against PAR) for the chosen contract, but that is not a differentiable function.
So I thought of using the SSE of the number of tricks taken in each strain, but that's a bit silly because we don't care about the number of tricks in irrelevant strains, i.e. 3-2 fits.
So maybe we could give higher weight to the critical strains, i.e. when the number of tricks are close to game. It is still irrelevant, though, whether we have 9 or 10 tricks in hearts when have 13 tricks in spades, though. So maybe I could weight the optimal strain higher, but it sounds like a sort of circular logic .... maybe I am not thinking clearly about this, but it somehow feels wrong to assign weights based on the outcome variable.
There's also the option of treating it as a classification problem. Not optimal either as there will be many hands where the two best contracts are almost equally good.