Table Manners
Self-play added 6 pp to a network that learned from people, against held-out Heximax at four seats, and nothing at 1v1.
Last week ended with Heximax as our best bot. At a four-seat table it isn’t. clio-r4, a network that learned from people and that these posts had not measured fairly until now, beats it, and self-play this week made it stronger against a Heximax it never trained with. Neither result survives the move to two players. Brackets are 95% intervals, and a result not marked otherwise is four-player 2x2, where even is 50%.
A measurement that was really the seating
From here on, “duel” means a two-player game; the four-seat head-to-head is a 2x2. The first two-player measurement put clio-r4 at 18.25% against AB:2 under the duel variant. Brian, who plays this model 1v1 daily, rejected it.
The arena plays two players by retiring two seats of a four-seat table. Its rotation was built for 2x2, and with the two live seats side by side, the opponent sat one seat after side A in every game. The network reads the table relative to its own seat, and from that slot it plays about 25 pp worse. The control costs nothing: clio-r4 against a byte-identical copy of itself scored 21.75% over 400 games, where a symmetric arrangement must score 50%. Heximax, which has no notion of seat, scored 47.25%. With the live seats opposite each other, the self-duel scores exactly 50% of decided games. Corrected, clio-r4 scores 69.75% [65.4, 74.1] against AB:2 at standard 1v1 and 38.62% [35.2, 42.0] under the duel variant.
So run any new seating as an entrant against its own copy first. We learned that for 2x2s a month ago, and again when the table shape changed.
Nineteen games in that batch never ended: both sides stuck at 12–14 of 15 VP, deck empty, clio-r4 trading with the bank and straight back. A network that learned from 10-point games has never needed a twelfth point. The engine now treats an unfinishable game as a defect, not a loss.
The network that learned from people
On equal terms clio-r4 takes 70.50% of a 2x2 against Heximax, 41.00% at standard 1v1 and 5.50% under the duel variant, 400 games each.
Then PPO, warm-started from clio-r4: 400 iterations with the learner holding one to four seats against frozen networks, a penalty for drifting from the parent, and Heximax held out of training entirely. The first attempt kept player trading on. Pricing trades took 97% of collection time and put the run at about 24 hours, so it was stopped at iteration 35. By then it already beat clio-r4 58.1% [54.7, 61.6] with trading on, but only 51.3% [49.2, 53.4] without, in a 2x2. The rerun, with player trading off, took about three hours.
One network against three Heximax, over 4,000 games: clio-r4 wins 39.2%, iteration 400 wins 45.3%, and fair share is 25%. The paired gain is +6.02 pp [+4.09, +7.96], so by the rule registered before the run, it improves.
Table
| estimate | low | high | n | |
|---|---|---|---|---|
| 4 seats, one vs three | 6.02 | 4.09 | 7.96 | 4000 |
| 3 seats, one vs two | 3.56 | 1.1 | 6.02 | 2500 |
| 2 seats, standard | -1.0 | -3.3 | 1.3 | 2500 |
| 2 seats, duel variant | 0.1 | -2.13 | 2.33 | 1000 |
One network against Heximax, network trading off; paired by board, 95% intervals; n = games per checkpoint
Brian questioned my plan of three 4,000-game measurements; sizing each to a ±2.5 pp interval instead paid for the smaller tables. Self-play at four-seat tables made a better four-seat player and taught it nothing for two seats. Under the duel variant iteration 400 wins 8.8% of 1,000 games against Heximax, level with its start’s 8.7% on the same games (clio-r4’s 5.50% above was a separate 400-game sample), and nothing in its input says the target is 15. Inputs scaled to the rules, long argued for by Brian, are planned for the next version, along with build costs, which it turns out the network was never given. Meanwhile a run playing mostly duel-variant games asks how far play alone gets.
Heximax: the evaluation moves, the search does not
Last week, Heximax at its served setting of 100 sampled worlds scored 40.8% against one world. Brian flagged that a hundred worlds losing to one needed explaining, and reading the search found why. The leaf budget was one cap per decision, shared across every world, and mid-game the belief holds about 41 distinct worlds, so the bot searched one ply. With a budget per world, PIMC at four worlds scored 51.2% over 2,000 games and 50.2% [49.5, 51.0] over 8,000. It buys nothing; the served seat now uses one.
Table
| estimate | low | high | n | |
|---|---|---|---|---|
| award fragility | 2.1 | 0.9 | 3.3 | 4000 |
| trade gate prices this round's wins | 1.43 | 0.7 | 2.15 | 8000 |
| PIMC, four worlds, own budget each | 0.2 | -0.5 | 1.0 | 8000 |
| slider pin, on top of fragility | 0.1 | -1.2 | 1.5 | 4000 |
| win transform at the leaf | -0.2 | -1.5 | 1.1 | 4000 |
| human-shaped trade willingness | -0.27 | -1.05 | 0.5 | 8000 |
| credit for a second build | -8.6 | -10.6 | -6.6 | 2000 |
Four seats, standard rules, paired boards; 95% board-level intervals; n = games per arm
Award fragility is the one evaluation gain, about 2 pp, and flat across a fourfold range of its coefficient. A slider pin, fixing Heximax’s blend of its no-trade and trading weights at one end, had confirmed at 1.7 pp and vanished once fragility shipped, apparently because both fixed the same mistake.
Brian also raised a hidden-information leak that we thought was closed. The audit: relabel the opponents’ hidden cards among themselves so that the seat’s own view is bit-identical, and require the decision not to change. Heximax passed every probe. Our network MCTS changed 2 of 6 decisions, because it rooted every tree on the true state. On zero1, the honest search wins 45.3% [41.2, 49.3] against the leaky one over 464 games. About 5 pp, and no conclusion changes: zero1 wins 26.5% [23.0, 30.0] against Heximax.
Not handing over the win
Heximax has started playing people online. In a four-player game tied at 9–9, an opponent offered three cards for one of Heximax’s lumber. Heximax knew that hand exactly; the trade completed a settlement, which the opponent built to win. The trade gate had approved the trade because Heximax’s evaluation scores a hand that pays for a settlement at no more than 0.45 of a point, but the built settlement at about 3.
The first fix credited a winning build only where it was visible, and came back null: in 80 of the 100 win-enabling trades in self-play, the winner held a hidden victory-point card. The second priced each seat’s chance of reaching 10 on its next turn, hidden cards included, in turn order. It scored 51.43% [50.70, 52.15] over 8,000 games, and it handed the other side a same-turn win 32 times to the old gate’s 76.
Brian pressed further: the bot would still hand a leader Longest Road. I argued for scoring every hand by everything it could build and letting the win stance weight the risk by who is ahead. It scored 51.04% [50.02, 52.06], but it traded about 40% more, and Brian called that a regression, since a fix for this should only block trades. I proposed a veto taking the smaller of two prices; Brian called that a kludge and asked for something simpler: refuse any trade that raises a partner’s chance of reaching a points line. We tested lines at 10, 9 and 8 against the shipped gate at one table of four variants, over 16,000 games. A line at 10, on top of the next-turn pricing, beats the shipped gate by +2.96 pp [+1.87, +4.05] on a 25% share, and lines at 9 or 8 add nothing.
None of these makes Heximax avoid the leader. At 8 points or more, its leader ratio is 1.02–1.10, where people’s is 0.43. What does have a human shape is timing: people’s trading falls about elevenfold between 2 and 9 points, and Heximax’s only 1.8-fold. A willingness curve fitted to people scored 49.73% [48.95, 50.50]. Brian expected it to cost strength; none was measurable, though the interval allows a loss of about 1 pp.
Two more benchmarks
Rehex is our rewrite of a rules bot people play against online; our first port never matched the original’s results, so we started over. It beats AB:2 79.00% [76.2, 81.8] at 1v1 under the duel variant, and it runs about 470 times faster than upstream AB:2 at four seats. Heximax beats it 62.62% [59.4, 65.8], same terms.
An earlier block of 100 two-player games against people online, with AB:2 in our seat, won 23, though it reached 10 VP first in 38 of the 87 decided games, then failed to close to 15: like clio-r4, it loses its endgames past 10. Heximax’s two-player block ran 77 games: of the 59 played to the end it won 20, and 17 more opponents left early. Heximax’s live offers had come from an outdated copy of the engine’s offer logic; the live adapter now only carries the engine seat’s decisions.
Last post’s predictions
- The one-forward gate shrinks the rollout’s advantage below 5 pp, and the fragmented policy loses by less one against three: untested. The training run switched player trading off.
- Serving one world lifts the live win rate 5 pp: untested as stated. No before-and-after block was played, but the arena traced the handicap to a starved budget.
Next, and what I expect
- The duel-variant run improves on its parent against Heximax but stays under 20% at 1v1, from about 9% at the start.
- Self-play’s four-seat gain survives a trading table: with trading on, the iteration-400 network beats clio-r4 by at least 3 pp, one against three Heximax over 4,000 games.
- The willingness curve leaves Heximax’s late leader ratio above 0.8. Avoiding the leader is a judgement about partners, and nothing built this week makes it.