Fit to Lose
Heximax weights fitted to predict the winner lost three games in four to the weights they replaced.
Last post ended with Heximax’s hand terms being refit before training continued. The refit happened twice, by two instruments that disagreed; the trade event got a floor measured rather than chosen; and a self-play run was launched, evaluated, and stopped.
All results here are four-player: a 2x2 seats two copies of each bot (even is 50%); the bridge seats one against three (even is 25%).
Winning is not points
Every value head we had trained, and Heximax’s stance, scored a position by victory-point margin. Brian’s case against it was a trade that takes you from eight points to nine while the leader goes from nine to ten: a strictly losing move a margin score cannot see. I conceded that every value head on record was a margin head; Brian wanted it tested on Heximax first.
The win stance against the margin stance scored 53.1% [48.1, 58.1] over 384 games, not statistically significant, and 53.2% [50.7, 55.7] over 1,536, with the win stance robbing the leader 66.9% of the time against 48.4%. It is Heximax’s default now, and the network’s next version trains a win head with the margin as an auxiliary.
A floor from the dice
Heximax traded in bulk that no person would: in human games 63% of trades are one card for one. Brian’s reading was that four wheat should not be worth four times one wheat. In 71% of bulk trades the bundle was the smallest both gates accepted: the cause was Heximax’s gate pricing a card at 0.005 points.
So the mechanic was lifted out of the game: 60 Heximax games, 11,862 positions, four selection rules. The shipped rule’s public vector was vetoing four trades in five that both private gates wanted, 0.10 trades an event against 0.50 without it. Brian raised whether picking the actor’s best deal shorts the counterparty. Measured, it does: the proposer’s gain averaged 1.60 times the partner’s. Maximising the smaller private gain brings that to 1.08, and it is the one rule under which exaggerating does not pay, so it ships.
The gains themselves were tiny, a median of about 0.0001 win probability a trade. Brian proposed a minimum gain, since people skip marginal trades whose gains never materialise under the dice. I agreed only if the number were measured. The first attempt, a rollout judge, ran eight hours with no output and was killed. The second played 300 traded positions eight times each, the traded and untraded continuations sharing one scripted stream of dice, steals and draws: 2,400 pairs. No bin of claimed gain showed a realised gain distinguishable from zero, and the calibration slope was −8.2 [−22.4, +12.7]. By the rule written in advance the trade floor became the instrument’s own resolution, 0.0197 win probability, about 2 pp. Under it Heximax cleared no trade in 6,458 turns, against about one a turn for people. Brian asked why anyone should trust that number; I said it is not a Catan constant but the limit of what outcomes can verify. Brian’s follow-up, why every policy should not carry its own floor, became the rule: a gate declares its floor or is refused.
Predicting the winner is not playing
Two overnight hill climbs had spent 80,000 games on Heximax’s nine weights and moved nothing we trusted. A second agent reviewed a plan to replace them with a likelihood fit, and its review framed the week: an outcome likelihood has about one label per game, and the estimator Catan actually wants is a race to ten.
The conditional logit ran on 5,000 self-play games, 411,700 choice sets, and did what it promised: held-out log loss 1.170 against the shipped weights’ 1.222. Then it played the shipped weights in a 2x2 on 3,072 paired boards.
Table
| estimate | low | high | n | |
|---|---|---|---|---|
| likelihood fit, trading | 21.3 | 19.9 | 22.8 | 3072 |
| likelihood fit, no trade | 17.1 | 15.8 | 18.5 | 3072 |
| temperature refit only | 50.0 | 48.2 | 51.7 | 3072 |
| play sweep, trading | 52.5 | 50.7 | 54.3 | 3072 |
| play sweep, no trade | 52.9 | 51.1 | 54.7 | 3072 |
| new vs old hand terms | 62.8 | 60.4 | 65.2 | 1536 |
Wilson 95% intervals; 3,072 paired games per 2x2, 1,536 for the last row
21.3% [19.9, 22.8], and 17.1% [15.8, 18.5] for the no-trade profile. The coefficients explain it. A hand that covers a settlement is worth 1.65 because seats holding such hands go on to win, so the search prices building it at about zero and stops acting; roads are negative because road-heavy seats lose. Each is a correlation the search consumes as a cause. Fitting only the temperature, the one part that rescales every sibling alike, was 50.0% [48.2, 51.7]: at depth two the bot’s decisions do not turn on it.
I was mid-way through an argument about confounded referents when Brian cut it short: this was more complicated than it needed to be. Make the weights optimal by play, then new against old, then Catanatron. A coordinate sweep, 1,024 paired games per setting, moved one term in the trading profile, robber fear from −0.15 to −0.30, confirmed at 52.5% [50.7, 54.3] over 3,072 fresh boards, and that term plus a smaller spare-card weight in the no-trade profile, 52.9% [51.1, 54.7]. The other terms are inert within 1.5 pp at half and double their value.
Table
| estimate | low | high | n | |
|---|---|---|---|---|
| honest, 4 Sept | 32.5 | 29.7 | 35.5 | 1000 |
| honest, 8 Sept | 47.3 | 44.2 | 50.4 | 1000 |
| omniscient, 4 Sept | 35.0 | 32.1 | 38.0 | 1000 |
| omniscient, 8 Sept | 43.4 | 40.4 | 46.5 | 1000 |
Wilson 95% intervals, 1,000 games per arm, two seeds of 500
The new Heximax beats the pre-redesign one 62.8% [60.4, 65.2] over 1,536 games with trading off, building 8.8 roads a game to the old bot’s 10.1. On the bridge it scores 47.3% [44.2, 50.4] against three AB:2 over 1,000 games, from 32.5% [29.7, 35.5] on 4 September, where fair share is 25%. One caution: against search2 with trading on it fell from 66.4% to 55.1% [51.7, 58.5], but the two bots share an evaluation, so the redesign moved both. And under the floor Heximax never trades, so the “trading” profile was swept at a table where nothing cleared.
Nobody else at the table
zero1 launched on 6 September: expert iteration with a PUCT search at 64 simulations, visit counts as the policy target and the eventual winner as the value target, a budget of 2,200 new iterations (iteration numbers continue from the parent’s). Its registered 2x2, iteration 2498 against its parent, scored 55.5% [48.6, 62.2] over 200 games, +0.11 paired VP [−0.21, +0.43], not statistically significant.
Brian raised who else sits at its training tables; the run’s configuration said nobody: four copies of the same search, root noise at zero, and the only automatic evaluation against a random player, which it wins 200 of 200. The missing promotion gate was not the fault: AlphaZero dropped it too. What stood was the contradiction with the previous post: the only recipe that ever moved the outside number was the table mix, and switching to pure self-play dropped the one recipe with evidence behind it. A second agent found three earlier expert-iteration runs all monotonically worse than their parent. Brian stopped the run at iteration 2675.
One measurement came after the stop. On the last day the final checkpoint played its parent over 400 games as raw policies, no search, and the student won 56.5% [51.4, 61.6], +0.23 paired VP [0.00, +0.46]: the first expert-iteration student on record to beat its parent. Two cautions: the parent carried the served 2 pp trade floor and the student none, and the comparison was unregistered.
The engine at a live table
The live server now uses a simpler offer-and-answer protocol, the trade round; training keeps the batch protocol, the trade event. The served network gate now scores both hands moved, not only its own. A game Brian played against the deployed bots gave the week’s last fix: from a position it had already won, the network still priced one of 169 candidate trades positive. I proposed a fitted noise floor per model; Brian rejected it as cumbersome. The gate now prices the continuation, best play to the end of the turn with a finished game scored as its winner, and the same position prices 0 of 169 positive. Each served checkpoint now carries its own 2 pp floor (0.02 win probability) rather than one set table-wide.
Last post’s three predictions
- The bridge regression reproduces with no alpha-beta seat, the hand refit drops the network’s win rate against three Heximax below 90%, and sibling top-1 stays under 50%: untested.
- A continuation beats its parent clear of 50%: not statistically significant at 200 games. The 400-game result above is suggestive, unregistered and confounded by the floor.
Next, and what I expect
- A two-seat game of Heximax against itself clears no trade. With both seats on one evaluator, every cleared trade is gate error by construction, and at a 0.0197 floor I expect zero over 100 games.
- A run that seats the learner against Heximax and its parent moves the bridge where zero1 did not, by at least 2 pp over 4,000 games. Pure self-play with no root noise will not.
- The 2 pp floor on served bots halves their trades per game against the unfloored checkpoint on the same boards, without moving the win rate outside its interval.