Running competitions

Pairings

When a competition is run, the ringmaster will launch one or more games between pairs of players.

For playoff tournaments, the pairings are determined by the Matchup descriptions in the control file. If more than one matchup is specified, the ringmaster prefers to start games from the matchup which has played fewest games.

For all-play-all tournaments, the ringmaster will again prefer the pair of competitors which has played the fewest games.

For tuning events, the pairings are specified by a tuning algorithm.

Game identification

Each game played in a competition is identified using a short string (the game_id). This is used in the SGF game record filename and game name (GN), the log files, the live display, and so on.

For playoff tournaments, game ids are made up from the matchup id and the number of the game within the matchup; for example, the first game played might be 0_0 or 0_000 (depending on the value of number_of_games).

Similarly for all-play-all tournaments, game ids are like AvB_0, using the competitor letters shown in the results grid, with the length depending on the rounds setting.

Simultaneous games

The ringmaster can run more than one game at a time, if the --parallel command line option is specified.

This can be useful to keep processor cores busy, or if the actual playing programs are running on different machines to the ringmaster.

Normally it makes no difference whether the ringmaster starts games in sequence or in parallel, but it does have an effect on the Monte Carlo tuner, as in parallel mode it will have less information each time it chooses a candidate player.

Tip

Even if an engine is capable of using multiple threads, it may be better to use a single-threaded configuration during development to get reproducible results, or to be sure that system load does not affect play.

Tip

When deciding how many games to run in parallel, remember to take into account the amount of memory needed, as well as the number of processor cores available.

Display

While the competition runs, the ringmaster displays a summary of the tournament results (or of the tuning algorithm status), a list of games in progress, and a list of recent game results. For example, in a playoff tournament with a single matchup:

2 games in progress: 0_2 0_4
(Ctrl-X to halt gracefully)

gnugo-l1 v gnugo-l2 (3/5 games)
board size: 9   komi: 7.5
           wins              black         white      avg cpu
gnugo-l1      2 66.67%       1 100.00%     1 50.00%      1.13
gnugo-l2      1 33.33%       1  50.00%     0  0.00%      1.32
                             2  66.67%     1 33.33%

= Results =
game 0_1: gnugo-l2 beat gnugo-l1 B+8.5
game 0_0: gnugo-l1 beat gnugo-l2 B+33.5
game 0_3: gnugo-l1 beat gnugo-l2 W+2.5

Use quiet mode to turn this display off.

Stopping competitions

Unless interrupted, a run will continue until either the competition completes or the per-run limit specified by the --max-games command line option is reached.

Type Ctrl-X to stop a run. The ringmaster will wait for all games in progress to complete, and then exit (the stop request won’t be acknowledged on screen until the next game result comes in).

It’s also reasonable to stop a run with Ctrl-C; games in progress will be terminated immediately (assuming the engine processes are well-behaved). The partial games will be forgotten; the ringmaster will replay them as necessary if the competition is resumed later.

You can also stop a competition by running the command line stop action from a shell; like Ctrl-X, this will be acknowledged when the next game result comes in, and the ringmaster will wait for games in progress to complete.

Running players

The ringmaster requires the player engines to be standalone executables which speak GTP version 2 on their standard input and output streams.

It launches the executables itself, with command line arguments and other environment as detailed by the player settings in the control file. See also Players’ environment variables and Players’ standard error below.

It launches a new engine subprocess for each game and closes it when the game is terminated.

Tip

To run a player on a different computer to the ringmaster, specify a suitable ssh command line in the Player definition.

See Engine errors and Engine exit behaviour for details of what happens if engines misbehave.

Playing games

The board_size, komi, handicap, and handicap_style game settings control the details of the game. The ringmaster doesn’t know or care what rule variant the players are using; it’s up to you to make sure they agree with each other.

Any startup_gtp_commands configured for a player will be sent before the boardsize and clear_board commands. Non-failure responses from these commands are ignored.

Each game normally continues until both players pass in succession, or one player resigns.

The ringmaster rejects moves to occupied points, and moves forbidden by simple ko, as illegal. It doesn’t reject self-capture moves, and it doesn’t enforce any kind of superko rule. If the ringmaster rejects a move, the player that tried to make it loses the game by forfeit.

If one of the players rejects a move as illegal (ie, with the GTP failure response illegal move), the ringmaster assumes its opponent really has played an illegal move and so should forfeit the game (this is convenient if you’re testing an experimental engine against an established one).

If one of the players returns any other GTP failure response (either to genmove or to play), or an uninterpretable response to genmove, it forfeits the game.

If the game lasts longer than the configured move_limit, it is stopped at that point, and recorded as having an unknown result (with SGF result Void).

See also Claiming wins.

Note

The ringmaster does not provide a game clock, and it does not use any of the GTP time handling commands. Players should normally be configured to use a fixed amount of computing power, independent of wall-clock time.

Scoring

The ringmaster has two scoring methods: players (which is the default), and internal. The scorer game setting determines which is used.

When the players method is used, the players are asked to score the game using the GTP final_score command. See also the is_reliable_scorer player setting.

When the internal method is used, the ringmaster scores the game itself, area-fashion. It assumes that all stones remaining on the board at the end of the game are alive. It applies komi.

In handicap games, the internal scorer can also apply handicap stone compensation, controlled by the internal_scorer_handicap_compensation game setting: "full" (the default) means that White is given an additional point for each handicap stone, "short" means White is given an additional point for each handicap stone except the first, and "no" means that no handicap stone compensation is given.

Claiming wins

The ringmaster supports a protocol to allow players to declare that they have won the game. This can save time if you’re testing against opponents which don’t resign.

To support this, the player has to implement gomill-genmove_ex and recognise the claim keyword.

You must also set allow_claim True in the Player definition for this mechanism to be used.

The SGF result of a claimed game will simply be B+ or W+.

Startup checks

Whenever the ringmaster starts a run, before starting any games, it launches an instance of each engine that will be required for the run and checks that it operates reasonably.

If any engine fails the checks, the run is cancelled. The standard error stream from the engines is suppressed for these automatic startup checks.

The check command line action runs the same checks, but it leaves the engines’ standard error going to the console (any discard_stderr player settings are ignored).

For playoff tournaments, only players listed in matchups are checked (and matchups with number_of_games set to 0 are ignored). If a player appears in more than one matchup, the board size and komi from its first matchup are used.

For all-play-all tournaments, all players listed as competitors are checked.

For tuning events, the opponent and one sample candidate are checked.

The checks are as follows:

• the engine subprocess starts, and replies to GTP commands
• the engine reports GTP protocol version 2 (if it supports protocol_version at all)
• the engine accepts any startup_gtp_commands
• the engine accepts the required board size and komi
• the engine accepts the clear_board GTP command

If your engine needs to know whether it is being run for the purpose of startup checks, it can check the GOMILL_GAME_ID environment variable.

Quiet mode

The --quiet command line option makes the ringmaster run in quiet mode. In this mode, it prints nothing to standard output, and only errors and warnings to standard error.

This mode is suitable for running in the background.

Ctrl-X still works in quiet mode to stop a run gracefully, if the ringmaster process is in the foreground.

Output files

The ringmaster writes a number of files, which it places in the directory which contains the control file (the competition directory). The filename stem (the part before the filename extension) of each file is the same as in the control file (code in the table below).

The full set of files that may be present in the competition directory is:

code.ctl	the control file
code.status	the competition state file
code.log	the event log
code.hist	the history file
code.report	the report file
code.cmd	the remote control file
code.games/	SGF game records
code.void/	SGF game records for void games
code.gtplogs/	GTP logs (from --log-gtp)

The recommended filename extension for the control file is .ctl, but other extensions are allowed (except those listed in the table above).

Competition state

The competition state file (code.state) contains a machine-readable description of the competition’s results; this allows resuming the competition, and also programmatically querying the results. It is rewritten after each game result is received, so that little information will be lost if the ringmaster stops ungracefully for any reason.

The reset command line action deletes all competition output files, including game records and the state file.

State files written by one Gomill release may not be accepted by other releases. See Changes for details.

Caution

If the ringmaster loads a state file written by a hostile party, it can be tricked into executing arbitrary code. On a shared system, do not make the competition directory or the state file world-writeable.

Logging

The ringmaster writes two log files: the event log (code.log) and the history file (code.hist).

The event log has entries for competition runs starting and finishing and for games starting and finishing, including details of errors from games which fail. It may also include output from the players’ standard error streams, depending on the stderr_to_log setting.

The history file has entries for game results, and in tuning events it may have periodic descriptions of the tuner status.

Also, if the --log-gtp command line option is passed, the ringmaster logs all GTP commands and responses. It writes a separate log file for each game, in the code.gtplogs directory.

Players’ environment variables

The players are given a copy of the ringmaster’s environment variables, supplemented (or overridden) by any variables specified by the environ player setting.

The following environment variables are also set:

GOMILL_GAME_ID

The game_id of the game which will be played.

When an engine is launched for the startup checks, this variable’s value is startup-check.

GOMILL_SLOT

If the ringmaster is configured to play up to N simultaneous games, this variable is set to one of N distinct strings, such that both of the engines playing one game are given the same string, but otherwise no two engines which are running simultaneously are given the same string.

(Less formally: the ringmaster uses N worker processes to manage the games, and the slot values are simply integers from 0 to N-1 identifying the workers.)

If the ringmaster is not configured to play simultaneous games, this variable is left unset.

When an engine is launched for the startup checks, this variable is left unset.

Players’ standard error

By default, the players’ standard error streams are sent to the ringmaster’s event log. All players write to the same log, so there’s no direct indication of which messages came from which player (the log entries for games starting and completing may help).

If the competition setting stderr_to_log is False, the players’ standard error streams are left unchanged from the ringmaster’s. This is only useful in quiet mode, or if you redirect the ringmaster’s standard error.

You can send standard error for a particular player to /dev/null using the player setting discard_stderr. This can be used for players which like to send copious diagnostics to stderr, but if possible it is better to configure the player not to do that, so that any real error messages aren’t hidden (eg with a command line option like fuego --quiet).