2021 Rules

※Base Rules come from AIIDE StarCraft AI Competition Rules

Tournament Structure

The tournament will be round-robin, with as many rounds as possible before the time the winners need to be announced. The run will be stopped at the end of a full round when we estimate there will not be enough time to finish another round or not enough time to prepare before the time the winners need to be announced.

Tournament Format

Random Seed

We will get “Seed Number” from each participant at Submission Questionnaire. XOR’ed Seed Numbers will be used as “Random Seed” to select maps.

For all entrants that successfully register (whose registration details are published shortly after the registration deadline), their seeds will be published but not until sometime after the submission deadline. The reason is so that entrants can not figure out which maps will be used before submitting their bots. Even if an entrant that successfully registered withdraws or does not submit or their last year’s entry is reused or their submission is not used for whatever reason, their seed number will still be used to select maps. The reason is so that the maps can be selected before the organizers start checking the submissions (for easier testing), and so that the maps will not be re-selected if any submissions don’t end up eventually being used for whatever reason. Note that we will not use seeds from entries that do not successfully register whose bots are reused from last year, i.e. same behavior as treating those as though they are zero.

Map

5 maps will be selected randomly from the Map-pack and used in the competition.
We will randomly select one 2-players map, two 3-players maps and two 4-players maps.
Map pool does not different from previous years.
We will use Python Standard Random Generator to choose maps.

Game Type

The only game type for the competition will be 1 vs. 1 full game StarCraft: BroodWar 1.16.1 with fog of war enabled.

Time Limit

Games will have a ‘frame limit’ of 85714 frames, to simulate one hour of gameplay. If a game goes this long, it will be stopped and the in-game score will be used to determine the winner.

Tournament Rules

Persistent File I/O

File I/O will work the same way as AIIDE StarCraft AI Competition Rules. You will have read access to folder ‘bwapi-data/read/’ and write access to folder ‘bwapi-data/write/’, both of which will be in the standard location under the StarCraft root directory.
IMPORTANT: File I/O works as follows:

  1. Before each game, the contents of the server-side read directory for your bot are copied to the client machine under ‘bwapi-data/read/’. For the first round of the tournament this directory will be blank since a full round has not yet been played.
  2. During a game, you have write access to the ‘bwapi-data/write/’ folder and read access to the ‘bwapi-data/read/’ folder on the client you are currently playing on
  3. After the game ends, the contents of the ‘bwapi-data/read/’ folder are deleted, and the contents of the ‘bwapi-data/write/’ folder are sent to the server and stored in your bot’s server-side write folder.
  4. WARNING: Step 3 may overwrite previous results if you are not careful in specifying unique filenames for each opponent that you play against.
  5. After each bot has played each other bot on the current map (one round robin on that map) the contents of each bot’s server-side write folder are copied into the respective server-side read folder and the write folder is cleared.
  6. WARNING: Step 5 will over-write previous round data inside the server-side read folder. Your file output must either be cumulative, or have a filename scheme such that no overwrites will happen in order for your data to be preserved.
  7. The bot’s server-side read folder is NEVER deleted throughout the tournament
  8. The next round starts, and from step 1 you will have access to your server-side read data from the previous rounds inside the client’s ‘bwapi-data/read/’ folder

Again, due to the nature of overwriting the read folder with the contents of the write folder, your file names should be unique at least to the current opponent. Two popular choices for file naming schemes are:

  1. Results_EnemyName.txt – Inside this file you store all data about your matches against bot EnemyName. This file will be overwritten every round on the server, so make sure to read its contents each game, and write out the cumulative results into the write folder so you do not lose data after a round
  2. For example, you can store one line per map played inside this file so you have something like “EnemyName MapName Wins Losses ExtraData”, then if you win a game on that map against that enemy, you read in the contents and then output “EnemyName MapName Wins+1 Losses ExtraData+NewData”
  3. Results_EnemyName_MapName_TimeStamp.txt – If you don’t want to have a single file per enemy storing cumulative results, you can instead have a single file per game to store results with a unique time stamp or random number suffix. Please not that this is much slower, as your bot will play thousands of games and then have to process each file in the directory each time. Any time-out due to file processing is still a timeout and you will get a game loss. I strongly recommend the first option.

BWAPI Version

The following BWAPI release versions will be supported this year:
Version 3.7.4, 4.1.2, 4.2.0, 4.4.0,

We are not allowed to use the rest of BWAPI version.

Version 3.7.5 of BWAPI is not supported (even though it should be fully compatible with 3.7.4, but only because StarcraftAITournamentManager does not currently support it).

Tournament Rules

Persistent File I/O

File I/O will work the same way as AIIDE StarCraft AI Competition Rules. You will have read access to folder ‘bwapi-data/read/’ and write access to folder ‘bwapi-data/write/’, both of which will be in the standard location under the StarCraft root directory.
IMPORTANT: File I/O works as follows:

  1. Before each game, the contents of the server-side read directory for your bot are copied to the client machine under ‘bwapi-data/read/’. For the first round of the tournament this directory will be blank since a full round has not yet been played.
  2. During a game, you have write access to the ‘bwapi-data/write/’ folder and read access to the ‘bwapi-data/read/’ folder on the client you are currently playing on
  3. After the game ends, the contents of the ‘bwapi-data/read/’ folder are deleted, and the contents of the ‘bwapi-data/write/’ folder are sent to the server and stored in your bot’s server-side write folder.
  4. WARNING: Step 3 may overwrite previous results if you are not careful in specifying unique filenames for each opponent that you play against.
  5. After each bot has played each other bot on the current map (one round robin on that map) the contents of each bot’s server-side write folder are copied into the respective server-side read folder and the write folder is cleared.
  6. WARNING: Step 5 will over-write previous round data inside the server-side read folder. Your file output must either be cumulative, or have a filename scheme such that no overwrites will happen in order for your data to be preserved.
  7. The bot’s server-side read folder is NEVER deleted throughout the tournament
  8. The next round starts, and from step 1 you will have access to your server-side read data from the previous rounds inside the client’s ‘bwapi-data/read/’ folder

Again, due to the nature of overwriting the read folder with the contents of the write folder, your file names should be unique at least to the current opponent. Two popular choices for file naming schemes are:

  1. Results_EnemyName.txt – Inside this file you store all data about your matches against bot EnemyName. This file will be overwritten every round on the server, so make sure to read its contents each game, and write out the cumulative results into the write folder so you do not lose data after a round
  2. For example, you can store one line per map played inside this file so you have something like “EnemyName MapName Wins Losses ExtraData”, then if you win a game on that map against that enemy, you read in the contents and then output “EnemyName MapName Wins+1 Losses ExtraData+NewData”
  3. Results_EnemyName_MapName_TimeStamp.txt – If you don’t want to have a single file per enemy storing cumulative results, you can instead have a single file per game to store results with a unique time stamp or random number suffix. Please not that this is much slower, as your bot will play thousands of games and then have to process each file in the directory each time. Any time-out due to file processing is still a timeout and you will get a game loss. I strongly recommend the first option.

BWAPI Version

The following BWAPI release versions will be supported this year:
Version 3.7.4, 4.1.2, 4.2.0, 4.4.0,

We are not allowed to use the rest of BWAPI version.

Version 3.7.5 of BWAPI is not supported (even though it should be fully compatible with 3.7.4, but only because StarcraftAITournamentManager does not currently support it).

Software Requirements

Open-Source

All entries submission folders, including source code, will be published on the competition website once the competition has finished.

Game Speed

All games will be played in a LAN UDP network game between the two players at setLocalSpeed(0) (i.e. no delay between each frame, so simulation will be as fast as possible) and the Game Speed slider in the game lobby will be set to the default which is “Normal” (i.e. latency frames = 3, also known as LF3), not “Fastest” (i.e. latency frames = 6, also known as LF6, which was only used in CIG 2018).

Software Time-Out

Make sure that each onframe call does not run longer than 42ms. Entries that slow down games by repeatedly exceeding this time limit will lose games on time. In particular:

In a match a bot LOSES (immediately) on time

>= 1 frames exceed 10 seconds, or
>= 10 frames exceed 1 second, or
>= 320 frames exceed 55ms

Code Obfuscation

You are not allowed to use encryption (e.g. store log files encrypted in your Persistent I/O data folders) unless your entry also automatically decrypts all the same information somewhere else too at the same time, i.e. decryption key(s) would need to be be provided and actually used.

We reserve the right to reject entries if we feel the authors attempt to obfuscate their code (like using silly or deceiving variable names or encoding their algorithm in a 10MB FSM transition matrix stored on disk). Authors who are concerned about the source code leaking information that has not been published yet, are encouraged to submit a technical report to their institutions that describes their entries prior to submitting the code.
To simplify this process we ask authors to only use standard development tools, that can be invoked via shell commands (such as Visual C++). In case remote access is not feasible, we will need detailed instructions for building each DLL.

Conflict of Interest

Organizing AND participating in events like this can be tricky, in particular if some choices, like which maps we use, are made “randomly”, and we get access to competitors source code prior to running the tournament.

We address these problems as follows:

  1. For choosing maps we will use the standard Python pseudo random number generator that will be seeded by the XOR combination of “random numbers” all program authors provide when they submit their entry.
  2. Before accepting zip files, we will distribute the SHA-1 check sums of our zip files to all participants. This way, participants can be confident that we don’t change our entry after seeing their source code.

Detailed Rules.

  • No entry fee
  • All replay files and source code will be made publicly available after the contest
  • StarCraft Brood War version 1.16.1 will be used for all games
  • All the games are 1v1. Game type is set to Melee.
  • Games will be run using StarcraftAITournamentManager (TM), so you may want to make sure that your bot is compatible with it before submitting.
  • StarCraft is a trademark of Blizzard Entertainment
  • Blizzard will not held liable for any damage caused
  • We will not be held liable for damage caused. This includes getting your CD key banned on Battle.net due to running a 3rd party StarCraft launcher
  • Entries must use the version of the Brood War API provided on this site
  • Bots that perform malicious behavior will be disqualified and banned from all future contests. This includes but is not limited to:
    1. Open network sockets (other than those required by BWAPI or TorchCraft).
    2. Connect to the Internet. Bots will be run on an isolated LAN.
    3. Install worms/viruses/malware on the host machine.
    4. Intentionally crash StarCraft.
    5. Spam the in-game console.
    6. Change the already pre-set current working directory (CWD) of the process.
    7. Read and write files outside of the CWD (read-only), ‘bwapi-data/read/’ (read-only), ‘bwapi-data/write/ and ‘bwapi-data/AI’ (read-only).
    8. Up to 250MB of disk space total for your ‘bwapi-data/read/’ + ‘bwapi-data/write/’ + ‘bwapi-data/AI/’ directories. You are not allowed to use more than this amount. If you go over this, it might disrupt the server/competition because the disk might get full. The organizers or the tournament software might need to clear everything in your bot’s ‘bwapi-data/read/’ and ‘bwapi-data/write/’ folders.
    9. Malicious utilization of resources such as sockets, files, zombie processes (Using 100% RAM and 100% CPU is permitted).
    10. Retrieve / store system or user information, e.g. username, hostname, IP address, network information, operating system type/version, CPU type/model/details, system time zone/offsets and other location/locale information (note: dates/times are ok so long as you don’t identify the time zone/offset).
    11. Bots that behave suspiciously, don’t run, crash/hang/timeout too much, or lag too much (e.g. lagging client based bots that the tournament module can’t detect) may be disqualified after some investigation.
    12. Use the system for other purposes, e.g. cryptocurrency mining, SETI@home/BOINC projects etc.
  • Programs that attempt to cheat will be disqualified. See the other points about StarCraft bugs/tricks and bugs/exploits. Also, this includes but is not limited to:
    – Bots must *NOT* enable the complete map information flag (BWAPI::Flag::CompleteMapInformation).
    – Use the getRandomSeed() method for decision making, e.g. inferring enemy start location or race. You are allowed to write it to log files etc, so long as you don’t use it for decision making.
    – Read the contents of files updated by the tournament software, e.g. `CWD/gameState.txt`.
  • Games in which an agent crashes StarCraft will be counted as a loss
  • Games will be run on Windows 10 64 Bit virtual machine environment.
    1. Entries must be tested on Windows 10 before submission
    2. We will not be held responsible for entries that crash StarCraft due to our configuration of the system
    3. Each virtual machine for AI system process to:
      – 2 CPU core of an AMD Ryzen Threadripper 1950X  16-Core Processor 3.39GHz.
      – 2GB of memory
  • No observer players will join any games.
  • Games will be run in a semi-automated fashion.
  • Only 1 entry is permitted per affiliation. However, the different lab in affiliation is permitted, so long as there is no collusion or significant interaction between labs for bot development (refer to the other point about collusion/interaction). If in doubt, e-mail the competition organizers at least one week before the registration deadline and explain what each lab has worked on and what interaction between labs there has been, e.g. working together to develop repositories/libraries/modules that your bots share.
  • Only one submission per team/author is allowed due to the possibility of collusion and other shenanigans. For people working together as a team entry, all past and current team members must be disclosed. Starting from the date of last year’s CoG competition submission deadline (or last year’s AIIDE submission deadline if you or your team submitted any entries), if you have been an active team member developing a team entry and the team submits an entry, but you also have your own personal bot, you are not allowed a separate entry for your personal bot. Starting from the time of last year’s CoG competition submission deadline (or last year’s AIIDE submission deadline if you  or your team submitted any entries), if you have worked on more than one entry likely to be entered, or if you left a team early without a significant contribution to the team’s bot and want to enter your personal bot, please discuss with your team and e-mail the competition organizers at least one week before the registration deadline. Basing your bot on other bots is allowed so long as there is no license infringement or collusion and so long as both of you were not interacting with each other’s source code repositories during development since last year’s CoG competition submission deadline (or last year’s AIIDE submission deadline if you or your team submitted any entries), e.g. git pull requests or merging changes between your repositories. Working together on versions of projects that your bot uses that were already publicly-released versions such as already released versions of Randomhammer, Steamhammer, CommandCenter, UAlbertaBot, TorchCraft, BWAPI/OpenBW-related libraries, FAP, JFAP, SparCraft, BWEB, BOSS, BWEM, BWTA, BWAPI4J, BWMIRROR is not a problem though.
  • Bots must not slow down the game speed.  Details:
    1. All games will be played in a LAN UDP network game between the two players at setLocalSpeed(0) (i.e. no delay between each frame, so simulation will be as fast as possible). Please make sure that your bot functions correctly on this setting.
    2. To avoid problems, base your ‘timings’ on unit counts or getFrameCount() and not machine time.
    3. Your bot *MUST NOT* call setLocalSpeed(…) or setFrameSkip(…) during the competition.  Not even setLocalSpeed(0) or setFrameSkip(0).  We suggest you comment out all calls to them.
    4. Games will be run using the Game Speed slider in the game lobby set to the default which is “Normal” (i.e. latency frames = 3, also known as LF3), not “Fastest” (i.e. latency frames = 6, also known as LF6, which was only used in CIG 2018).
    5. Games in which a bot slows down the game speed significantly will be counted as a loss.
    6. If your bot is always slow or is slow too often, your bot *WILL BE* disqualified.  It may be disabled part-way through the competition. The results for all games involving the bot will be ignored and excluded.  The organizers will decide whether to restart the competition run from scratch.
  • Participants are required to disclose all source code and to provide a short description of implementation details. We will check to ensure that the source code matches the binaries that are used in the competition.
  • Entries are only allowed to make use of free software components – including BWTA and BWSAL
  • Entries are allowed to utilize a single socket to communicate with a remote process
    1. The remote process must be a 1-click to run application (e.g. we are not going to install a LISP compiler on the host machine to run a bot)
    2. Additional details on writing a remote process are available here
  • Bots are not allowed to pause the game, games in which a bot pauses the game will be counted as a loss
  • The following StarCraft bugs/tricks are permitted:
    1. Plague on interceptor
    2. Units pressed through
    3. Drops to defuse mines
    4. Mineral walk
    5. Manner Pylon
    6. Lurker hold position
    7. Observer over turret
    8. Stacking air units
  • All other bugs/exploits are forbidden. Bots caught attempted these exploits will be disqualified. This includes but is not limit to:
    1. Flying drones and templars
    2. Terran sliding buildings
    3. Stacking non-burrowed ground units is forbidden (stacking burrowed units is allowed though) except when you are using “Mineral walk” or “Units pressed through”. E.g. Stacked Recall is forbidden, and stacking your workers using “gas walk” is forbidden, but “Mineral walk” is allowed, and “Units pressed through” is allowed.
    4. Allied mines
    5. Gas walk (in all cases including: to get through blocked entrances or ramps; to attack any units)
  • Bad mannered in game behavior is discouraged, but not forbidden
  • Bot names can only contain alphanumeric characters with no spaces, e.g. “ExampleBot”, not “Example bot”.
  • Bots must stay one race for the entire tournament. Random race is allowed, i.e. the race drop-down will be set to “Random” in the game lobby.
  • Bots might be used to play against humans in show-matches using Game Speed slider in the game lobby set to “Normal”, i.e. LF3, but bots do not need to support playing against humans.
  • The ‘bwapi-data/read/’ and ‘bwapi-data/write/’ folders for participating bots will be cleared after testing. Every bot’s ‘bwapi-data/read/’ and ‘bwapi-data/write/’ folders will be empty at the start of the competition. Entries are not allowed to submit files for their bot to read from ‘bwapi-data/read/’. If your bot needs static files such as configuration files or pre-trained data, please ensure you bot reads such files from the ‘bwapi-data/AI/’ folder, not from the ‘bwapi-data/read/’ folder. Note: the purpose of the ‘bwapi-data/read/’ folder is for reading data that your bot previously wrote during earlier games in this competition.
  • The bot submissions from the authors (including source code), the bot binaries that were compiled by the competition organizers (and any source code modifications by the organizers) plus any initial configuration/pre-trained data that was actually used during the competition, replays, read/write folders, detailed game results table and summary tables/charts etc and detailed registration form will be published but not until after the AIIDE Starcraft AI Competition (in this year) organizers are ready after their submission deadline.
  • If you copy other bots or use IP/files/source code/logic/techniques etc from other bots, you must familiarize yourself with their licenses and ensure that you are not infringing their licenses. Copying other bot(s) is allowed, so long as it does not infringe their licenses and so long as you modify their logic or if you use/wrap it without modifying it and add some of your own logic on top of it, similar to how MegaBot used Skynet/Xelnaga/NUSBot in year 2017, or wrapping/modifying Randomhammer/UAlbertaBot/CommandCenter etc. If you do something like this, you must provide the source code and compilation instructions etc of the bots that you use, so that we can compile them. We decided that to foster research it is best to have the next generation of programmers stand on the shoulders of giants, rather than re-invent the wheel. We encourage authors to take code from old years of this competition and improve it.
    If you copy a bot, please uphold the spirit of competition and ensure you make significant modification or addition before you submit it. We don’t want multiple apparently near-identical copies of the same bot competing!
  • *DO NOT* have any logic that only works on a specific version of windows or specific types of environment or only works on a hard-coded list of game maps or only works against the player names that register. Entries must run/play properly when run in other similar environments and on other similar game maps and against other player names. This is so that they can easily be used in next year’s competition – the players will be different, and the environment/maps might be different – and so that they can easily be used in future by other researchers / developers / competitions. Bots should run/play properly regardless of what context it is run in.  In particular, regardless of what:
    1. The bot’s own player name is.
    2. The tournament management software sets setLocalSpeed() to (e.g. regardless of whether games are played sped-up or at real time speed etc), e.g. should not automatically leave the game if the game lasts too long according to wall clock time – do not expect to necessarily be run sped-up.
    3. The system date/time is (should not stop playing properly based on particular date/time thresholds etc).
    4. Whether their player name and/or opponent player names contains space character(s) (so that it can work easily “out of the box” in other competitions/ladders such as SSCAIT).
    5. What games are played. For example, the bot should not stop playing properly if a series of games are played very quickly against a single opponent (as opposed to occasionally playing games games during the round robin etc).
    6. What tournament module or tournament manager software is used to run the bot.
  • Rules are not necessarily retrospective when applied to re-used entries from past years – the rules that applied during the corresponding old year apply to each re-used entry, but newer/updated rules may or may not be applied, at the discretion of the competition organizers.