Before we even start thinking about creation of a bot, we have to know about the CTF game rules, that are little more complicated than free-for-all DeathMatch.

CTF is a two-team game, where each team tries to score points by taking enemy flag and carrying it to its team base. Rules of the Capture The Flag game:

For more information about CTF tactics consult Garvelous CTF guide for UT 2004.

Every bot can specify the team it belongs to in a team attribute of the INIT message that is send during handshake of the bot with the Unreal. The INIT message supplies data necessary for creation of the bot's avatar in the game (such as initial location, name, skin or team) and GameBots creates the avatar upon recieving. Bot can get total number of teams in the game from attribute MaxTeams of GameInfo message.

    @AgentScoped
    public class Hunter extends UT2004BotModuleController<UT2004Bot> {
        // ...
        @Override
        public Initialize getInitializeCommand() {
            int maxTeams = this.game.getMaxTeams();
            return new Initialize().setName("NavigationBot").setTeam(1);
        }
        // ...
    }
                
                

Valid team number is from 0 to GameInfo.MaxTeams - You should also note, that team's size is not unlimited, the maximal size of each team is GameInfo. MaxTeamSize. If the team is not specified in the INIT message, GameBots will use team number 255 in DeathMatch and team number 0 in CTF games.

In some situations, you may want to select team of a bot based on number of players per team or some other metric thus balancing number of players in teams. This is useful in many situations, such as running same bot program multiple times and still creating two teams with same number of players (one team may have one player more than the other) or if you want your bot to join the team that has lower score and many other situations.

For further details, consult GameBots documentation.

In CTF game, information about flags is send as synchronous message from the GameBots, so the bot always have up-to-date info about flags. You can retrieve current info from the worldview, e.g. like this:

    public FlagInfo getFlag(int team) {
        Collection<FlagInfo> flags = bot.getWorldView().getAll(FlagInfo.class).values();
        for (FlagInfo flagInfo : flags) {
            if (flagInfo.getTeam() == team) {
                return flagInfo;
            }
        }
        throw new IllegalStateException("FlagInfo about with seeked team not found.");
    }
                    
                    

You'll get location and possibly holder if you can see the flag (and possibly its carrier), other information is always available. FlagInfo. getState() is probably most useful.

Unfortunately, you can't directly query the worldview for info about the team base, but we know that base is located at navpoint with a specific id. Following code is showing how to find the home for the specified team:

    // unreal assigns names to teams by default, team 0 is red, 1 is blue, 2 is green and 3 is gold
    private static String[] teamColors = new String[] {"Red", "Blue", "Green", "Gold"};

    public NavPoint getTeamBase(int team) {
        String flagBaseStr = "x" + teamColors[team] + "FlagBase0";

        for (NavPoint navpoint : bot.getWorldView().getAll(NavPoint.class).values()) {
            if (navpoint.getId().getStringId().contains(flagBaseStr)) {
                return navpoint;
            }
        }
        throw new IllegalStateException("Unable to find base for " + teamColors[team] + " team.");
    }
                    
                    

We are going to create our simple CTF bot using FSM(Finite State Machine), so before we start programming anything, it is necessary to specify how the bot should make decisions. You should always start with planning architecture of a bot's logic, it usually saves a lot of trouble and time. We are going to use work in progress library for FSMs that is currently being developed.

The bot has three different state types and multiple transitions. Bot starts at the hunt state, so it should wander around map, collect add-ons and kill enemies. When he sees enemy's flag he switches to pickup enemy flag and picks it up, and after he picks it up, he switches into go home. Unfortunately, team flag may be missing, because enemy bot got to it, so bot can switch into hunt mode and hoping to find enemy and kill him (NOTE: that won't return flag home, but if level is small enough, there is a good chance bot will accidently move over the dropped flag while in hunt mode). After flag has been delivered home, go back to the hunt mode. The rest of transitions is for events, when something doesn't go in the expected way, e.g. transition from hunt to go home is for case, when bot accidently moves over enemy flag, that he couldn't see before.

As you are aware, FSM consists of list of states, start state and transitions between states. Start state is the state "Hunt," bot will switch into that state when he enters the environment or when he is killed.

States used in the FSMBot library are not exactly states in classical FSM definition, but more like activities the bot is supposed to do. List of states:

There are many shortcoming, e.g. state Hunt should preferably shoot at enemy holding our flag or state "Pick up enemy flag" and "Go home" should shoot at visible enemy while racing for their destination.

Transitions are a little tricky, basically there are two types of transitions that differs by eligibility (under what circumstances are they considered for potential transition from state A to B).

In order to properly utilize FSMBot library, you need to have an idea how do states look like, how to program them and how are they processed. Every state has to implement interface IFSMState. Careful, there is an interface with the same name in afsm library, that is used during communication with GameBots.

There are basically two paths state can go:

I attemp to make sure that destination state of transition sees the environment same way the transition did, so the condition that triggered the transition itself is still valid even in a destination state (e.g. if transition is supposed to trigger when visible player appears, the destination state will see player at the same position and still visible). This behavior is not guaranteed if state has to loop over any of following methods:

In this section we will create the project and set up dependencies necessary for us to build FSM bot.

Before we start with our translation of CTF bot schema into the states and transitions, we need to create some infrastructure, that will execute a FSM machine with our states.

Necessary infrastructure consists of

With necessary infrastructure in place, lets create our very first state, that by itsefl won't do anything useful, think of it as foundation for all other states.

Start up Unreal server and build and run CTFBot project, bot should appear in the game and last line in the Output window of Netbeans should contain following text: (CTFBot1) [FINE] 18:02:58.791 <User> State "Empty" is being initialized.

Although we finally have working FSM bot, it is not a very useful one. Let's create new state, StateHunt from our specification. Just to remind you, bot in the state Hunt should do following:

That is easy.

Feel free to test single-state hunt bot. Not perfect, of course, but it runs and kills. We could implement things like changing to weapon with ammo, either with some additional lines in run() or utilizing hierarchical sub FSM with states Hunt and ChangeToBestArmedWeapon and interrupt transitions HaveBetterWeapon and sequential transition True.

The ChangeToBestArmedWeapon would probably have all necessary code in init() and isFinished() would immediately return true, so sequential transition True would immediately occur.

The demonstrated process is pretty much standard for all other states. Either program them yourself based on the requirements from CTF FSM schema or use provided sample. I would of course recommend programming them yourself, in order to familiarize yourself with FSMLib and Flag handling.

"Pickup enemy flag" is state that is trying to pickup enemy flag, that is visible by the bot on the ground(=enemy flag is in "dropped" or "home" state). State is finished, when bot loses sight of enemy flag (some obstacle hides it during movement to the flag...) or when someone picks the flag(bot, teammate).

StatePickupEnemyFlag has no initialization, and standard cleanup (stop moving). To the CTFBot add method getFlag, shown above.

    public void run(CTFBot ctx) {
        // do I still move along the path to the enemy flag?
        if (ctx.getPathExecutor().isMoving())
            return;

        // Go to enemy flag.
        int enemyFlagId = 1-ctx.getAgentInfo().getTeam();
        FlagInfo enemyFlag = ctx.getFlag(enemyFlagId);

        // enemy flag always have valid location, because it is visible (thx to isFinished running before run)
        PathHandle path = ctx.getPathPlanner().computePath(enemyFlag.getLocation());
        ctx.getPathExecutor().followPath(path);
    }

    public boolean isFinished(CTFBot ctx) {
        FlagInfo enemyFlag = ctx.getFlag(1-ctx.getAgentInfo().getTeam());

        // flag is visible and on the ground, state is not finished.
        boolean onGround = "Dropped".equalsIgnoreCase(enemyFlag.getState()) || "Home".equalsIgnoreCase(enemyFlag.getState());
        if (enemyFlag.isVisible() && onGround)
            return false;
        return true;
    }
                                

Method isFinished is ensuring that necessary conditions are still valid (enemy flag is still visible and on the ground) before run is called. Method run will repeatedly plan path to the enemy flag and go there. Why repeatedly? In theory, someone could pickup the flag and be immediately (few tens of ms) killed, therefore dropping enemy flag a little off its original position. The bot would go to the original flag position none the wiser and state would never end (because bot would see the flag in its new position).

Add the StatePickupEnemyFlag to the createFSM method of CTFBot, and, if you want to test it out, set the initial state to the instance of StatePickupEnemyFlag and spectate in the Unreal, how bot runs from its initial position to the enemy flag, if it can see it. If you are using startGamebotsCTFServer.bat to start the server (recommended), you are using map CTF-1on1-Joust, where bot starts at a position, where he can always see the enemy flag. If enemy flag is not visible or if your bot has picked the flag, state will end, because isFinished returns true. In such case, the Netbeans output window should contain message: (CTFBot1) [FINE] 14:13:47.494 <User> State "Pickup enemy flag" is finished.

You may have noticed that programing and especially testing individual states is far easier than in complex logic method, where you mingle enormous ammount if interactions.

Now we have two states, but no way to change between them. In order to do that, we will have to add a transition (see the FSM schema). Our transition will be interrupt transition from StateHunt to StatePickupEnemyFlag, that will be triggered, when bot will see enemy flag.

Transitions are added using method addTransition of interface IBotFSM. That method accepts following parameters:

Modify createFSM() to create new transition.

        // add states
        fsm.addState(hunt);
        fsm.addState(pickupEnemyFlag);

        // add transitions
        // hunt-pickupEnemyFlag
        ICondition<CTFBot> seeDroppedEnemyFlag = new ICondition<CTFBot>() {
            public boolean satisfied(CTFBot ctx) {
                FlagInfo enemyFlag = getFlag(1-ctx.getAgentInfo().getTeam());
                boolean onGround = !"held".equalsIgnoreCase(enemyFlag.getState());
                return enemyFlag.isVisible() && onGround;
            }
        };
        fsm.addTransition(seeDroppedEnemyFlag, hunt, pickupEnemyFlag, 0, TransitionType.INTERRUPT);

        // set starting state of FSM
        fsm.setInititialState(hunt);                
                

We create ICondition as anonymous class, but feel free to put them into their own class file. If you look at FSM schema, you will see this transition there, with exactly the same properties as this one.

Let's create the last state "Go home", that will find and execute path from current bot position to its base. The state is used when bots holds enemy flag and wants to get home in order to score a point. It will ignore enemy fire and all other distractions. State is finished, when bot arrives to its home base.

Use class name StateGoHome in ctfbot.state for the state. State has standard interrupt and cleanup methods (stop bot is necessary), but init and run are little different.

    public void init(CTFBot ctx) {
        NavPoint homeBase = ctx.getTeamBase(ctx.getAgentInfo().getTeam());
        PathHandle path = ctx.getPathPlanner().computePath(homeBase);

        ctx.getPathExecutor().followPath(path);
    }

    public boolean isReady(CTFBot ctx) {
        return ctx.getPathExecutor().isMoving();
    }

    public void run(CTFBot context) {
    }

    public boolean isFinished(CTFBot ctx) {
        return !ctx.getPathExecutor().isMoving();
    }
                                

In this case, we only need to get home and can therefore set up everything during initialization and no additional code is necessary in the run. The isReady makes us wait until the bot starts to move, if we hadn't waited until bot starts moving, the isFinished method would prematurely signal that state has been finished. That would result in StateGoHome finishing prematurely.

We have finally all states completed, transitions are sill missing. In this section, we will modify createFSM so it creates the FSM bot from our schema. First, make sure you have all states from schema instantiated and inserted into FSM:

    private IBotFSM createFSM() {
        BotFSM fsm = new BotFSM();
        // create states
        IFSMState<CTFBot> hunt = new StateHunt();
        IFSMState<CTFBot> pickupEnemyFlag = new StatePickupEnemyFlag();
        IFSMState<CTFBot> goHome = new StateGoHome();
        IFSMState<CTFBot> huntEnemyCarrier = new StateHunt();

        // add states
        fsm.addState(hunt);
        fsm.addState(pickupEnemyFlag);
        fsm.addState(goHome);
        fsm.addState(huntEnemyCarrier);

        ...
                

Notice that we create two different instances of state Hunt, the huntEnemyCarrier is used, when bot has flag, but his flag is not at home and he therefore can't score.

Now it is necessary to add the remaining transitions, rather boring and gruesome work, do not miss a transitions nor parameters.

        // add transitions
        // hunt-pickupEnemyFlag
        ICondition<CTFBot> seeDroppedEnemyFlag = new ICondition<CTFBot>() {

            public boolean satisfied(CTFBot ctx) {
                FlagInfo enemyFlag = getFlag(1 - ctx.getAgentInfo().getTeam());
                boolean onGround = !"held".equalsIgnoreCase(enemyFlag.getState());
                return enemyFlag.isVisible() && onGround;
            }
        };
        ICondition<CTFBot> someoneElseTookEnemyFlag = new ICondition<CTFBot>() {

            public boolean satisfied(CTFBot ctx) {
                FlagInfo enemyFlag = ctx.getFlag(1-ctx.getAgentInfo().getTeam());
                return !ctx.getAgentInfo().getId().equals(enemyFlag.getHolder());
            }
        };
        fsm.addTransition(seeDroppedEnemyFlag, hunt, pickupEnemyFlag, 0, TransitionType.INTERRUPT);
        fsm.addTransition(someoneElseTookEnemyFlag, pickupEnemyFlag, hunt, 100, TransitionType.INTERRUPT);

        // pickup enemy flag - go home
        ICondition<CTFBot> botHasEnemyFlag = new ICondition<CTFBot>() {

            public boolean satisfied(CTFBot ctx) {
                FlagInfo enemyFlag = ctx.getFlag(1-ctx.getAgentInfo().getTeam());
                return ctx.getAgentInfo().getId().equals(enemyFlag.getHolder());
            }
        };
        ICondition<CTFBot> always = new ICondition<CTFBot>() {

            public boolean satisfied(CTFBot context) {
                return true;
            }
        };
        fsm.addTransition(botHasEnemyFlag, pickupEnemyFlag, goHome, 100, TransitionType.NORMAL);
        fsm.addTransition(always, pickupEnemyFlag, hunt, 150, TransitionType.NORMAL);

        // hunt - go home
        ICondition<CTFBot> botLostEnemyFlag = new ICondition<CTFBot>() {

            public boolean satisfied(CTFBot ctx) {
                FlagInfo enemyFlag = ctx.getFlag(1-ctx.getAgentInfo().getTeam());
                return !ctx.getAgentInfo().getId().equals(enemyFlag.getHolder());
            }

        };
        fsm.addTransition(botHasEnemyFlag, hunt, goHome, 100, TransitionType.INTERRUPT);
        fsm.addTransition(botLostEnemyFlag, goHome, hunt, 100, TransitionType.INTERRUPT);

        // hunt enemy carrier - go home
        ICondition<CTFBot> teamFlagIshome = new ICondition<CTFBot>() {

            public boolean satisfied(CTFBot ctx) {
                FlagInfo teamFlag = ctx.getFlag(ctx.getAgentInfo().getTeam());
                return "home".equalsIgnoreCase(teamFlag.getState());
            }
        };
        ICondition<CTFBot> teamFlagIsNotHome = new ICondition<CTFBot>() {

            public boolean satisfied(CTFBot ctx) {
                FlagInfo teamFlag = ctx.getFlag(ctx.getAgentInfo().getTeam());
                return !"home".equalsIgnoreCase(teamFlag.getState());
            }
        };
        fsm.addTransition(teamFlagIshome, huntEnemyCarrier, goHome, 100, TransitionType.INTERRUPT);
        fsm.addTransition(teamFlagIsNotHome, goHome, huntEnemyCarrier, 100, TransitionType.INTERRUPT);

        // set starting state of FSM
        fsm.setInititialState(hunt);
                
                

In this case, we are using botHasFlag twice, but we can do that only becase it has no internal structures. Generally, you should always create new instance of ICOndition<CONTEXT>.

That's all. You now have reasonably working CTF FSM bot. Start up Unreal server on some CTF map (e.g. startGamebotsCTFServer.bat), run bot and play against him, if you want.