alicerunsonfedora / costumemaster

Applied AI with GameplayKit project

Marquis Kurt

Hey there! 👋 I'm Marquis, and I'm a software developer. Feel free to follow my ramblings on https://rambles.marquiskurt.net. If you like the work I do and want to support me, consider buying me a coffee on Ko-fi or supporting me on GitHub Sponsors.

🖋 Languages I Use

📁 Current Projects

• Fedigardens: A simplified, discussion-driven Mastodon app for iPhone and iPad
• Indexing Your Heart: A Witness-like puzzle game with story components about building relationships under masks.
• Give Me A Sniglet: A small app to generate sniglets validated by a machine learning model

🕹 Some of My Game Projects

You can also visit my page on Itch.io for a more complete list of games.

• Unscripted: A visual novel about software development
• Bug Bounty: Quick game about squashing bugs (Entry for 129th Trijam)
• Planting Uneasy Feelings: A game about inflicting curses (Entry for Godot Wild Jam 36)
• No Love: A game about relationships (Entry for Wowie! Jam 3.0)
• Package Resolved: A game about collecting packages as quickly as possible

🔗 Some Helpful Links

• Github (you're already here!)
• Itch.io page
• Discord server
• Mastodon (@[email protected])

costumemaster's Issues

View appears completely blank on Monterey 12.3

It appears that the level is failing to load or isn't being rendered correctly.

Create user preferences

Games usually come with some preferences. Currently, a storyboard for the preferences window exists, with a dummy debugging settings page. This should be expanded upon to store settings.

• Create user preferences with UserDefaults object
• Hook up code to storyboard
• Hook up Preferences menu to window
• Add "Options" button to main menu and hook it up

Determine how to score puzzles for AI

Currently, there is one major issue for the AI when it comes to playing the game: how to get the AI to actually solve the puzzle. Most of the GameplayKit functions are centered around games that have scores like Connect 4, chess, etc.:

• The minmax strategist heavily relies on an opponent, but there isn't any in this game.
• The Monte Carlo strategist heavily relies on determine what's a winning state and what's a losing state. Since the game does not contain any hazardous materials, there's no real way to determine if there's a loss at all.
• The AI has no idea about the visual cues that the player can see, such as activated textures, hints, etc.

Essentially, the game has a very similar complexity to Portal, which may be difficult for the AI to actually do anything. I should figure out what I can do to make the AI be able to solve puzzles on its own.

Walls and wall structures drift on their own

If you sit on a level for a while, the walls and other physics objects that are resting drift without any intervention.

Notes

• This behavior occurs regardless of size or distance
• This only occurs with nodes that contain physics bodies, but are resting and pinned
• This appears to be a known problem in SpriteKit: https://developer.apple.com/forums/thread/27057

Implement timed buttons and weighted plates

The game contains two basic input methods for activating things: computers and levers. Computers and levers pretty much act the same and act as a simple boolean toggle for doors. However, the game should be a bit more challenging. In the original design spec, I mentioned timers and heavy objects as potential elements. These should probably be implemented.

• Create a timer and add the textures
• Create an "inventory" system for the player where they carry a single object (will be superseded)
• Implement heavy weighted plates that act like computers/levers

Create RND strategist implementation

Now that I've determined that I will be utilizing random network distillation to test the agent, I will need to make an implementation based off of the GKStrategist protocol.

Clean up and optimize GameScene

GameScene contains a lot of code for a given puzzle level. This isn't necessarily a bad thing, but it would be nice to start abstracting some of the things in there so there can be room for other implementations of things like scores (se #13). I've started to do this with the changes from #14, moving requisite generation to LevelDataConfiguration.

• Make tile map parsing a higher order function.
  • Create a data structure to contain important information about the current tile definition.
  • Abstract away!

Determine how to score levels

Currently, there is no implementation of how a score is calculated for a level. This can be problematic since Game Center achievements and leaderboards may require a better system, and the AI agents will definitely need it. A scoring system should be in place to help facilitate this.

Notes to keep in mind:

• The current codebase doesn't allow direct movement (i.e., "move in x direction"); rather, it uses a physics based system where the sprite is given an impulse force and moves.
• There may be different ways to score a level, including times, efficient paths, costume usage, etc.
• Game Center challenges, achievements, and levels may not necessarily need to use the same scoring system as the agents do.

Start implementing switch logic

Switches are a primary component of this game and add challenges. The logic for switches should be implemented. The following will need to be done:

• Create a base switch class that all elements can inherit from.
• Create subclasses for switch elements.
• Create tile set images for switches.
• Update GameScene code and other utilities to account for switches of multiple types.

Start working on base classes and a sample scene

Now that I have a premise for the game (see #1), I should start working on some potential parts of the game to at least get a playable scene:

• Create basic logic to handle player movement and collisions
• Create an observable object with @State to better handle states
• Start work on player assets and animations
• Begin creating a tileset to use for tile maps

Create a subclassed version of ChallengeGameScene

The game's source currently includes a ChallengeGameScene that keeps track of the current time in seconds, as well as costume switches, for challenges such as leaderboard score submissions and achievements based around those statistics like Overclocker and Costumemastery.

Ideally, the AI should be able to use those statistics to assess the current game state and evaluate a score based off of it, since most agents depend on scores to make a decision. I should create a subclass of ChallengeGameScene that also accounts for the following:

• Whether the exit door is active or not
• The number of inputs that directly link to the exit door and their statuses
• A method to evaluate the score based on scene conditions

Create basic Monte Carlo strategist

Once all of the required frameworks are in place, the Monte Carlo strategist in GameplayKit should be instantiated and let it determine the best moves in a given scenario.

• Implement a AIStrategist structure
• Add a new scoring system to AIGameState

Black lines continuously appear after moving

When moving in a level, black lines appear in the level. This happens regardless of position.

My suspicion is that something is happening to the walls themselves and are trying to react to the forces that are applied to the player node (since the game is essentially a giant physics simulation).

Write new methods for player control

New methods for the AIBaseAgent class will need to be implemented so that the AI can actually do things besides movement.

• Pick up/drop items
• Switch costumes (next, prev.)
• Activate/use item (i.e., pressing "E" key)
• (USB only) Deploy/retract clones

Due to how some of the abstraction works with the AIBaseAgent, game scene (AIAgentInteractableScene), and game model (AIGameState), some classes may need extra modification to determine what can happen.

Notes

• For use, the method may need to look like the following signature:

func use(with inputs: [GameSignalSender]) {

• For picking up/dropping items, the state will probably need to keep track of both the player's current inventory and the objects in the game world.

Add GameKit integration (Game Center)

Some parts of the game may be more appealing if there were achievements and scores involved. Since I will eventually need to design a scoring method for a level for AI agents anyway, I might as well take the time to incorporate Game Center into the game.

• Set up Game Center entitlements
• Introduce scoring system for levels (superseded by #13)
• Create authentication process for game
• Add Game Center leaderboards/achievements
  • Main menu achievement
  • Chapter/level achievements

Start designing levels

Now that most of the base elements are in place for creating levels, it's time to start making levels for the game. Levels should vary in difficulty and make use of the mechanics present in the current codebase.

Add more achievements to Game Center

There are currently a couple of achievements in Game Center, but it would be nice to get some other achievements implemented into both the game and the App Store Connect page.

This page will be updated over time with new achievements as I come up with them.

• Quickfooted: (25pts) Switch to the bird costume at least 100 times.
  • App Store Connect entry
  • Implementation

• Overclocker (50pts) Complete "Divergent" in 100 seconds or less as the flash drive.
  • App Store Connect entry
  • Implementation
• It's About Perspective (15pts) Break out of bounds and find the secret.
  • App Store Connect entry
  • Implementation
• Completionist (100pts) Complete The Costumemaster. (Replaced with Underneath It All)
  • App Store Connect entry
  • Implementation
• Costumemastery (50pts) Complete a level with 10 costume switches or less.
  • App Store Connect entry
  • Implementation

Update level configuration logic

Some parts that contain user data aren't being used in the level configuration, and these bits of code are cluttering up the GameScene class itself. Additionally, there's no defined way to mark an exit door since the location is currently overridden when there are multiple doors.

• Move requisite logic inside of the level configuration structure
• Add exit door as a property of the level configuration

Determine game's premise

Goal
I should determine what this game will essentially be. Below are the things I should look out for:

• What is the general plot of the game? Who are the characters? - Abandoned factory escape.
• Where does the game take place? - Abandoned factory.
• What mechanics will be involved with this puzzle game? - see pitch below.

Add a pause menu

Most games include a pause menu that lets a player get back to the main menu, configure options, or return to the level. A pause menu should be implemented to that it's easy to get to things while in-game.

• Create a scene with the pause menu buttons
• Create PauseScene class to hook up to pause scene
• Create a keybinding for pause menu (default is Esc.)
• Update GameScene and listen for keybinding

Finish implementing apply method in AIGameState

The method apply in AIGameState currently only covers two cases: the default case and the move up case. This function should be updated to account for new changes.

Add challenge game scene classes

Adding challenges may get tricky when implementing them inside of GameScene. To get around this, I could probably implement a subclass of GameScene that is dedicated to challenges and the different challenge types. Similarly, GameScene can be subclassed for AI purposes, though this can come later when Milestone 2 hits.

• Create ChallengeScene subclass
  • Add timed challenge component
  • Add costume increment components
  • (Maybe?) Add "least steps" component
• Change some scenes to use ChallengeScene instead of GameScene

Start implementing costume logic

Costumes are another important component to the game; they enable the player to manipulate the environment or perform different actions. The logic for this should be implemented. The following needs to be done:

• Add conditional logic on computer switches (see #3) to require a costume.
• Disable the walls' physics bodies when the bird costume is active.
• Add heavy objects into the game and implement specific pickups. (superseded by #19)

Add missing documentation

According to the docs site hosted at https://costumemaster.marquiskurt.net, the project is ~60% documented. Functions, structures, etc., that are important and undocumented should be documented before release.

Add a HUD to the game

Currently, the player cannot tell if they are holding an item and what costumes are available to them in a given level. A HUD should be designed so that the player knows this information.

Limitations

• I am unable to target a corner of the game without doing extremely complicated maths between the window and the SpriteKit scene due to the conflicting coordinate systems.
  • Likewise, the HUD should be compact but informative.
• I should be able to put this UI in the GameScene class without adding this HUD node to the level files themselves.

Design better assets

Proper visuals make the game more appealing and easy to read/follow. This isn't necessarily important for the AI agents since they'll be gaining knowledge from the code in and of itself, but it certainly makes debugging look cooler. Forked from #2. This is what needs to be done:

• Update the wall textures to a wider variety of tiles
• Add textures for computers, switches, timers, etc.
• Update costume change assets.
• Add and implement walking animations.

Add new biometric input

The game does allow for a state where the player can be costume-less, but it isn't used. Along with the death pit feature update, I want to introduce a new biometric input door. The implementation should be like this:

• The player must not be wearing any costumes to activate the input.
• The input work similarly to a computer where a costume check is affected.
• The door connecting to this input will remain on for a few seconds.
• If the player is near the door and they can't get to the lever, it will open for them.

Extend Player to be suitable for AI

Currently, the player moves with a face being applied to the physics body of the player's sprite. However, if an AI were to call move(_:unit:), the AI won't move much unless the function is repeatedly called.

To get around this, there should be an additional move function via overloading that accounts for this. The implementation might look like the following:

func move(_ direction: PlayerMoveDirection) {
   while newPosition.distance(between: currentPosition) < 64 {
        self.move(direction, unit: CGSize(width: 128, height: 128)
   }
}