umair-nasir14 / word2world

Word2World is an LLM-based PCG system that creates playable 2D world from stories

Word2World

This repository contains to code for Word2World: Generating Stories and Worlds through Large Language Models.

Abstract:

Large Language Models (LLMs) have proven their worth across a diverse spectrum of disciplines. LLMs have shown great potential in Procedural Content Generation (PCG) as well, but directly generating a level through a pre-trained LLM is still challenging. This work introduces Word2World, a system that enables LLMs to procedurally design playable games through stories, without any task-specific fine-tuning. Word2World leverages the abilities of LLMs to create diverse content and extract information. Combining these abilities, LLMs can create a story for the game, design narrative, and place tiles in appropriate places to create coherent worlds and playable games. We test Word2World with different LLMs and perform a thorough ablation study to validate each step.

Usage:

Clone the repo:

https://github.com/umair-nasir14/Word2World.git

Install the environment and activate it:

cd Word2World
type > word2world/.env
conda env create -f environment.yml
conda activate word2world

Add your API key to the .env file created in word2world folder:

OPENAI_API_KEY="sk..."

Run with default configs:

python main.py

Or run with specified configs:

python main.py \
--model="gpt-4-turbo-2024-04-09" \
--min_story_paragraphs=4 \
--max_story_paragraphs=5 \
--total_objectives=8 \
--rounds=1 \
--experiment_name="Your_World" \
--save_dir="outputs"

To play the generated game:

python word2world/play_game.py "path_to_game_data\game_data.json"

where game_data.json is generated when the Word2World loop is finished and is saved to \outputs\game_data.json. This can be modified in configs or as --save_dir arg.

To play an example world:

python word2world/play_game.py

Results:

LLM comparison:

Worlds:

Note:

• The most stable model is "gpt-4-turbo-2024-04-09".
• Currently only OpenAI models are supported.
• OS supported: Windows

To-dos:

• Add support for Anthropic.
• Add support for Groq.
• Add support for Linux.
• Clean Code for easy integrations of new platforms, e.g. huggingface.

Cite:

@article{nasir2024word2world,
  title={Word2World: Generating Stories and Worlds through Large Language Models},
  author={Nasir, Muhammad U and James, Steven and Togelius, Julian},
  journal={arXiv preprint arXiv:2405.06686},
  year={2024}
}

word2world's Issues

PackagesNotFoundError while building environment

Hello, thanks for your interesting work. I tried to run Word2World but stuck in building conda environment.

When I run conda env create -f environment.yml , the errors below were reported:

Channels:

• defaults
• nodefaults
• conda-forge
  Platform: linux-64
  Collecting package metadata (repodata.json): done
  Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

• zlib==1.2.13=h8cc25b3_0
• xz==5.4.2=h8cc25b3_0
• wheel==0.41.2=py310haa95532_0
• vs2015_runtime==14.27.29016=h5e58377_2
• vc==14.2=h21ff451_1
• tk==8.6.12=h2bbff1b_0
• sqlite==3.41.2=h2bbff1b_0
• setuptools==68.0.0=py310haa95532_0
• pip==23.2.1=py310haa95532_0
• openssl==3.0.11=h2bbff1b_2
• libffi==3.4.4=hd77b12b_0
• ca-certificates==2023.08.22=haa95532_0
• bzip2==1.0.8=he774522_0

Current channels:

• https://repo.anaconda.com/pkgs/main/linux-64
• https://repo.anaconda.com/pkgs/r/linux-64
• https://conda.anaconda.org/nodefaults/linux-64
• https://conda.anaconda.org/conda-forge/linux-64

To search for alternate channels that may provide the conda package you're
looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.

Specified version might be tricky sometimes, so I tried to remove the last version code.
Even so, still two packages were not found:

• vs2015_runtime=14.27.29016*
• vc=14.2*

My environment is Linux, conda version is 24.3.0.

I'm really interested at your work and wonder if I can get help here to run this project.

Thanks!

[Linux] Exceptions/Warning during generating stage

@umair-nasir14

Hi! Thanks for all the help you gave me, I already ran your project successfully!

However, as we discussed, there are some warnings/exceptions happened during generating stage on below:

TOTAL REWARD for EPISODE: 350
Exception raised: TiffWriter.write() got an unexpected keyword argument 'fps'
 Traceback (most recent call last):
  File "/home/jupyter/Word2World/word2world/generators/openai_model.py", line 688, in action_generation
    video.append_data(frame)
  File "/opt/conda/envs/word2world/lib/python3.10/site-packages/imageio/v2.py", line 226, in append_data
    return self.instance.write(im, **self.write_args)
  File "/opt/conda/envs/word2world/lib/python3.10/site-packages/imageio/plugins/tifffile_v3.py", line 224, in write
    self._fh.write(image, **kwargs)
TypeError: TiffWriter.write() got an unexpected keyword argument 'fps'

Exception raised: '>' not supported between instances of 'builtin_function_or_method' and 'int'
 Traceback (most recent call last):
  File "/home/jupyter/Word2World/word2world/generators/openai_model.py", line 434, in world_generation
    story_paragraphs, total_objectives, no_of_important_tiles, bad_feedback_prompt, good_feedback_prompt = self.feedback_checks(world_eval_dict, previous_eval, story_paragraphs, total_objectives, no_of_important_tiles)
  File "/home/jupyter/Word2World/word2world/generators/generation_base.py", line 90, in feedback_checks
    if round > 0:
TypeError: '>' not supported between instances of 'builtin_function_or_method' and 'int'

Exception raised: 'NoneType' object is not subscriptable
 Traceback (most recent call last):
  File "/home/jupyter/Word2World/word2world/generators/openai_model.py", line 529, in action_generation
    astar_path, _, _, game_map_updated, _ = astar_agent.getSolution(game_state,maxIterations=10000)
  File "/home/jupyter/Word2World/word2world/solvers.py", line 286, in getSolution
    queue.put(Node(state.clone(), None, None))
  File "/home/jupyter/Word2World/word2world/solvers.py", line 622, in clone
    clone.player={"x":self.player["x"], "y":self.player["y"]}
TypeError: 'NoneType' object is not subscriptable

Exception raised: 'NoneType' object is not subscriptable
 Traceback (most recent call last):
  File "/home/jupyter/Word2World/word2world/generators/openai_model.py", line 607, in action_generation
    state, _r, done, _ = env.step(action)
  File "/home/jupyter/Word2World/word2world/agent.py", line 76, in step
    self.move_player(action)
  File "/home/jupyter/Word2World/word2world/agent.py", line 90, in move_player
    new_row = self.player_pos[0] + dx
TypeError: 'NoneType' object is not subscriptable

Exception raised: max() arg is an empty sequence
 Traceback (most recent call last):
  File "/home/jupyter/Word2World/word2world/generators/openai_model.py", line 427, in world_generation
    llm_agent_reward, astar_path, objectives = self.action_generation(round,story['choices'][0]['message']['content'],"protagonist","antagonist", character_discriptions_dict,world_map_fixed,world_map_fixed_with_chars,used_char_dict,used_char_dict_with_char,"color_tiles_img_with_char",
  File "/home/jupyter/Word2World/word2world/generators/openai_model.py", line 702, in action_generation
    return max(all_episodes_rewards), len(astar_path), objective_tile_dict
ValueError: max() arg is an empty sequence

You said these warnings wouldn't hurt the quality of generated stage, but some look kinda dangerous.
The world I generated didn't generated enemy, maybe it's because I just ran the algorithm for only two rounds?

I'll upload and merge the environment on my Linux machine later. It might not be the minimal environment but works on my Linux machine.

May I ask a small question here?
Is it possible to generate games in other domain using Word2World? (e.g. billiards/ bowling game)
It seems very possible if we can adjust the algorithm and prompt to LLM for me, just curious what you think about this possibility of Word2World.

Thanks!