Comments (3)
Are you running your code on a Slurm based cluster? If so are you running an array of jobs? The way we currently run this code is by bashing an array of Slurm jobs and use the array_idx to chunk the jobs (so we can benefit from running on multiple CPUs). I could share bash script for that if you want. Let me know
from camel.
There are probably plenty of us that gets excited about this, and despite us having no right to expect such a contribution, it would be fantastic to get the bash script and maybe even a small instruction set regarding setting up/using a Slurm cluster as well.
Are there any simpler, standalone, versions of the script for the code example that you just haven't released yet?
from camel.
Sorry for the late reply @phober ! This is a sample slurm script to run the generation of AI Society. In this example we chunk the 50 roles into 5 jobs each containing 10 assistant roles. Even though this used to be very helpful before, it seems that with the rate limit we are not really benefiting from this much anymore as we exceed rate limit when we do that. Setting up a slurm cluster is a way bigger thing (we ourselves never set one but rather use the one we have at the university).
#!/bin/bash --login
#SBATCH --job-name=ChatGPT
#SBATCH --output=Output/%A_%a.out
#SBATCH --error=Error/%A_%a.err
#SBATCH --array=0-4
#SBATCH --time=1:59:00
#SBATCH --mem=50G
#SBATCH --cpus-per-task=20
export OPENAI_API_KEY="..."
conda activate camel
python examples/ai_society/role_playing_multiprocess.py
from camel.
Related Issues (20)
- [Question] I want to know can the camel choose the role automatically according to the preliminary? HOT 1
- [Roadmap] Make markdown tutorials for core modules HOT 2
- [BUG] Difficulty setting OpenAI API key environment variable on Windows HOT 4
- Refactor `EmbodiedAgent.execute_code` with `CodePrompt.execute`
- Refactor `EmbodiedAgent.get_explanation_and_code` with `BaseMessage.extract_text_and_code_prompts`
- [Feature Request] Add `code` scenario to the demo UI HOT 1
- [Feature Request] Control code excution in `CodePrompt.execute` HOT 3
- [BUG] The package uses input() as core functionality
- [BUG] Task files could be downloaded automatically if no exist
- [BUG] Code defects in examples and in camel package
- [Feature Request] Refactor code to simplify the concepts HOT 2
- [BUG] Call super() to class without a base class HOT 1
- [Feature Request] Add examples for generating `math` and `science` datasets
- [Feature Request] More reliable and user-friendly chat history compression
- [Feature Request] Typing checking and data validation with `pydantic` HOT 2
- [Feature Request] Support streaming output and remove `print_text_animated`
- [Feature Request] Format prompt and extract template key words in a cleaner way
- [Feature Request] Allow setting output language from UI
- [Feature Request] Reproduce more SOTA LLM agents HOT 8
- [Feature Request] OpenAI function call HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from camel.