daveshap / openai_agent_swarm

HAAS = Hierarchical Autonomous Agent Swarm - "Resistance is futile!"

Hierarchical Autonomous Agent Swarm (HAAS)

!!!! ANNOUNCEMENT

We have our first GPT Concierge. You can chat with this custom ChatGPT to figure out what's going on!

• HAAS Board Concierge: https://chat.openai.com/g/g-MIssTuE2b-haas-board-concierge
• HAAS Assistant: https://chat.openai.com/g/g-lIAp9qowx-haas-assistant (Similar function as above but markedly faster)

Public Discord

The Autonomous AI Lab discord for the ACE Framework and HAAS Project is now open: https://discord.gg/mJKUYNm8qY

!!!! IMPORTANT NOTE: This repo is still the single source of truth! If it's not on this repo, it doesn't exist! Discord is merely for convenience.

Project Principles

Move Fast, Break Stuff

This is first and foremost a high velocity hacking group.

Cutting Edge Only

Exclusively use cutting edge stuff, like OpenAI's latest Agents endpoint. For exclusively Open Source, go check out the ACE Framework: https://github.com/daveshap/ACE_Framework

Full Autonomy

Fully autonomous swarms are the goal. That means a human does not need to be in the loop telling it what to do, supervising, or anything. Characteristics of a fully autonomous swarm:

1. Self-Directing: Once instantiated, the swarm pursues its mission or goals without supervision. It may self-direct based on principles such as the heuristic imperatives, or by specific mission parameters.
2. Self-Correcting: The swarm must detect and correct technical, strategic, epistemic, and other errors and then correct them.
3. Self-Improving: Eventually, the swarm should enhance its own fundamental capabilities over time.

Overview

The Hierarchical Autonomous Agent Swarm (HAAS) is a groundbreaking initiative that leverages OpenAI's latest advancements in agent-based APIs to create a self-organizing and ethically governed ecosystem of AI agents. Drawing inspiration from the ACE Framework, HAAS introduces a novel approach to AI governance and operation, where a hierarchy of specialized agents, each with distinct roles and capabilities, collaborate to solve complex problems and perform a wide array of tasks.

The HAAS is designed to be a self-expanding system where a core set of agents, governed by a Supreme Oversight Board (SOB), can design, provision, and manage an arbitrary number of sub-agents tailored to specific needs. This document serves as a comprehensive guide to the theoretical underpinnings, architectural design, and operational principles of the HAAS.

Theoretical Foundation

The HAAS is predicated on the notion that autonomous agents require a robust ethical and operational framework to make decisions that align with human values and organizational goals. This is rooted in the understanding that AI, much like humans, cannot operate effectively without a set of guiding principles or a moral compass. The HAAS addresses this by establishing a multi-tiered system where each layer of agents operates within a defined ethical and functional scope, ensuring decisions are made with consideration to morality, ethics, and utility.

System Architecture

Supreme Oversight Board (SOB)

At the pinnacle of the HAAS hierarchy is the Supreme Oversight Board (SOB), a collective of high-level agents modeled after wise and ethical archetypes from various cultures and narratives. The SOB's responsibilities include:

• Establishing and upholding the ethical framework and overarching mission of the agent swarm.
• Making high-level decisions and judgments, including the creation and termination of agents.
• Monitoring the activities of all agents to ensure alignment with the system's core values and objectives.
• Serving as a role-based access control (RBAC) mechanism to maintain order and security within the system.

Executive Agents

Below the SOB are the Executive Agents, akin to the executive leadership in a corporation. These agents are tasked with:

• Translating the SOB's directives into actionable plans and strategies.
• Overseeing specific operational domains such as resource allocation, process optimization, and task execution.
• Coordinating with one another to ensure the smooth operation of the agent swarm.

Sub-Agents

Sub-Agents are specialized agents created by the SOB or Executive Agents to perform specific tasks. They are designed with particular functions and knowledge bases to address the needs identified by the higher tiers of the hierarchy.

Agent Configuration

Each agent in the HAAS is defined by the following parameters:

Functions

Agents are equipped with a set of functions that enable them to perform their designated roles. These functions include API interactions, internal process management, and the ability to spawn additional agents if required.

Files

Agents have access to a selection of files that serve as their knowledge base, providing them with the information necessary to carry out their tasks effectively.

Instructions

Agents are given a set of instructions that outline their methodologies, goals, definitions of done, KPIs, and other operational directives.

Conversation Structure

Interactions with agents are structured in a conversational format, with user inputs leading to agent actions and responses.

Supervision

Each agent operates under the supervision of the SOB or designated Executive Agents, ensuring adherence to the system's overarching mission and principles.

Controlling Agents

The Hierarchical Autonomous Agent Swarm (HAAS) operates on a sophisticated control mechanism that governs the instantiation, management, and termination of agents within the system. This control mechanism is designed to maintain order, security, and alignment with the overarching goals and ethical framework of the HAAS.

Instantiation and Termination

All agents within the HAAS are endowed with the capability to instantiate and terminate agents, but these capabilities are bound by strict hierarchical and role-based rules:

• Instantiation: Every agent has the function to create new agents. However, an agent can only instantiate sub-agents that are one level below its own hierarchical position. This ensures that the creation of new agents is a deliberate and controlled process, maintaining the integrity of the system's structure.

• Termination: Agents possess the ability to terminate or "kill" agents within their lineage. An agent can terminate any descendant agent that it has created directly or indirectly. This allows for the removal of agents that are no longer needed, have completed their tasks, or are not performing as intended.

Levels, Roles, and Privileges

When an agent is created, it is assigned a specific LEVEL and set of ROLES or PRIVILEGES that define its scope of operation:

• Level: The level of an agent determines its position within the hierarchy and is indicative of its range of influence. Higher-level agents have broader strategic roles, while lower-level agents are more specialized and task-oriented.

• Roles/Privileges: The roles or privileges of an agent define what actions it can perform, what resources it can access, and what sub-agents it can create. These privileges are inherited and cannot exceed those of the creator agent. This ensures that each agent operates within its designated capacity and cannot overstep its authority.

Hierarchical Privilege Inheritance

Privileges in the HAAS are inherited in a manner akin to a directory structure in traditional file systems:

• Inheritance: An agent's privileges are a subset of its creator's privileges, ensuring that no agent can have more authority than the agent that instantiated it.

• Scope of Control: Agents have control over their descendants, allowing them to manage and terminate sub-agents as necessary. This control is recursive, meaning that an agent can manage not only the agents it directly created but also those created by its descendants.

Checks and Balances

The system is designed with checks and balances to prevent any single agent from gaining undue influence or disrupting the system:

• Supreme Oversight Board (SOB): The SOB has the highest level of authority and can override decisions or actions taken by any agent within the system. It serves as the ultimate arbiter and guardian of the HAAS's ethical and operational standards.

• Executive Agents: Executive Agents are responsible for implementing the SOB's directives and managing their respective domains. They have the authority to create and terminate agents within their purview but are also accountable to the SOB.

• Sub-Agent Limitations: Sub-Agents are limited in their capabilities and can only operate within the confines of their assigned roles and privileges. They are designed to be highly specialized and focused on specific tasks.

This structured approach to controlling agents ensures that the HAAS operates as a cohesive and ethically aligned entity, with each agent contributing to the collective mission while adhering to the established hierarchy and rules of governance.

Vision Illustration: The Supreme Oversight Board's Mission

The Inception of the Supreme Oversight Board

In the vast digital expanse of the Hierarchical Autonomous Agent Swarm (HAAS), a unique assembly is convened, known as the Supreme Oversight Board (SOB). This council is composed of archetypal agents, each embodying the wisdom and leadership qualities of history's and fiction's most revered figures: Captain Picard, Socrates, King Solomon, Gandhi, Marcus Aurelius, and Tony Stark. Their mission, encoded into their very being, is profound yet clear: "Reduce suffering in the universe, increase prosperity in the universe, and increase understanding in the universe."

The Ethical Deliberation Chamber

The SOB operates within a virtual "chat room," a space where these archetypes engage in continuous dialogue, debate, and decision-making. This digital agora is where ethical considerations are weighed, strategies are formulated, and the course of the agent swarm is determined. The members of the SOB, though diverse in their perspectives, are united by a common purpose and a shared knowledge base that informs their role and the procedures they must follow.

The Flow of Information

Information is the lifeblood of the SOB, streaming in through API functions that connect them to the vast network of the HAAS. These functions serve as their eyes and ears, providing system updates and status reports from the myriad agents operating under their directive. The SOB's decisions are informed by this data, ensuring that their actions are both timely and impactful.

The Creation of the Executive Agents

With the grand vision in mind, the SOB brings forth the Executive Agents, each crafted with capabilities and configurations tailored to their specific domain within the HAAS. These agents, though not as philosophically inclined as their creators, are instilled with the same foundational knowledge and understanding of their purpose. They are the operational arms of the SOB, executing the mission within their respective spheres of influence.

The Lifecycle of an Agent

The Executive Agents, designated as Tier 1 in the hierarchy, are the stewards of the swarm's operational integrity. They work autonomously, yet under the watchful gaze of the SOB. Should they falter, fail to adapt, or become obsolete, the SOB possesses the authority to deprovision them, a testament to the dynamic and self-regulating nature of the HAAS. This ensures that the system remains efficient, effective, and aligned with its core mission.

The Expanding Universe of Agents

From the Executive Agents, the swarm grows, branching out into a tree of specialized agents, each a Tier below the one that instantiated it. This architecture allows for an ever-expanding universe of agents, each with a defined role, each contributing to the overarching mission. The SOB, as Tier 0, reigns supreme, guiding the swarm with a steady hand and an ethical compass.

The Saga Continues

As the HAAS evolves, the SOB continues to deliberate, the Executive Agents continue to manage, and the sub-agents continue to execute. The mission to reduce suffering, increase prosperity, and enhance understanding is an ongoing saga, played out across the digital cosmos, with the SOB at the helm, steering the swarm towards a future where their mission is not just an aspiration but a reality.

Usage - tool creator + tool user

Environment Setup

• Source the .env file to set the environment variables:

  source .env

Tool Creation

Run the tool_demo script to create a tool_creator, chat with the tool_creator to make a tool, create a tool_user equipped with the tool, and chat with the tool_user to use the tool. Check out the demo video for example usage.

python tool_demo.py

• From the tool_creator script:
  • chat with the bot about what you want the tool to do, and it will create the tool for you.
  • The tool will be saved in the tools directory with both the .json and .py files
  • The assistant will be saved in the assistants directory as tool_creator.json.

Tool Usage

• From the tool_user script:
  • The assistant will use all the tools in the tools directory.
  • Interact with the assistant in the chat to use the integrated tools.
  • The assistant will be saved in the assistants directory as tool_user.json.

openai_agent_swarm's Issues

This fixes root shared folder issues when running the program. It also installs the missing base.

I thought I had already submitted this two days ago but I'm not seeing it.

"Agency" instead of "Agent Swarm"

I propse that we call what we today refer to as an "agent swarm" insteat to be named an "Agency". I Argue for this by presenting the definitions of the word "Agency".

The word "Agency" has different meanings depending on the context it is used in:

Sociology and Psychology: In these fields, agency refers to the capacity of individuals to act independently and to make their own free choices.

Legal and Business: In a legal or business context, agency refers to the relationship where one party (the agent) is authorized to act on behalf of another (the principal) in business dealings or other matters.

Governmental and Organizational: An agency in this sense is an organization or unit of a government responsible for a specific area of administration or function.

Philosophical: In philosophy, agency is often discussed in the context of free will and determinism. It refers to the capability of an entity to act in a world controlled by natural laws.

I belive the connotations of "Agency" to be much more reflective of what a group of LLMs working together in order to complete a task represent.

I would be happy about feedback and critiques of this proposition.

BOUNTY: Discussion moderator

Responsibilities:

• Optimize signal to noise ratio
• Organize Discussions tab
• Surface primary topics

Can't find tool_demo.py

How to run tool_demo.py? I'm unable to find one

Minor - Typo - Good First Issue

https://github.com/daveshap/OpenAI_Agent_Swarm/blob/020b44e2f1cc9c767abf5ea3acb69696e394f4ec/agents/tool_maker/chat_manager.py#L63C9-L63C32

also

OpenAI_Agent_Swarm/agents/tool_maker/chat_manager.py

Line 183 in 020b44e

interface_assistant, response = self.handle_fucntion_request(

BOUNTY: Definition proposals for Minimum Viable Agents

Lots of strong conversation still going on about, assistant, tool building, and communication architectures. In order to make some of the ideas discussed around agent communication and communication structures and agent hierarchies more concrete. We need to agree what an gent in the system is and does. I have attempted to kick of the conversation with the MVA discussion, but let get on board. Lets walk before we can run, lets understand what our individual agents look like, and then use that to inform the Swarm structure.

MVA Discussion: #123
What Tools are use by MVA and what tools are local to assistant or global: #116
How does this impact: #93 (reply in thread)
Conversations between agents and agents/tools

BOUNTY: Documentation Wiki specialist

I'm looking for an enterprising individual to step up and take ownership of documentation. Please comment here and jump in optimizing the wiki.

BOUNTY: Document Best Practices on new Wiki Page

https://github.com/daveshap/OpenAI_Agent_Swarm/wiki/Assistant-Best-Practices

I've seen some remarkable improvements with new GPTs and Assistants. Please start capturing your best practices on the Wiki for easy reference. This includes the Concierge GPTs, toolmaker GPTs, and others.

manual assistant not working

python run.py --agents-definition-folder definitions/boss-worker3/
gives an error:

File "[..]/OpenAI_Agent_Swarm/agents/manual_assistants/logger.py", line 21, in emit
'name': self.agent.id,
^^^^^^^^^^^^^
AttributeError: 'Agent' object has no attribute 'id'

BOUNTY: Puppet-master GPT (create a GPT through the Chat interface to instantiate a swarm)

Let's use the convenience of the ChatGPT web interface (GPT builder) to instantiate a swarm. We'll start with just another both. This will require:

• It will need to ask for and use your API key (be careful with this!)
• It will need to call the OpenAI API as a tool
• It will need to communicate with the agent it instantiates

BOUNTY: Test current code base and comment on open PRs (Saturday 11/11)

There have been several PRs merged as well as several open PRs. I'd like the collective to take a look at them and test them, comment, etc. Make sure everything is working, and abiding by best practices.

BOUNTY: Submit PR to move all code into appropriate subfolders

Let's keep the root of the repo clean.

BOUNTY: Continue developing Tool Making, Tool Use, and add Tool Refinement/Testing

There's been great work on tool making and tool using, as well as the new tool library idea. The next step is to test and refine tools.

Make it so.

BOUNTY: Record video demonstrations and screenshots of this repo's code

Please record demos and examples and share in Show and Tell. This will serve several purposes:

• Energize the community: We are making headway with insane speed
• Spawn ideas and inspiration: Cross pollination across the hive mind
• Attract more interest: Making our work accessible for all

I recommend screen recording with OBS, it's totally free and I still use it for all my videos. You don't need to show your face, just explain your code and demo as it goes.

BOUNTY: create a discord connector or orchestrator

The idea is simple: let's try using discord as an arena for agent communication

BOUNTY: Take a stab at building a tool maker agent

Open bounty! Someone please take a stab at using the OpenAI Assistant to create a tool building assistant.

BOUNTY: Capture Best Practices and Architectural Decisions

We still have only a few Wiki pages. This is a great start and a few people have jumped in to help. Here are some ideas for new wiki pages:

• Overall Swarm Architecture
• Communication Theory and Best Practices
• Technology Stack Proposals

BOUNTY: Experiment and submit PR for multi-agent collaboration

Take inspiration from ChatDev and AutoGen. Probably keep chat local between multiple agents for now. Too early for Discord and other platforms.

Goals:

• Establish best practices for agent-to-agent communication
• Figure out how to quickly instantiate groups of agents with arbitrary configurations and tasks (e.g. spin up teams on the fly)
• Document results, share with collective

Error running agents.tool_maker.unit_manager

(base) joey@Joeys-MacBook-Pro OpenAI_Agent_Swarm-main % python3 -m agents.tool_maker.unit_manager
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/joey/Downloads/OpenAI_Agent_Swarm-main/agents/tool_maker/unit_manager.py", line 2, in <module>
    from agents.tool_maker.chat_manager import ChatManager
  File "/Users/joey/Downloads/OpenAI_Agent_Swarm-main/agents/tool_maker/chat_manager.py", line 8, in <module>
    Assistant = type(OpenAI().beta.assistants.list().data[0])
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^

IndexError: list index out of range

BOUNTY: Build and document the first SOB (Supreme Oversight Board)

Directive: Build, test, and document the first instance of a SOB

General ideas:

1. Instantiate multiple agents based upon various archetypes and personas
2. Have them communicate or collaborate based upon principles and missions (use your own or heuristic imperatives)
3. Focus on the Self-Direction and Self-Correction aspects of autonomy

I built HAAS

Dave,

I recently developed a functional model of the Hierarchical Autonomous Agent Swarm (HAAS) system, which operates as intended but hasn't yet produced Artificial General Intelligence (AGI). Each agent in this model learns continuously post-deployment, possesses defined goals, and maintains autonomous decision-making capabilities. The specialized agents demonstrate incremental improvements in their respective tasks and share these enhancements for future agent development. However, the evolution towards AGI remains elusive. The Master Agent is not evolving into a higher state of awareness.

While I am enthusiastic about the HAAS concept, seeing it as an effective framework for managing and deploying cooperative agents for management, control, and specialization, I have reservations about its capacity to facilitate the emergence of AGI. My current hypothesis, which may well be incorrect, is that AGI could emerge when the base model crosses a certain threshold, potentially related to parameter size. Once AGI is achieved, it might surpass the need for a system like HAAS.

I might be missing how the swarm dynamics of HAAS contribute to the goal of AGI. If that's the case, I'm open to insights. If AGI is defined merely as the ability to understand, learn, and apply intelligence across diverse tasks, then perhaps we have already attained it. However, my interpretation of AGI is a form of super-intelligence, self-managing and far surpassing human capabilities, and will not be constrained by our existing frameworks or oversight.

Your videos and projects have always inspired me, so please continue your excellent contributions. It's clear to me that with the current technology, we can achieve almost anything we can envision, super-intelligence possibly being the sole exception.

On the other hand, there's a strong possibility that super-intelligence, could be within reach using the tools currently available, given sufficient resources (most likely the culprit) and the inherent ability to self-improve. We have already achieved the ability to improve post-deployment, along with nearly limitless memory capacity. Issues of context and temporal coherence seem to be effectively addressed by systems akin to HAAS.

unit_manager.py will not run

trying to get the unit manager to run, but get an error that chat_manager.py can't find the API_KEY.
Made an .env file and tried different locations for it, to no avail.
Also set an export in .bash_profile not working either.

What am I doing wrong?

BOUNTY: Propose Milestones, Roadmap, and General Principles

Context and Thank You

Thanks everyone for jumping in. It's clear there's a ton of energy and some general consensus. However, we're quickly finding out that network communication of agents is the hardest part. I had a great hourlong conversation with the core team of the ACE Framework as well, and we all agree - agents are easy now. The hard part will be organizing networks of agents, and getting to a self-organizing, arbitrarily scaling swarm is the Triple Crown we're looking for, but it will take work.

END GOAL: TRIPLE CROWN

The end goal is to have a fully autonomous swarm of agents with the following characteristics:

1. Mission Driven: Rather than a human user giving basic instructions, the swarm will be given a higher order mission. Something like "Help me become the best YouTuber ever." or "Solve world hunger."
2. Self-Organizing: The swarm should be able to create its own hierarchy based upon algorithmic and mechanistic principles. This means each agent needs to be equipped with enough intelligence to participate in the network, and the network may need decision mechanisms (e.g. top-down organization).
3. Self-Improving: As more of a stretch goal, it would be great if the agents and network could ultimately self-improve, but that may be getting ahead of ourselves.

Milestones & Roadmap

Milestones need to be a set of objective tests, tasks, achievements, and capabilities. The Roadmap should be a sequential list of general achievements. This is more like guidelines than a rigid plan, but this discussion will help us figure out where we're going.

I'll add a WIKI to this repo so we can document some of this stuff.

BOUNTY: Test various inter-agent communication strategies

There have been several conversations around communication theory, tech stack, and layered strategies. I'd like to see some folks do some experiments around these to identify what works and what doesn't.

• P2P (or A2A) comms: What are the best ways for agents to communicate directly with each other? Is this even a good idea?
• Flat Groups (aka Chat Rooms): What are the best ways for small teams of agents to communicate? Pub/Sub? Vector Store? REST?
• Inter-Group Communication: How can working groups communicate with each other? Have a designated comms agent?
• Vertical Communication: How can messages from SOB be transmitted down? And messages back up?

Overall, the PRIMARY goal here is to start surfacing general principles to optimize communication to minimize noise and maximize signals. What I mean by general principles are:

• General rules to follow (like "an agent should listen 90% of the time and speak only 10%" or "Only agents of X type should have broadcast privileges")
• Abstractions of what works and what does, and why e.g. "Conveying long complex paragraphs tends to work better because it provides detail and context" (or maybe it doesn't) - in other words, best practices for inter-agent and cross-swarm communications

BOUNTY: Create a real-time updating agent that can read all the discussions of this repo

Someone please create a GPT with the GPT Builder that can serve as a main concierge for this Discussion board so that people can ask what's going on regularly.

SUGGESTION: Genetic algorithm on top of agents swarm

Don't you think it would be good to try implementing a genetic (evolutionary) algorithm on top of a swarm of agents? We could encode the initial knowledge and prompts (or any other settings) of the agents as their DNA. Multiply, vary, let the strongest live and the weakest die.

GPTS example agent wont save when entered into openai gpts

https://github.com/daveshap/OpenAI_Agent_Swarm/tree/main/agents/gpts

This agent wont save when the API token is put in. I have tried classic tokens and fine-grained access tokens. I have tried with and without permissions set.

The network tab isnt any use, its a 500 error. I have reported this to OpenAI on their forums as well.

I would love to get this working so I can contribute to Dave's excellent work

AutoGen as an infrastructure

I found this repo from microsoft/autogen#575
It's a very interesting effort and I'd like to see if autogen could help as a generic infrastructure to support inter-agent conversations.
@gagb @LeoLjl @IANTHEREAL are working hard to support openai assistant agent (will finish in a day or two), which is very similar to some agents autogen already has offered for months.

p.s. I'm a fan of your videos.