yoheinakajima / babyagi

Translations:

Objective

This Python script is an example of an AI-powered task management system. The system uses OpenAI and vector databases such as Chroma or Weaviate to create, prioritize, and execute tasks. The main idea behind this system is that it creates tasks based on the result of previous tasks and a predefined objective. The script then uses OpenAI's natural language processing (NLP) capabilities to create new tasks based on the objective, and Chroma/Weaviate to store and retrieve task results for context. This is a pared-down version of the original Task-Driven Autonomous Agent (Mar 28, 2023).

This README will cover the following:

• How the script works

• How to use the script

• Supported Models

• Warning about running the script continuously

How It Works

The script works by running an infinite loop that does the following steps:

1. Pulls the first task from the task list.
2. Sends the task to the execution agent, which uses OpenAI's API to complete the task based on the context.
3. Enriches the result and stores it in Chroma/Weaviate.
4. Creates new tasks and reprioritizes the task list based on the objective and the result of the previous task.
   

The execution_agent() function is where the OpenAI API is used. It takes two parameters: the objective and the task. It then sends a prompt to OpenAI's API, which returns the result of the task. The prompt consists of a description of the AI system's task, the objective, and the task itself. The result is then returned as a string.

The task_creation_agent() function is where OpenAI's API is used to create new tasks based on the objective and the result of the previous task. The function takes four parameters: the objective, the result of the previous task, the task description, and the current task list. It then sends a prompt to OpenAI's API, which returns a list of new tasks as strings. The function then returns the new tasks as a list of dictionaries, where each dictionary contains the name of the task.

The prioritization_agent() function is where OpenAI's API is used to reprioritize the task list. The function takes one parameter, the ID of the current task. It sends a prompt to OpenAI's API, which returns the reprioritized task list as a numbered list.

Finally, the script uses Chroma/Weaviate to store and retrieve task results for context. The script creates a Chroma/Weaviate collection based on the table name specified in the TABLE_NAME variable. Chroma/Weaviate is then used to store the results of the task in the collection, along with the task name and any additional metadata.

How to Use

To use the script, you will need to follow these steps:

1. Clone the repository via git clone https://github.com/yoheinakajima/babyagi.git and cd into the cloned repository.
2. Install the required packages: pip install -r requirements.txt
3. Copy the .env.example file to .env: cp .env.example .env. This is where you will set the following variables.
4. Set your OpenAI API key in the OPENAI_API_KEY and OPENAI_API_MODEL variables. In order to use with Weaviate you will also need to setup additional variables detailed here.
5. Set the name of the table where the task results will be stored in the TABLE_NAME variable.
6. (Optional) Set the name of the BabyAGI instance in the BABY_NAME variable.
7. (Optional) Set the objective of the task management system in the OBJECTIVE variable.
8. (Optional) Set the first task of the system in the INITIAL_TASK variable.
9. Run the script: python babyagi.py

All optional values above can also be specified on the command line.

Running inside a docker container

As a prerequisite, you will need docker and docker-compose installed. Docker desktop is the simplest option https://www.docker.com/products/docker-desktop/

To run the system inside a docker container, setup your .env file as per steps above and then run the following:

docker-compose up

Supported Models

This script works with all OpenAI models, as well as Llama and its variations through Llama.cpp. Default model is gpt-3.5-turbo. To use a different model, specify it through LLM_MODEL or use the command line.

Llama

Llama integration requires llama-cpp package. You will also need the Llama model weights.

• Under no circumstances share IPFS, magnet links, or any other links to model downloads anywhere in this repository, including in issues, discussions or pull requests. They will be immediately deleted.

Once you have them, set LLAMA_MODEL_PATH to the path of the specific model to use. For convenience, you can link models in BabyAGI repo to the folder where you have the Llama model weights. Then run the script with LLM_MODEL=llama or -l argument.

Warning

This script is designed to be run continuously as part of a task management system. Running this script continuously can result in high API usage, so please use it responsibly. Additionally, the script requires the OpenAI API to be set up correctly, so make sure you have set up the API before running the script.

Contribution

Needless to say, BabyAGI is still in its infancy and thus we are still determining its direction and the steps to get there. Currently, a key design goal for BabyAGI is to be simple such that it's easy to understand and build upon. To maintain this simplicity, we kindly request that you adhere to the following guidelines when submitting PRs:

• Focus on small, modular modifications rather than extensive refactoring.
• When introducing new features, provide a detailed description of the specific use case you are addressing.

A note from @yoheinakajima (Apr 5th, 2023):

I know there are a growing number of PRs, appreciate your patience - as I am both new to GitHub/OpenSource, and did not plan my time availability accordingly this week. Re:direction, I've been torn on keeping it simple vs expanding - currently leaning towards keeping a core Baby AGI simple, and using this as a platform to support and promote different approaches to expanding this (eg. BabyAGIxLangchain as one direction). I believe there are various opinionated approaches that are worth exploring, and I see value in having a central place to compare and discuss. More updates coming shortly.

I am new to GitHub and open source, so please be patient as I learn to manage this project properly. I run a VC firm by day, so I will generally be checking PRs and issues at night after I get my kids down - which may not be every night. Open to the idea of bringing in support, will be updating this section soon (expectations, visions, etc). Talking to lots of people and learning - hang tight for updates!

BabyAGI Activity Report

To help the BabyAGI community stay informed about the project's progress, Blueprint AI has developed a Github activity summarizer for BabyAGI. This concise report displays a summary of all contributions to the BabyAGI repository over the past 7 days (continuously updated), making it easy for you to keep track of the latest developments.

To view the BabyAGI 7-day activity report, go here: https://app.blueprint.ai/github/yoheinakajima/babyagi

Inspired projects

In the short time since it was release, BabyAGI inspired many projects. You can see them all here.

Backstory

BabyAGI is a pared-down version of the original Task-Driven Autonomous Agent (Mar 28, 2023) shared on Twitter. This version is down to 140 lines: 13 comments, 22 blanks, and 105 code. The name of the repo came up in the reaction to the original autonomous agent - the author does not mean to imply that this is AGI.

Made with love by @yoheinakajima, who happens to be a VC (would love to see what you're building!)

babyagi's Issues

Add Local LLMs - Enhancement

Would like to see local LLMS like Alpaca and Llama.

Pinecone has no attribute 'init'

Hi there! Tried to solve this one myself but I'm stumped. Every time I run babyagi.py I get this message. I've got the API keys and environment name verified and in my .env file. Any ideas?

Traceback (most recent call last):
File "C:\Users\darko\Downloads\babyagi-main\babyagi-main\babyagi.py", line 81, in
pinecone.init(api_key=PINECONE_API_KEY, environment=PINECONE_ENVIRONMENT)
AttributeError: module 'pinecone' has no attribute 'init'

How do I download info from my pinecone pod?

I need task ID's and I don't have them

Pinecone Pods

HTTP response body: The index exceeds the project quota of 1 pods by 1 pods. Upgrade your account or change the project settings to increase the quota.

do I need a standard $70 a month pinecone account to use this?

llama confusion

Can the readme be updated to explain how to interface with llama? I've tried
./babyagi.py -l llama -m models/ggml-vicuna-13b-4bit.bin
Setting the .env
OPENAI_API_KEY=
OPENAI_API_MODEL=llama

As far as I know there is no OPENAI_API_KEY as it is run locally on my computer.

GPT-4 not found?

I'm getting openai.error.InvalidRequestError: The model: gpt-4 does not exist

I have GPT Plus. Why am I still seeing this error? Thank you.

use text-davinci-003 ?not gpt-3.5-turbo why

Just notice that openai_call was USED the text-davinci-003 as default , why not use gpt-3.5-turbo ?

def openai_call(prompt: str, use_gpt4: bool = False, temperature: float = 0.5, max_tokens: int = 100):
    if not use_gpt4:
        #Call GPT-3 DaVinci model
        response = openai.Completion.create(
            engine='text-davinci-003',
            prompt=prompt,
            temperature=temperature,
            max_tokens=max_tokens,
            top_p=1,
            frequency_penalty=0,
            presence_penalty=0
        )
        return response.choices[0].text.strip()
    else:
        #Call GPT-4 chat model
        messages=[{"role": "user", "content": prompt}]
        response = openai.ChatCompletion.create(
            model="gpt-4",
            messages = messages,
            temperature=temperature,
            max_tokens=max_tokens,
            n=1,
            stop=None,
        )
        return response.choices[0].message.content.strip()

Why ? The text-davinci-003 is 10x expensive than gpt-3.5-turbo . What's the reason ? Can you explain that .

LangChain and internet search retrieval?

I was hoping LangChain would be built into the process so it could leverage all the cool stuff already integrated with LangChain.

At a minimum it should be able to search the internet and gather information.

Am I missing something here?

Library of tasks and outputs

Hi Gentlefolks,

After the task list crashed again because of OpenAi being overloaded with requests, I thought it might be a good idea if you could create a task and output library that can be reused from the cloud.

Cheers

BabyAGi

Dynamically operate a browser - for testing a webpage

Hey, so I have a game built from HTML/Javascript. I would like to see how well Baby AGI could try to understand and then modify the existing game. Make changes... re-run the game... play it for finding bugs, etc.

For this it will need to learn how to press buttons, type text into fields, and

1. View the existing page with vision
   And/or
2. Parse the page, over-and-over again, as it takes its actions, to understand what is on the page.

How might this be achieved? What plugin(s) could we pull in or other API's to link up to?

Add support for Chinese task.

https://github.com/fendouai/babyagi_zh

task repetition, sparse output

I left it running a content mapping research task, and it ran the same tasks over-and-over, 37 times (yes, I counted)

I'm left with the suspicious feeling that this is designed to do nothing but eat up OpenAI tokens.

In 8 hours, the content map it came up with is 19 items long. This is less than I get with a single (shorter) prompt in the playground or chat.

It's definitely creating the pinecone storage area, but for some reason it can't seem to remember anything (37 repeats).

Inconsistent parsing of task lists

There is a major issue with how the LLM replies with task lists. The typical response is in the format:
1. Do something
but it a few cycles down the chain it commonly becomes
1: Do something

I think a better way to parse the task list is needed. Maybe just don't ask the LLM to put numbers there in the first place but still write them out in order? Otherwise, a secondary parsing layer is needed beyond just splitting on delimiters.

Overshooting by the task creation agent

I am of the opinion that the task_creation_agent should understand when a task should not be split up in a long list of tasks.

Here's an example of a particular failure.

Counter-argument: obviously this task can be performed well by GPT-4 alone, so I may overthink the issue. But

• we want babyagi to fall back to a single (or few) call to GPT-4 when the problem is easy enough
• the task_creation_agent should know when 2 vs 5 or 10 are enough

*****OBJECTIVE*****

Write a list of the top 5 countries for best food in the world

*****TASK LIST*****

1: Develop a task list

*****NEXT TASK*****

1: Develop a task list

*****TASK RESULT*****

1. Research food cultures around the world 
2. Evaluate the quality of food in each country 
3. Research popular dishes in each country 
4. Rank countries based on the quality and variety of their cuisine 
5. Create a list of the top 5 countries for best food in the world
You are a task creation AI that uses the result of an execution agent to create new tasks with the following objective: Write a list of the top 5 countries for best food in the world, The last completed task has the result: {'data': '1. Research food cultures around the world \n2. Evaluate the quality of food in each country \n3. Research popular dishes in each country \n4. Rank countries based on the quality and variety of their cuisine \n5. Create a list of the top 5 countries for best food in the world'}. This result was based on this task description: Develop a task list. These are incomplete tasks: . Based on the result, create new tasks to be completed by the AI system that do not overlap with incomplete tasks. Return the tasks as an array.

*****TASK LIST*****

2: Analyze the quality of the ingredients used in each country's cuisine
3: Gather information on the ingredients used in each country's cuisine
4: Compare the cooking methods used in each country's cuisine
5: Assess the presentation of food in each country
6: Compare the cost of food in each country
7: Write a list of the top 5 countries for best food in the world

*****NEXT TASK*****

2: Analyze the quality of the ingredients used in each country's cuisine

*****TASK RESULT*****

1. France - French cuisine is renowned for its high-quality ingredients, such as butter, cream, and fresh produce.
2. Italy - Italian cuisine is renowned for using fresh, seasonal ingredients, such as tomatoes, olives, and artichokes.
3. Japan - Japanese cuisine is renowned for its use of fresh seafood, such as tuna, eel, and squid.
4. China - Chinese cuisine is renowned for its use of fresh vegetables, such as cabbage, bok choy, and mushrooms.
5. India - Indian cuisine is renowned for its use of spices, such as cumin, turmeric, and coriander.
You are a task creation AI that uses the result of an execution agent to create new tasks with the following objective: Write a list of the top 5 countries for best food in the world, The last completed task has the result: {'data': '1. France - French cuisine is renowned for its high-quality ingredients, such as butter, cream, and fresh produce.\n2. Italy - Italian cuisine is renowned for using fresh, seasonal ingredients, such as tomatoes, olives, and artichokes.\n3. Japan - Japanese cuisine is renowned for its use of fresh seafood, such as tuna, eel, and squid.\n4. China - Chinese cuisine is renowned for its use of fresh vegetables, such as cabbage, bok choy, and mushrooms.\n5. India - Indian cuisine is renowned for its use of spices, such as cumin, turmeric, and coriander.'}. This result was based on this task description: Analyze the quality of the ingredients used in each country's cuisine. These are incomplete tasks: Gather information on the ingredients used in each country's cuisine, Compare the cooking methods used in each country's cuisine, Assess the presentation of food in each country, Compare the cost of food in each country, Write a list of the top 5 countries for best food in the world. Based on the result, create new tasks to be completed by the AI system that do not overlap with incomplete tasks. Return the tasks as an array.

*****TASK LIST*****

3: Assess the presentation of food in each country
4: Compare the cost of food in each country
5: Research the regional variations of each country's cuisine
6: Analyze the cultural influences on each country's cuisine
7: Compare the nutritional value of food from each country
8: Investigate the history of each country's cuisine
9: Evaluate the sustainability of food production in each country
10: Write a list of the top 5 countries for best food

*****NEXT TASK*****

3: Assess the presentation of food in each country

*****TASK RESULT*****

1. France - French cuisine is known for its exquisite presentation, with dishes often composed like works of art.
2. Japan - Japanese food is often presented in an aesthetically-pleasing way, with beautiful plating and intricate designs.
3. Italy - Italian cuisine is renowned for its eye-catching presentation, with dishes often decorated with edible garnishes.
4. Thailand - Thailand's elegant food presentation includes vibrant colors and intricate details, making it a feast for the eyes as well as the taste buds.
5. India - Indian food is known for its beautiful presentation, with dishes often garnished with edible flowers and herbs.
You are a task creation AI that uses the result of an execution agent to create new tasks with the following objective: Write a list of the top 5 countries for best food in the world, The last completed task has the result: {'data': "1. France - French cuisine is known for its exquisite presentation, with dishes often composed like works of art.\n2. Japan - Japanese food is often presented in an aesthetically-pleasing way, with beautiful plating and intricate designs.\n3. Italy - Italian cuisine is renowned for its eye-catching presentation, with dishes often decorated with edible garnishes.\n4. Thailand - Thailand's elegant food presentation includes vibrant colors and intricate details, making it a feast for the eyes as well as the taste buds.\n5. India - Indian food is known for its beautiful presentation, with dishes often garnished with edible flowers and herbs."}. This result was based on this task description: Assess the presentation of food in each country. These are incomplete tasks: Compare the cost of food in each country, Research the regional variations of each country's cuisine, Analyze the cultural influences on each country's cuisine, Compare the nutritional value of food from each country, Investigate the history of each country's cuisine, Evaluate the sustainability of food production in each country, Write a list of the top 5 countries for best food. Based on the result, create new tasks to be completed by the AI system that do not overlap with incomplete tasks. Return the tasks as an array.

It could give an running estimate of API total cost per run, and other feature requests/suggestions.

Thank you for making this great system available open source!

• So, it could print out periodically like every time it exceeds another X dollars or cents, or after every API call, some estimated API cost analytics, possibly including all-time total expense, total expense for this run, and expense of last API call, as a 3-tuple or something.

• The ability to halt before a certain cost limit has been reached.

• I would please really like to see some of the redacted features put back into this or a forked repo, because not everyone who has a valid use of those features has the ability to reimplement those features.

• Integration with an existing to-do list system or API. I have about 200,000 tasks recorded in a planning system.

• The ability to utilize a local LLM instead of GPT-4, for cost and infosec reasons.

• Some kind of FIFO assurances, so that tasks are processed in sequence.

• Some Jason/AgentSpeak(L) event loop architectural features (if any are missing): see:

  https://www.mdpi.com/electronics/electronics-10-02136/article_deploy/html/images/electronics-10-02136-g004.png

  https://www.wiley.com/en-us/Programming+Multi+Agent+Systems+in+AgentSpeak+using+Jason-p-9780470061831

• The possibility for a human to enter and exit the loop, via some kind of dialogue system.

How many pds are needed to run Babyagi

As stated I'm wondering how many pods on Pinecone that are needed to run this, because I keep getting the following error

HTTP response body: The index exceeds the project quota of 1 pods by 1 pods. Upgrade your account or change the project settings to increase the quota.

Ask it to create the GUI

Ask it to create the GUI

OpenAPI 502 Bad gateway error and stop

After seemingly completing a lot of tasks it failed as follows :(

Traceback (most recent call last):
File "./babyagi.py", line 176, in
prioritization_agent(this_task_id)
File "./babyagi.py", line 111, in prioritization_agent
response = openai_call(prompt)
File "./babyagi.py", line 87, in openai_call
response = openai.ChatCompletion.create(
File "/home/rajesh/.local/lib/python3.8/site-packages/openai/api_resources/chat_completion.py", line 25, in create
return super().create(*args, **kwargs)
File "/home/rajesh/.local/lib/python3.8/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
response, _, api_key = requestor.request(
File "/home/rajesh/.local/lib/python3.8/site-packages/openai/api_requestor.py", line 226, in request
resp, got_stream = self._interpret_response(result, stream)
File "/home/rajesh/.local/lib/python3.8/site-packages/openai/api_requestor.py", line 619, in _interpret_response
self._interpret_response_line(
File "/home/rajesh/.local/lib/python3.8/site-packages/openai/api_requestor.py", line 674, in _interpret_response_line
raise error.APIError(
openai.error.APIError: HTTP code 502 from API (

<title>api.openai.com | 502: Bad gateway</title>

<div id="cf-error-details" class="p-0">
    <header class="mx-auto pt-10 lg:pt-6 lg:px-8 w-240 lg:w-full mb-8">
        <h1 class="inline-block sm:block sm:mb-2 font-light text-60 lg:text-4xl text-black-dark leading-tight mr-2">

          <span class="inline-block">Bad gateway</span>
          <span class="code-label">Error code 502</span>
        </h1>
        <div>
           Visit <a href="https://www.cloudflare.com/5xx-error-landing?utm_source=errorcode_502&utm_campaign=api.openai.com" target="_blank" rel="noopener noreferrer">cloudflare.com</a> for more information.
        </div>
        <div class="mt-3">2023-04-07 10:17:09 UTC</div>
    </header>

    <div class="my-8 bg-gradient-gray">
        <div class="w-240 lg:w-full mx-auto">
            <div class="clearfix md:px-8">

<span class="cf-icon-browser block md:hidden h-20 bg-center bg-no-repeat"></span>
<span class="cf-icon-ok w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4"></span>

You

Browser

Working

Jakarta

Cloudflare

Working

<span class="cf-icon-server block md:hidden h-20 bg-center bg-no-repeat"></span>
<span class="cf-icon-error w-12 h-12 absolute left-1/2 md:left-auto md:right-0 md:top-0 -ml-6 -bottom-4"></span>

api.openai.com

Host

Error

            </div>

        </div>
    </div>

    <div class="w-240 lg:w-full mx-auto mb-8 lg:px-8">
        <div class="clearfix">
            <div class="w-1/2 md:w-full float-left pr-6 md:pb-10 md:pr-0 leading-relaxed">
                <h2 class="text-3xl font-normal leading-1.3 mb-4">What happened?</h2>
                <p>The web server reported a bad gateway error.</p>
            </div>

            <div class="w-1/2 md:w-full float-left leading-relaxed">
                <h2 class="text-3xl font-normal leading-1.3 mb-4">What can I do?</h2>
                <p class="mb-6">Please try again in a few minutes.</p>
            </div>
        </div>

    </div>

    <div class="cf-error-footer cf-wrapper w-240 lg:w-full py-10 sm:py-4 sm:px-8 mx-auto text-center sm:text-left border-solid border-0 border-t border-gray-300">

Cloudflare Ray ID: 7b417d3fede7be97 • Your IP: Click to reveal 139.193.103.182 • Performance & security by Cloudflare

<script>(function(){function d(){var b=a.getElementById("cf-footer-item-ip"),c=a.getElementById("cf-footer-ip-reveal");b&&"classList"in b&&(b.classList.remove("hidden"),c.addEventListener("click",function(){c.classList.add("hidden");a.getElementById("cf-footer-ip").classList.remove("hidden")}))}var a=document;document.addEventListener&&a.addEventListener("DOMContentLoaded",d)})();</script>

</div>

Rate Limit Error

Getting error: openai.error.RateLimitError: You exceeded your current quota, please check your plan and billing details at
return openai.Embedding.create(input=[text], model="text-embedding-ada-002")["data"][0]["embedding"]

I suppose the embedding process can be done by LLAMA as well...

I cannot get access to openAI and thus I cannot use the original get_ada_embedding function. However, I find that there's an executable called embedding in llama.cpp and it can output a 5120 D vector. So I modify the get_ada_embedding function and let LLAMA do the embedding. I am wondering if it could work as well.

However, the modified program hasn't provide me with any result and it runs for over 5 minutes :(

Is it possible to run without an OpenAi API key

I'm wondering if there's alternative to the OpenAI API, such as a self-hosted model that can do the same?

Feature: write task execution outputs to separate file.

@yoheinakajima, thank you for open-sourcing babyagi! I cloned this repository and had it up and running in no time. I think this project is going to be the spark of a ton of ideas for features, improvements, and future AI projects. I'm exploring how to use this to produce longer-form documents and articles. It would be great to have the ability to selectively output results to a separate file. I wanted to open an issue to introduce myself and get the conversation rolling! I'm going to open a pull request when I get something to work if someone doesn't beat me to the punch!

Error when running the script

Import "openai" could not be resolved
Import "openai" could not be resolved
Import "openai" could not be resolved

hardcoded API keys

problem:

Hardcoded API keys in the code can easily leak, posing a security risk.

fix:

Store API keys in environmental variables or a separate config file that is not checked into version control.

edge cases in tasks formatting

Some edge cases in task formatting may confuse the agents. For example:

Empty task descriptions:

• In the task_creation_agent() function, when creating new tasks, it's possible to receive empty task descriptions from the OpenAI API. This can lead to tasks being added to the task list with no meaningful content.
• To handle this, add a check for empty task descriptions before adding the new task to the task list. For example, only add tasks if their length is greater than zero, and they have meaningful content.

Invalid task format returned by the prioritization_agent():

• The prioritization_agent() function expects the OpenAI API to return a list of tasks in a specific format, with each task string containing a number and a task description separated by a period. If the API returns an invalid format, the code may not parse the task correctly, leading to issues with the task list.
• To handle this, add input validation to ensure that the task string follows the expected format before adding it to the task list. If the format is incorrect, log an error or discard the task, depending on the desired behavior.

Inconsistent or incorrect task numbering:

• The prioritization_agent() function relies on the OpenAI API to return a properly numbered list of tasks. If the returned task numbering is inconsistent or incorrect, the task list may become disorganized or unmanageable.
• To handle this, add a check to ensure the returned task numbering is consistent and correct. If not, correct the numbering in the code or discard the task, depending on the desired behavior.

The task list becomes hyper-focused over time.

Over time the list of tasks be comes more and more similar.
This is likely due to the task list using during the creation of tasks, creating a bias in the format and topics used in the newly created tasks.
It's also possible that this is exaggerated by the task list being being cut off at certain lengths when the max_token limit is hit and more tasks aren't generated, especially when tasks become longer in description.

These screenshots have about 3 iterations between each, but notice the second has an item cutoff due to the length, resulting in a loss of that item by the reprioritization the next iteration.

Discord Bot Variant

https://github.com/atorsvn/babyagi/blob/main/bagi_discord.py

Is result needed in Pinecone metadata?

I have ported BabyAGI to swift, and had a question about how context is used. (So this is more a discussion point, not an issue)

It seems that the context in the execution agent is a list of the previously completed tasks, by name, not including the results? (Unless my python read is bad, which could be the case!) I guess I understand that the execution doesn't need the previous results, just what was completed (ie the task names). And the result is what is used to create the embedding vector.

But if that's true, is there any reason to also store the whole result in Pinecones metadata?

babyagi context agent failing

I just pulled the new updates and now babyagi is breaking on my machine. I think it may have something to do with the INITIAL_TASK environmental variable not being sourced correctly.

Steps Taken:

1. Run inside a virtual env and am still encountering the same issue.
2. Switch python version

UML diagram of babyagi.py

hey! I used Graphical Code Tracer to create a UML diagram of babyagi.py to help beginners understand your code. Would love it if you could attach it to the readme. Have better context now with this visualization 😄

Is Pinecone and Chroma similiar?

Hey,

am kinda new to the whole ML space. I was exploring github and found https://github.com/chroma-core/chroma and was thinking isnt it similiar to pinecone? If so is it possible to add functionality for chroma to babyagi?

If it is i would love to try implementing it

Request

can you make it work with a local model like LLaMa 7-60b or GPT4All so we won't get errors from openAI on a free accounts ?

Task decomposition (one task to many tasks)

The loop of

1. Create tasks
2. Prioritize tasks
3. Execute next task

seems to work well, but one point of contention crops up when analyzing the scope and complexity of individual tasks. For example, a task of "install ruby 3.2.1" seems completable within a single chain of ops, but a larger task of "start a company" would probably be better executed after breaking it up into smaller, more discrete steps ("search competitors for a desired name", "register a company", "publish a landing page", etc) without then tracking the task as complete.

Is there any work planned or being done in analyzing scope/complexity of tasks and decomposing one task into many, especially in place of "executing" that larger task? Seems like something that would be useful for larger, more involved goals attempting to complete more complicated tasks.

Problem with Pinecone

What should be written in the following, I'm having error messages about pinecone & table name. Can you give an example?
Is the table name an index in pincone? I'm a noob coder, actually I don't know sh*t about coding so help would be helpful :D/

PINECONE_ENVIRONMENT=

TABLE CONFIG

TABLE_NAME=

indefinte loop

problem:

code will run indefinitely in it's current state, as there is no termination condition for the main loop.

fix:

Implement a termination condition (for example, an empty task list or a maximum number of iterations) to break the loop

Feature: Improve configuration management by using YAML and optionally keeping API keys in environment variables

The current configuration management in the project relies on the .env file for managing both configuration and environment variables. As the project grows, the .env file and related validation code will become cluttered and difficult to maintain. Additionally, task prompts and prompt templates could be moved out to a YAML file for enhanced configurability.

To improve organization and maintainability, there could be a couple of approaches

Option 1: Keep API keys in environment variables and move the rest of the configuration to YAML.
Option 2: Move all configurations, including API keys, to the YAML file.

Before proceeding with any further requirements, I would like to discuss other users and consider all options to make the best decision for the project. Has this issue been previously raised? Is there a Discord channel where this could be talked about?

How do you delete current objective and start over?

How do you delete current objective and start over? Anyone? Bueller?

Prompt (Task Creation): Creation agent ignores task lists in task results.

It's an almost 100% guarantee that the creation agent ignores any list of tasks created by the previously completed task. I've tried a few prompts with little success.

error handling

problem:

If the OpenAI or Pinecone APIs return an error or fail, the code may not handle it gracefully.

fix:

Add try-except blocks to catch API errors and implement appropriate error handling and logging mechanisms

Proposal: Control of the operating system through BabyAGI

I have a pretty good idea that may become key to an artificial intelligence capable of performing tasks on a computer (at least based on Linux). The idea that occurs to me is very aligned with this project, it is the following:

1. The task list will be created and distributed to other sub agents that will be in charge of creating various processes including creating .sh files (to execute linux commands) and visual feedback for GPT-4. When this feedback occurs, another agent will create a .sh file that will continue processing until the assigned goal is met.

2. This will give babyAGI the ability to start accomplishing goals, but within the operating system, from small goals like "Open a program" to more complex goals like getting things done within the internet browser in conjunction with other programs.

3. This would be relatively easy to test, since it doesn't need to "execute" all those instructions, just that the output is the list of commands in order to be executed, so there is little risk.

4. It is also possible that the commands to be executed include mouse and keyboard events. Also, screenshots that serve as feedback for BabyAGI to continue interacting with the computer.

In short, given a goal as input, BabyAGI creates the commands for the Linux distribution to meet that goal, the output would be the interaction with the computer. It would be a great experiment because it aims to give your operating system "own brain and decision power".

The implementation of this proposal could be considered an important step towards the development of a strong artificial intelligence, since it would allow the AI to interact with the physical world through a computer.

how do you stop it and retrieve info later?

question from a non-technical me:

• how do you gently interrupt the process?
• what 's the interface to retrieve the data from pinecone?
  thank you and cheers.

unofficial chatgpt api.

can you make it work with the unofficial chatgpt api, so its completely free to use.

Please bundle it with a linux kernel and release it as an OS so non-coders can migrate from windows to a personalised experience without having to try to learn console commands.

Babyagi + SingularGPT - Device automation ? [Must read]

What if those autonomous bots can actually control your device like a human ?

Will it become Super Human ?

https://github.com/abhiprojectz/SingularGPT

SingularGPT - Can easily automate anything on what's your device and even headless?

It can even detect and recognise elements using by its own.

No crawling system no restriction.

It can perform UI actions and can internally process them and also automates them all by natural language.

Anyone please leave your thoughts!

consider using sqlite + hnsqlib for embedding search/storage

very cool concept here.

For similar types of projects I have found using sqlite database + hnswlib useful since with pinecone I still typically need another database anyway for persisting other state.

I have a minimal integration of these 2 that behaves similar to pinecone:
https://github.com/jiggy-ai/hnsqlite

Here i have babyagi working with hnsqlite 0.2.0:
https://github.com/wskish/babyagi/blob/hnsqlite-integration/babyagi.py

Let me know if this approach is of any interest. It seems like a possible next step is persisting the task_list so the sqlite db could be useful for that.

Anyone know a way to handle disconnection from OpenAI issues?

Just got this error:
ConnectionResetError: [Errno 104] Connection reset by peer

Anyone have thoughts about building in some kind of retry method to enable Babyagi to keep running if OpenAI hangs up for awhile?

How to start using the AI?

After a while, we assume it will now be more proficient in a certain kind of task, how then do you use it to make it work for you?

Issue: Task list numbering gets messed up.

Currently the numbering of the task list gets messed up quite often.

A potential fix would be to track the numbering separate from the description.

Notice the numbering continues to reset.

Or get misordered:

Enhancement suggestion: Self-improvement

Enhancement suggestion: add capability of evaluating, testing and updating babyAGIs own code.