lavague-ai / lavague Goto Github PK
View Code? Open in Web Editor NEWLarge Action Model framework to develop AI Web Agents
Home Page: https://docs.lavague.ai/en/latest/
License: Apache License 2.0
Large Action Model framework to develop AI Web Agents
Home Page: https://docs.lavague.ai/en/latest/
License: Apache License 2.0
Add telemetry to help us improve performance/reduce failures:
Collect:
URL
Date
HTML code
Retrieved nodes by Llama Index
LLM used
Error message
Code produced
Screenshot
Assigning an ID to Selenium drivers and locking them for individual users
We have a monolithic installation process, for instance cuda is installed by default, which is too heavy for some use cases.
We will provide soon a version where the Action Engine can be provided through a SaaS to make it easier for people who don't want to do computing locally, so their package will be very lightweight.
@lyie28 Could you have a look? I guess this can be done in conjunction with #59
Hi,
First of all excellent work. I am facing issue with timeouts. I am able to install and run the demo. That's good. However when go the gradio app like this.
I typed in https://youtube.com
Then I supplied the following query
It went on to execute this statement forever. Moreover it does not display any errors or stack trace.
The command center module should:
Hello folks,
I'm trying to get the project off its feet with the source code.
I gave the file path in the lavague launch command parameters, this did not work.
I tried the commands in the normal document.
I started with the setup.sh file, there was no error here.
Chrome driver unzip was done successfully.
I am working in a virtual environment with conda.
There's openai.py file on path lavague-files and examples/api
When the command tried on those paths, chromedriver error.
lavague-launch --file_path hf.txt --config_path openai.py
Traceback (most recent call last):
File "/opt/anaconda3/envs/LaVague/bin/lavague-launch", line 8, in <module>
sys.exit(launch())
File "/Users/serdarkaracay/AiWebScraby/LaVague/src/lavague/__init__.py", line 121, in launch
action_engine, get_driver = load_action_engine(config_path, streaming=True)
File "/Users/serdarkaracay/AiWebScraby/LaVague/src/lavague/utils.py", line 18, in load_action_engine
config_module = import_from_path(path)
File "/Users/serdarkaracay/AiWebScraby/LaVague/src/lavague/utils.py", line 13, in import_from_path
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 846, in exec_module
File "<frozen importlib._bootstrap_external>", line 982, in get_code
File "<frozen importlib._bootstrap_external>", line 1039, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'openai.py'
(LaVague)
LaVague
❯ conda info
active environment : LaVague
active env location : /opt/anaconda3/envs/LaVague
shell level : 5
user config file : /Users/serdarkaracay/.condarc
populated config files : /Users/serdarkaracay/.condarc
conda version : 24.1.2
conda-build version : 24.1.2
python version : 3.11.7.final.0
solver : libmamba (default)
virtual packages : __archspec=1=m1
__conda=24.1.2=0
__osx=14.4.1=0
__unix=0=0
base environment : /opt/anaconda3 (writable)
conda av data dir : /opt/anaconda3/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/osx-arm64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/osx-arm64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /opt/anaconda3/pkgs
/Users/serdarkaracay/.conda/pkgs
envs directories : /opt/anaconda3/envs
/Users/serdarkaracay/.conda/envs
platform : osx-arm64
user-agent : conda/24.1.2 requests/2.31.0 CPython/3.11.7 Darwin/23.4.0 OSX/14.4.1 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.6 aau/0.4.3 c/QkeNtkUrgwz7nVYVSzIrAg s/5f_5979BIZ-G0e4oRXaKzg e/ATRt1h_Xl4cCAzCR1M6zcQ
UID:GID : 501:20
netrc file : None
offline mode : False
MacOS M1 Max Sonoma 14.4.1
Python 3.9.19
I realize that depending on the model, the generated code can have different issues and needs to be cleaned differently.
For instance, with Hugging Face API, the generation continues after the end of the markdown cell
# First we need to identify the component first, then we can click on it.
# Based on the HTML, the link can be uniquely identified using the class "w-full rounded-full border border-gray-200 text-sm placeholder-gray-400 shadow-inner outline-none focus:shadow-xl focus:ring-1 focus:ring-inset dark:bg-gray-950 h-7 pl-7"
# Let's use this class with Selenium to identify the link
search_bar = driver.find_element(By.XPATH, "//*[@class='w-full rounded-full border border-gray-200 text-sm placeholder-gray-400 shadow-inner outline-none focus:shadow-xl focus:ring-1 focus:ring-inset dark:bg-gray-950 h-7 pl-7']")
search_bar.click()
# Now we can type the asked input
search_bar.send_keys("The Stack")
# Finally we can press the 'Enter' key
search_bar.send_keys(Keys.ENTER)
---
HTML:
<!DOCTYPE html>
<html>
<head>
<title>Mock Page</title>
</head>```
For Azure OpenAI, it starts with ```python, with Mixtral on Fireworks AI it starts with a natural answer like "Here is an answer".
We should have the Action Engine be initialized with a cleaning function.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Do you plan to support playwright?
Problem: not clear how to install and run locally on macOS.
Proposed solution: Allow to install the tool via the Homebrew package manager on macOS. Example: brew install lavague
Add modular different export options that will interact with hub when built: python, pytests, FastAPI etc.
Is it possible to connect LM Studio?
If not, can you add it?
Thanks.
Take code from Google Colab example and make it into a modular Python library
Is your feature request related to a problem? Please describe.
Currently the docker has only 2 commands, build and launch, and it launches the config.py file, what If I want to use huggingface_interface.py?
Can we modify the entrypoint.sh? and add the support for huggingface and possibly for others?
Describe the solution you'd like
Modifying the entrypoint.sh file and adding the support for the same.
Describe alternatives you've considered
I am not sure about the alternatives, but modifying the entrypoint.sh seems a good solution at this point.
Additional context
-> Specs: doesn't require developer mode
Describe the bug
When simply running all fields one for one in the google colab I get the following:
NoSuchDriverException Traceback (most recent call last)
in <cell line: 40>()
38
39 # Choose Chrome Browser
---> 40 driver = webdriver.Chrome(service=webdriver_service, options=chrome_options)
41
42 # action_engine = ActionEngine(llm, embedder)
2 frames
/usr/local/lib/python3.10/dist-packages/selenium/webdriver/common/driver_finder.py in get_path(service, options)
42
43 if path is None or not Path(path).is_file():
---> 44 raise NoSuchDriverException(f"Unable to locate or obtain driver for {options.capabilities['browserName']}")
45
46 return path
NoSuchDriverException: Message: Unable to locate or obtain driver for chrome; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location
I was able to fix it by removing the service from the initialization so instead of # Choose Chrome Browser
driver = webdriver.Chrome(service=webdriver_service, options=chrome_options) I did driver = webdriver.Chrome( options=chrome_options)
Is your feature request related to a problem? Please describe.
It would be nice to support local llms as most companies have data privacy policies especially around recent AI developments
Describe the solution you'd like
It would be nice to point to a local llm or have some integration with standard libraries like ollama
When I run this, irrespective of which site I simply get "error" inside the virual screen. Tried several different websites and browsers and I get the same thing. No error in the code so presumably all dependencies are installed correctly... This example is the IRS site. Happens within the link and within colab.
Litellm is open source project you can read about them if you want https://github.com/BerriAI/litellm
Many projects are integrating with it because with this we can use easily 100+ different LLMs as it helps in creating local proxy server which api structure is open ai compatible and with one apo structure it becomes easy to change model and api key and use easily without additional configurations and reading and implementing from individual llm docs so please think about it
Is your feature request related to a problem? Please describe.
Add a way for people to test and run this locally - not in a codelab. Docker is an easy path for folks.
Describe the solution you'd like
I'd like clear instructions on how to run this locally.
@HiImMadness : see with @mbrunel to integrate the work you did on improving the retriever and the new prompt template to increase performance
#29 highlighted some recent issue with llama-index streaming not working anymore.
Have to fix it
It hasn't been effective on most of the sites I've attempted to use it on, as it's often identified as a bot. Are there any plans to enhance the agent's resilience against such detections?
examples/api
docs/docs/integrations following the other integration notebooks as an example
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.