nousresearch / obsidian Goto Github PK
View Code? Open in Web Editor NEWMaybe the new state of the art vision model? we'll see 🤷♂️
License: Apache License 2.0
obsidian's People
Contributors
Stargazers
Watchers
Forkers
catle2aurecon anminhhung cevatkerim techthiyanes signalprime plurigrid merveenoyan josephrp avant-garde-ai rodrigobercini yiming992 aloobun art3mis0707 rochemedia mentordotgitobsidian's Issues
Any benchmark results?
By the way, I think Flamingo 3B is also a multi-modal LVLM in size 3B
Pin pydantic and fastapi versions
Hello 👋
This repository has both deepspeed and fastapi as a dependency. deepspeed doesn't support pydantic > 2.0.0 which results in below error (included in DeepSpeed/issues/3963) when I install this repository with pip:
RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
'FieldInfo' object has no attribute 'required'
For the time being, it would be nice to pin the fastapi and pydantic versions in this repository.
Addition: Also llava requires transformers 4.31.0 but Obsidian depends on Mistral integration which is >4.34.0
How do you actually use the model?
I didn't see in the readme how to actually use the model. I'd like to try using it as a replacement for the Llava models if that's even possible using the transformers library...
Transformers 4.35.0 support
I had the misfortune of following the instructions 5 hours after release of transformers v4.35, the instructions guide to upgrade to the latest release, so I got the following error:
$ python -m llava.serve.controller --host 0.0.0.0 --port 10000 (obsidian)
[2023-11-02 21:08:06,589] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Traceback (most recent call last):
File "[...]/miniconda3/envs/obsidian/lib/python3.10/runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "[...]/miniconda3/envs/obsidian/lib/python3.10/runpy.py", line 110, in _get_module_details
__import__(pkg_name)
File "[...]/Obsidian/llava/__init__.py", line 1, in <module>
from .model import LlavaLlamaForCausalLM
File "[...]/Obsidian/llava/model/__init__.py", line 3, in <module>
from .language_model.llava_mpt import LlavaMPTForCausalLM, LlavaMPTConfig
File "[...]/Obsidian/llava/model/language_model/llava_mpt.py", line 26, in <module>
from .mpt.modeling_mpt import MPTConfig, MPTForCausalLM, MPTModel
File "[...]/Obsidian/llava/model/language_model/mpt/modeling_mpt.py", line 19, in <module>
from .hf_prefixlm_converter import add_bidirectional_mask_if_missing, convert_hf_causal_lm_to_prefix_lm
File "[...]/Obsidian/llava/model/language_model/mpt/hf_prefixlm_converter.py", line 15, in <module>
from transformers.models.bloom.modeling_bloom import _expand_mask as _expand_mask_bloom
ImportError: cannot import name '_expand_mask' from 'transformers.models.bloom.modeling_bloom' ([...]/miniconda3/envs/obsidian/lib/python3.10/site-packages/transformers/models/bloom/modeling_bloom.py)
As the immediate workaround, downgrading to v4.34 (pip install --upgrade transformers==4.34.0) works.
[sample_notebook] Trying to run the sample notebook but facing some connectivity problem
Can you please help me spot the source of this error?,
Thanks!
Typo in the docs.
sh script/download_mm_projector.sh
should be sh scripts/download_mm_projector.sh
The Colab seems to have an issue
I tried it twice, confirmed the model had been fully downloaded then loaded the web server. The error when submitting the prompt is NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE
I tried it on my MPS device but it's not ready, I changed builder and model_worker to use MPS rather than CUDA and ran into the issue regarding half tensors like float16 not working, so I tried to move it to cpu but didn't get much further which is when I tried the Colab notebook.
Thanks for any input as to why or how to fix the Colab, thanks guys
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.