sail-sg / clot Goto Github PK

10分钟跑不完一次，如何解决？？

Can you provide a complete usage process? For example, the required dependency environment can be found in the published model weights related to qwen. How should we use this aspect

We released our dataset 和 checkpoint. 👈😆Please try it!

I'm unable to access them, and I'm not certain what the issue might be.

这个好有意思，但是我真不明白要怎么跑起来。求一个对 ai 入门者更友好的使用教程，我研究了半天也没搞懂要什么环境以及需要下载哪些你们的配置等等...

How to use Apple Mac M1 chip

Some lines are "T2T" type, but still have no question, is there any mistake?

After deploy this project, I found two problems here.

1. When I use the example you provided, there is no results.
2. And if I just use text ti generate results, there always answer half sentence as the following pic shows.
   

When will the code and pre-trained model be released? I'm looking forward to it

Following Instruction templates in Appendix of your paper and provided code, it seems model learned some special tags like and box tags during training. Can you provide the specific prompt templates in both English and Chinese that contains those tags for ranking/selection/conditions? Thanks

fail to load the gradio page, any suggestion?

I used Qwen-chat vl int4, but there's no result generate on my 3060. And also there's no explicit error.
Here is the log message:
(base) F:\CLoT>python inference.py
CUDA extension not installed.
CUDA extension not installed.
C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\transformers\modeling_utils.py:4225: FutureWarning: _is_quantized_training_enabled is going to be deprecated in transformers 4.39.0. Please use model.hf_quantizer.is_trainable instead
warnings.warn(
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 5/5 [00:04<00:00, 1.02it/s]

And also the gradio has some timeout error :
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 5/5 [00:05<00:00, 1.02s/it]
Running on local URL: http://127.0.0.1:2333
Exception in thread Thread-4 (_do_normal_analytics_request):
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 69, in map_httpcore_exceptions
yield
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 233, in handle_request
resp = self._pool.handle_request(req)
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_sync\connection_pool.py", line 216, in handle_request
raise exc from None
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_sync\connection_pool.py", line 196, in handle_request
response = connection.handle_request(
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_sync\connection.py", line 99, in handle_request
raise exc
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_sync\connection.py", line 76, in handle_request
stream = self._connect(request)
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_sync\connection.py", line 122, in _connect
stream = self._network_backend.connect_tcp(**kwargs)
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_backends\sync.py", line 205, in connect_tcp
with map_exceptions(exc_map):
File "d:\miniconda3\lib\contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectTimeout: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "d:\miniconda3\lib\threading.py", line 1016, in _bootstrap_inner
self.run()
File "d:\miniconda3\lib\threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\gradio\analytics.py", line 63, in _do_normal_analytics_request
httpx.post(url, data=data, timeout=5)
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_api.py", line 319, in post
return request(
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_api.py", line 106, in request
return client.request(
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 827, in request
return self.send(request, auth=auth, follow_redirects=follow_redirects)
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 914, in send
response = self._send_handling_auth(
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 942, in _send_handling_auth
response = self._send_handling_redirects(
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 979, in _send_handling_redirects
response = self._send_single_request(request)
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1015, in _send_single_request
response = transport.handle_request(request)
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 232, in handle_request
with map_httpcore_exceptions():
File "d:\miniconda3\lib\contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 86, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectTimeout: timed out

Thank you for this interesting and amazing work! 
When do you plan to release the training / dataset preprocessing code? Thank you.

Hi, since int4 version of Qwen-vl is avaialble and more friendly for low end gpu, is it a plug and play model for clot?

A.
B.you
C.need
D.1
E.2
F.ah

请问这个结果正常吗？看起来并不是很搞笑

How to solve this problem,just download the Qwen-VL-chat checkpoints and merge it with CLoT_cn into a folder?
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like Qwen/Qwen-VL-Chat is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.

I try to run the program on my macbook pro with M2 Max CPU, it throws AssertionError: Torch not compiled with CUDA enabled.

Detail log is shown as following:

/Users/chockiezhu/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
  warnings.warn(
Loading checkpoint shards:   0%|                         | 0/10 [00:01<?, ?it/s]
Traceback (most recent call last):
  File "/Users/chockiezhu/practice/CLoT/gradio_demo.py", line 369, in <module>
    main()
  File "/Users/chockiezhu/practice/CLoT/gradio_demo.py", line 365, in main
    _launch_demo(args)
  File "/Users/chockiezhu/practice/CLoT/gradio_demo.py", line 114, in _launch_demo
    model, tokenizer = _load_model_tokenizer(args.checkpoint_path)
  File "/Users/chockiezhu/practice/CLoT/gradio_demo.py", line 71, in _load_model_tokenizer
    model = AutoPeftModelForCausalLM.from_pretrained(
  File "/Users/chockiezhu/Library/Python/3.9/lib/python/site-packages/peft/auto.py", line 104, in from_pretrained
    base_model = target_class.from_pretrained(base_model_path, **kwargs)
  File "/Users/chockiezhu/Library/Python/3.9/lib/python/site-packages/transformers/models/auto/auto_factory.py", line 558, in from_pretrained
    return model_class.from_pretrained(
  File "/Users/chockiezhu/Library/Python/3.9/lib/python/site-packages/transformers/modeling_utils.py", line 3531, in from_pretrained
    ) = cls._load_pretrained_model(
  File "/Users/chockiezhu/Library/Python/3.9/lib/python/site-packages/transformers/modeling_utils.py", line 3958, in _load_pretrained_model
    new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
  File "/Users/chockiezhu/Library/Python/3.9/lib/python/site-packages/transformers/modeling_utils.py", line 812, in _load_state_dict_into_meta_model
    set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs)
  File "/Users/chockiezhu/Library/Python/3.9/lib/python/site-packages/accelerate/utils/modeling.py", line 399, in set_module_tensor_to_device
    new_value = value.to(device)
  File "/Users/chockiezhu/Library/Python/3.9/lib/python/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")

Interesting work!
Looking forward to the release of code and dataset
Do you have any plan?

请问下目前开源的模型只使用了大喜利的数据进行训练吗？是否能分享下使用了微博推特等更多数据的最新模型呢？