v3ucn / causallm14b_llama_webui_adult_version

Causallm14b大模型量化版本,基于DPO算法改进,无内容审查,无**钢印,百无禁忌Webui

Causallm14b大模型量化版本，基于DPO算法优化

无内容审查，无**钢印

---

Quick Install with Conda

conda create -n venv python=3.10
conda activate venv

Install Manually

Make sure you have python3 installed

sudo apt-get install python3.10

Pip Packages

pip install -r requirements.txt

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 

CUDA Toolkit:

Download and install CUDA Toolkit 12.2 from NVIDIA’s official website.

https://developer.nvidia.com/cuda-12-2-0-download-archive?target_os=Windows

Verify the installation with nvcc --version and nvidia-smi.

GPU Version Install

pip uninstall -y llama-cpp-python
set CMAKE_ARGS=-DLLAMA_CUBLAS=on
set FORCE_CMAKE=1
pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir

download model

https://huggingface.co/tastypear/CausalLM-14B-DPO-alpha-GGUF

put it in ./mymodel

change model

edit config.py and change model_path

how to use

python3 app.py

In the use of the model, attention should be paid to conforming to the local laws and regulations.

Of course, in use, gentlemen can give full play to their imagination, but to pay attention to the issue of legal compliance, they must conform to the core socialist values and carry forward the positive energy.

It is emphasized here that prosperity, democracy, civilization, harmony, freedom, equality, justice, the rule of law, patriotism, dedication, integrity and friendliness are the basic contents of socialist core values.

causallm14b_llama_webui_adult_version's Issues

这个可以用于35b的model吗

还是说只有14b可以用啊

生成报错 ValueError: Need to enable queue to use generators.

使用 conda 部署，点击生成后报错

Traceback (most recent call last):
  File "C:\software\miniconda3\envs\causallm14b\lib\site-packages\gradio\routes.py", line 534, in predict
    output = await route_utils.call_process_api(
  File "C:\software\miniconda3\envs\causallm14b\lib\site-packages\gradio\route_utils.py", line 226, in call_process_api
    output = await app.get_blocks().process_api(
  File "C:\software\miniconda3\envs\causallm14b\lib\site-packages\gradio\blocks.py", line 1550, in process_api
    result = await self.call_function(
  File "C:\software\miniconda3\envs\causallm14b\lib\site-packages\gradio\blocks.py", line 1193, in call_function
    raise ValueError("Need to enable queue to use generators.")
ValueError: Need to enable queue to use generators.

求大佬解决一下。

如何指定端口

我没有一张好显卡，所以我租了几个小时，可是我不知道怎么把启动的host改成0.0.0.0，和端口8080
:(

启动报错

启动时显示Traceback (most recent call last): File "app.py", line 11, in <module> llm = Llama( TypeError: __init__() got an unexpected keyword argument 'chat_format'
有大佬知道怎么回事吗