Getting dependencies issues while installing on CPU about llm-on-ray

[xwu99]

could you add -f https://developer.intel.com/ipex-whl-stable-cpu to pip install?

[nkanike07]

could you add -f https://developer.intel.com/ipex-whl-stable-cpu to pip install?

I followed the instructions and executed below command. I see its already added.
pip install .[cpu] -f https://developer.intel.com/ipex-whl-stable-cpu -f https://download.pytorch.org/whl/torch_stable.html

[xwu99]

We did not verify or support the package for Windows. Could you try it in Linux?

[jiafuzha]

@nkanike07 could you please check your pip version by 'pip -V'? Please let me know what you get.

You should get similar request as below.

pip 20.2.4 from /usr/lib/python3.9/site-packages/pip (python 3.9)

[nkanike07]

@nkanike07 could you please check your pip version by 'pip -V'? Please let me know what you get.

You should get similar request as below.

pip 20.2.4 from /usr/lib/python3.9/site-packages/pip (python 3.9)

[nkanike07]

@nkanike07 could you please check your pip version by 'pip -V'? Please let me know what you get.

You should get similar request as below.

pip 20.2.4 from /usr/lib/python3.9/site-packages/pip (python 3.9)

I downgraded my pip to 20.2.4 and tried to install dependencies but faced same issue again. FYR, screenshot below

[jiafuzha]

@nkanike07 could you please check your pip version by 'pip -V'? Please let me know what you get.
You should get similar request as below.
pip 20.2.4 from /usr/lib/python3.9/site-packages/pip (python 3.9)

I tried to upgrade my pip to your version, but still cannot reproduce your issue. Is it possible for you to use install conda and create an empty conda environment to install llm-on-ray? Because there might be an existing package in your env has conflict with our llm-on-ray packages.

[nkanike07]

@nkanike07 could you please check your pip version by 'pip -V'? Please let me know what you get.
You should get similar request as below.
pip 20.2.4 from /usr/lib/python3.9/site-packages/pip (python 3.9)

I tried to upgrade my pip to your version, but still cannot reproduce your issue. Is it possible for you to use install conda and create an empty conda environment to install llm-on-ray? Because there might be an existing package in your env has conflict with our llm-on-ray packages.

sure, will give a try and update same here

[jiafuzha]

@nkanike07 I just reproduced the issue in Windows. The issue is caused by ipex (intel extension for pytorch) doesn't support windows. So, please switch to linux system. thanks for reporting the issue.

[nkanike07]

I tried downloading a ubuntu docker image and could successfully run all dependencies in that.

I'm currently facing one issue while running the inference.

When ran below inference command
$python inference/serve.py --config_file inference/models/gpt2.yaml --simple

Getting below missing module error

ModuleNotFoundError: No module named 'inference.api_openai_backend'

Could someone help me here?

[jiafuzha]

@nkanike07 We are working on it. Will update you soon.

[jiafuzha]

@nkanike07 Unfortunately, we cannot reproduce your issue. Let's have a talk in Teams.

[jiafuzha]

@nkanike07 I just fixed the issue and merged to the main branch. Please check out the latest code and try again.
The root cause is pip and setuptools perform different between bare-metal and container. I adjusted the way setuptools finds project files. As verified, it worked for both bare-metal and container.
thanks for your reporting.