slai-labs / get-beam

Run GPU inference and training jobs on serverless infrastructure that scales with you.

Run GPU inference and training jobs on serverless infrastructure that scales with you

Get Started

The examples folder in this repo contains various examples of programs built with Beam.

You can run any of these examples yourself. All you need is a free account on Beam.

What can you do with Beam?

• 🛰 Develop locally on remote hardware. Beam provides a brand new type of cloud development experience. You can write code on your laptop and execute it on cloud hardware immediately, with lightning fast build times.
• 📦 Instantly containerize any Python function and run it on a GPU. Configure your runtime in Python - tell us how many GPUs you need and which libraries you want installed, and Beam will spawn a remote environment for you.
• 🚀 Deploy apps as serverless functions. Deploy your apps as serverless REST APIs, scheduled cron jobs, or task queues - all with just a single line of Python. It's great for deploying LangChain apps, Stable Diffusion APIs, or Dreambooth.
• ⏰ Run scheduled jobs. You can run any code on a schedule, and do things like train machine learning models and setup data pipelines on S3 buckets.
• 🗂️ Mount storage volumes. Read and write data to highly-performant distributed file systems.

get-beam's Issues

Model not being moved to GPU

In this replit-code, I feel the model is not being moved to GPU:
https://github.com/slai-labs/get-beam/blob/main/examples/replit-code/replit-code.py

Also the cache_dir is not being used in the instantiation of AutoModelForCausalLM

Please correct me if I am wrong

Installation Documentation - missing install of python cli beam-sdk

The install documentation has a curl command to install beam, which is also used to update beam. (https://docs.beam.cloud/getting-started/installation)

The Mac/Linux and Windows instructions include only that curl command to install, and does not include the instructions to install the python cli:

python3 -m pip install --upgrade beam-sdk

The uninstall instructions do mention the beam-sdk, so the install instructions were just missed.

To replicate:

curl https://raw.githubusercontent.com/slai-labs/get-beam/main/get-beam.sh -sSfL | sh
python> import  beam

ModuleNotFoundError: No module named 'beam'

To verify the fix:

curl https://raw.githubusercontent.com/slai-labs/get-beam/main/get-beam.sh -sSfL | sh
python3 -m pip install --upgrade beam-sdk
python
python> import  beam

Security concern

Hello 👋

I run a security community that finds and fixes vulnerabilities in OSS. A researcher (@hainguyen0207) has found a potential issue, which I would be eager to share with you.

Could you add a SECURITY.md file with an e-mail address for me to send further details to? GitHub recommends a security policy to ensure issues are responsibly disclosed, and it would help direct researchers in the future.

Looking forward to hearing from you 👍

(cc @huntr-helper)

T5 LLM

I am having tons of problems ---
last are

beam-sdk was not found using "python3 -m pip freeze | grep beam-sdk"... Proceeding to install beam-sdk into the environment

! App file 'app.py' not found.

Can someone help please?

Thanks I am justtrying to follow your instructions - https://github.com/slai-labs/get-beam/tree/main/examples/t5- but it seems much harder than it seemed --- :D

Beam not working CentOS

Running beam to deploy T5+ :
(venv) [XXX@gpu-server codeT5+]$ beam deploy app.py
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'importlib.metadata'
! beam-sdk was not found using "python3 -m pip freeze | grep beam-sdk"... Proceeding to install beam-sdk into the environment
Requirement already satisfied: beam-sdk in ./venv/lib/python3.6/site-packages (0.1.0)
Traceback (most recent call last):
File "/usr/lib64/python3.6/runpy.py", line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/lib64/python3.6/runpy.py", line 109, in _get_module_details
import(pkg_name)
File "/home/XXX/llm/codeT5+/venv/lib64/python3.6/site-packages/beam/init.py", line 24, in
_config = Config.resolve()
File "/home/XXX/llm/codeT5+/venv/lib64/python3.6/site-packages/beam/config.py", line 164, in resolve
raise RuntimeError('Unable to locate config file')
RuntimeError: Unable to locate config file
! Invalid app definition.
Usage:
beam deploy [filename.py:function_name] [flags]

Flags:
-h, --help help for deploy
-p, --profile string Authenticate with a specific beam profile. (default "default")

(venv) [XXX@gpu-server codeT5+]$ uname -a
Linux gpu-server 3.10.0-1127.10.1.el7.x86_64 #1 SMP Wed Jun 3 14:28:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
(venv) [XXX@gpu-server codeT5+]$ python3 --version
Python 3.6.8
(venv) [XXX@gpu-server codeT5+]$ cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)

Fine-tuning example doesn't work for custom datasets

The fine-tuning script provided does not work with custom datasets. It appears there is a bug related to PEFT. The InvalidHeaderDeserialization message has been reported by other users. Strangely, the recommended changes of not calling torch.compile didn’t work for me. However, after looking at this comment I removed the following code and training completed successfully.