Light

namtranase / gemma-cpp-python Goto Github PK

View Code? Open in Web Editor NEW

43.0 43.0 4.0 773 KB

A Python wrapper for gemma.cpp

License: MIT License

CMake 5.84% Python 46.15% C++ 44.06% Shell 3.96%

gemma-cpp-python's Introduction

Hi there, I'm Nam 👋

About Me

I'm an enthusiastic AI learner

My Projects

Open Source Contributions

gemma-cpp-python: A Python wrapper for gemma.cpp repo from google
temi: A terminal assistant for developers. Find out more, or introduction website
llama.cpp: Contributing to the open-source community. Check my contributions
👯 I’m looking to collaborate on other open-source projects!

Research Projects

AirSim-SAR at Sea with UAV: Research on using UAVs for search and rescue at sea. Explore the project
AirSim UAV Indoor Obstacle Avoidance: Developing UAV technology for indoor navigation. Learn more

gemma-cpp-python's People

Contributors

Stargazers

Watchers

Forkers

phamson02 pagakarthik mandojos3

gemma-cpp-python's Issues

Failed building wheel for pygemma

Hey there, I get an error when running pip install pygemma

This seems to be similar to an issue raised 2 weeks ago

I'm running on windows 11 using Python 3.12.2

Collecting pygemma
  Using cached pygemma-0.1.2.tar.gz (4.0 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: pygemma
  Building wheel for pygemma (pyproject.toml): started
  Building wheel for pygemma (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error

  Building wheel for pygemma (pyproject.toml) did not run successfully.
  exit code: 1

  [63 lines of output]
  running bdist_wheel
  running build
  running build_ext
  CMake Warning:
    Ignoring extra path from command line:

     "C:\Users\user\AppData\Local\Temp\pip-install-wkyx2b9t\pygemma_0078c98a806c4d5b81a7ca4d10006f36"


  CMake Error: The source directory "C:/Users/user/AppData/Local/Temp/pip-install-wkyx2b9t/pygemma_0078c98a806c4d5b81a7ca4d10006f36" does not appear to contain CMakeLists.txt.
  Specify --help for usage, or press the help button on the CMake GUI.
  Traceback (most recent call last):
    File "C:\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
    File "C:\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\build_meta.py", line 410, in build_wheel
      return self._build_with_temp_dir(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\build_meta.py", line 395, in _build_with_temp_dir
      self.run_setup()
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 56, in <module>
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\__init__.py", line 103, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
      dist.run_commands()
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\dist.py", line 963, in run_command
      super().run_command(command)
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
      cmd_obj.run()
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\wheel\bdist_wheel.py", line 368, in run
      self.run_command("build")
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\dist.py", line 963, in run_command
      super().run_command(command)
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
      cmd_obj.run()
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\_distutils\command\build.py", line 131, in run
      self.run_command(cmd_name)
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\dist.py", line 963, in run_command
      super().run_command(command)
    File "C:\Users\user\AppData\Local\Temp\pip-build-env-jq82rw7z\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
      cmd_obj.run()
    File "<string>", line 25, in run
    File "<string>", line 47, in build_extension
    File "C:\Python312\Lib\subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', 'C:\\Users\\user\\AppData\\Local\\Temp\\pip-install-wkyx2b9t\\pygemma_0078c98a806c4d5b81a7ca4d10006f36', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=C:\\Users\\7\\AppData\\Local\\Temp\\pip-install-wkyx2b9t\\pygemma_0078c98a806c4d5b81a7ca4d10006f36\\build\\lib.win-amd64-cpython-312', '-DPYTHON_EXECUTABLE=C:\\Python312\\python.exe']' returned non-zero exit status 1.
  [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pygemma
Failed to build pygemma
ERROR: Could not build wheels for pygemma, which is required to install pyproject.toml-based projects

TODO features

Add new Griffin model to gemma-cpp-python

Detail: https://arxiv.org/pdf/2402.19427.pdf
PR from gemma.cpp: google/gemma.cpp#136

Add option to pass input and receive output

Hi again.
For my project, I would like to send prompts and receive answers from within Python.
Would it be possible to add that to the roadmap?
Thank you in advance.

Slow infererence speed with the binding

With the same prompt, gemma.cpp's interactive mode can complete the answer in ~2s while gemma-cpp-python takes >1 min. This might be due to the GIL as shown in the CPU's cores usage in two screenshots below.

(1) gemma-cpp-binding

(2) gemma.cpp

Might related to this issue:
[1] How to use pybind11 in multithreaded application
[2] pybind11's doc

Remove pre-commit run-time dependency

The pre-commit dependency improves the consistency of the repo. But it introduces non-functional dependencies into pygemma. Hence, I suggest removing precommit dependency.

Pygemma not getting build on mac when I do it in virtual environment.

Collecting pygemma==0.1.3
Using cached pygemma-0.1.3.tar.gz (4.0 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: pygemma
Building wheel for pygemma (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for pygemma (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [63 lines of output]
running bdist_wheel
running build
running build_ext
CMake Warning:
Ignoring extra path from command line:

     "/private/var/folders/_f/5wjn/T/pip-install-gi3rew2y/pygemma_682956"
  
  
  CMake Error: T"/opt/homebrew/Cellar/[email protected]/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '/private/var/folders/_f/5w00gn/T/pip-install-gi3rew2y/pygemma_68a6', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/private/var/folders/_f/5wjp0gn/T/pip-install-gi3rew2y/pygemma_6829ca6/build/lib.macosx-14.0-arm64-cpython-312', '-DPYTHON_EXECUTABLE=/Users/name/Documents/C++/venv/bin/python3.12']' returned non-zero exit status 1.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pygemma
Failed to build pygemma
ERROR: Could not build wheels for pygemma, which is required to install pyproject.toml-based projects

pip error

Hi, thanks for building this!
Could you maybe have a look at this error upon pip install pygemma:

Collecting pygemma
  Downloading pygemma-0.1.0.tar.gz (3.6 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: pygemma
  Building wheel for pygemma (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for pygemma (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [63 lines of output]
      running bdist_wheel
      running build
      running build_ext
      CMake Warning:
        Ignoring extra path from command line:
      
         "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-install-5p_blrjo/pygemma_e39722359cdf49649c38ba1285a1a47d"
      
      
      CMake Error: The source directory "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-install-5p_blrjo/pygemma_e39722359cdf49649c38ba1285a1a47d" does not appear to contain CMakeLists.txt.
      Specify --help for usage, or press the help button on the CMake GUI.
      Traceback (most recent call last):
        File "/Users/k/opt/miniconda3/envs/paper2audio/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/k/opt/miniconda3/envs/paper2audio/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/k/opt/miniconda3/envs/paper2audio/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 410, in build_wheel
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir
          self.run_setup()
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 56, in <module>
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 963, in run_command
          super().run_command(command)
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 963, in run_command
          super().run_command(command)
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 963, in run_command
          super().run_command(command)
        File "/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-build-env-yzu0w6m1/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 25, in run
        File "<string>", line 47, in build_extension
        File "/Users/k/opt/miniconda3/envs/paper2audio/lib/python3.11/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', '/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-install-5p_blrjo/pygemma_e39722359cdf49649c38ba1285a1a47d', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/private/var/folders/jr/td9rb8x92llg9mmgn92sw8ch0000gn/T/pip-install-5p_blrjo/pygemma_e39722359cdf49649c38ba1285a1a47d/build/lib.macosx-11.0-arm64-cpython-311', '-DPYTHON_EXECUTABLE=/Users/k/opt/miniconda3/envs/paper2audio/bin/python']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pygemma

System: macOS 14.3.1 on Macbook Air M2
Python 3.11

Why I do this repo?

My purpose is to learn the best thing from the Google team.
In the long run, I predict that gemma.cpp will add on a lot of cool features. And this is my chance to adapt and add it to this repo!

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

Jobs

Jooble