GitHub Action Compiling Dependencies about llamasharp

[AsakusaRinne]

It's possible to launch the github action on a server with gpu to compile cuda. However I only have my personal computer.😶‍🌫️ I could help to compile cuda version and test manually until there's a solution

[martindevans]

I've managed to work out the Linux cuda build, turns out it was basically the same as the Windows one. Fortunately we don't need a GPU for compiling CUDA, although it might be a good idea to run some CI on a GPU server in the future.

Someone gave me some tips on how to build for metal, so I will investigate adding that to the GitHub action. But that probably won't be until next weekend.

[SignalRT]

In my Fork I have a different workflow and a Self-Hosted action to pass the MacOS test in my computer.

[martindevans]

WIP: https://github.com/martindevans/LLamaSharp/tree/cron_job

[AsakusaRinne]

I'm late to see this good idea. Is there any help needed for the binary release? BTW, I remember that github action only has intel MAC instead of M1/M2 MAC. Is there already a fix for it?

[martindevans]

I'm late to see this good idea. Is there any help needed for the binary release?

I've done most of the work for the compiling of binaries in the new GitHub action (currently only manually triggered). This pulls in the master branch from llama.cpp and builds for:

• Windows
• Windows+CUBLAS
• Linux
• MacOS (ARM64)
• MacOS (x86_64)

I have not yet done a metal build for MacOS. I have no idea how that works, so any help there would be really good.

BTW, I remember that github action only has intel MAC instead of M1/M2 MAC. Is there already a fix for it?

Unfortunately not, Mac runners are still all x86_64. I've set up the new action to cross compile, so it can create the ARM64 binaries, it just can't run them. That does mean we still have no way to test ARM64 though :(

[martindevans]

I just noticed the GitHub action doesn't include cuda .so files either, so that needs doing as well.

[SignalRT]

@martindevans , I post that on the issue related to the flags used to compile on MacOS with and without Metal #38 (comment)

[SignalRT]

@martindevans Once that we are able to generate all the binaries from Actions. Do we agree that this binaries should be committed to master/LLama/runtime in a daily frequency (that's an example)?. In that way we are sure that we are aligned with llama.cpp and that the binaries are generated without modifications.

[martindevans]

Yeah that's basically the idea. I was thinking that rather than committing directly to master it would open up a PR with the new binaries occasionally (e.g. weekly). The CI would then run on that PR and tell us if anything has broken (since llama.cpp has such an unstable interface I would expect some human intervention required most times).

[martindevans]

I just noticed that one of the recent updates (https://github.com/ggerganov/llama.cpp/releases/tag/b1250) has enabled building the shared libs for all Windows builds. Maybe the GitHub action for building binaries should be updated to just download those directly?

That would save a lot of time in the build process (compiling Windows+CUDA takes about 20 minutes).