shamblett / alpaca

An alpaca implementation in Dart

Hi there 👋

I'm Steve Hamblett, a long time Dartisan and C/C++/C# developer in my day job, with a particular interest in IOT/embedded stuff. See my Dart Pub publisher site for a few more details. For now here are some stats for my repositories -

A few badges -

alpaca's Issues

Optimise the Dart code

The Dart code should be analysed by dev tools to optimize it's performance.

Update to Dart 3

Make Ggml a standalone package rather than a library in alpaca

It may be worth making ggml into its own package if there is enough call for this.

Dandify the Dart code

On first release the code is pretty much a straight port from C++ and this is very noticeable, the code should be refactored/restructured into a more idiomatic Dart style(without compromising performance).

Treatment of size_t type by ffigen

ffigen is translating size_t type definitions to ff.Int, this needs to be ffi.Uint64 or the ggml library will not work correctly. There some way of mapping this type change in ffigen.yaml but I've yet to find it.

Fix massive memory usage.

After asking one question the memory usage of the chat application balloons to 47GB on my system(slightly less if compiled), the Alpaca chat application memory increases by approx 5GB after one question and stays there on subsequent questions.

The memory usage of the Dart application needs to be analysed to see what is going on here.

any video tutorial?

@shamblett Man.. thanks a lot for taking the task to do this! I wonder if you can do a simple video tutorial on how to install and use this package.. thanks in advance!

Build libggml.so for Windows

The shared libggml.so library should also be built as a dll for Windows.