A flake following the main branch of ollama. It's purpose is to build the most recent version supporting new models until the version in nixpkgs is updated.

• Install
  • Version
  • Backend
• Update
• Remove
• License

You need to have nix flakes enabled; if you don't, see the nixos wiki for information on flakes and enabling them.

Install ollama to user profile:

nix profile install github:abysssol/ollama-flake

Create a temporary shell with ollama:

nix shell github:abysssol/ollama-flake

Use as an input in another flake:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

    ollama.url = "github:abysssol/ollama-flake";
    # do not do this -> `ollama.inputs.nixpkgs.follows = "nixpkgs";`
    # because a specific version of nixpkgs is required
    # overriding nixpkgs will most likely break the flake
  };

  outputs = { nixpkgs, ollama, ... }:
    let
      system = abort "system needs to be set";
      # to access the rocm package of the ollama flake:
      ollama-rocm = ollama.packages.${system}.rocm;
      #ollama-rocm = inputs'.ollama.packages.rocm; # with flake-parts

      pkgs = nixpkgs.legacyPackages.${system};
      # you can override package inputs like with nixpkgs
      ollama-cuda = ollama.packages.${system}.cuda.override { cudaGcc = pkgs.gcc11; };
    in
    {
      # output attributes go here
    };
};

You can specify a version by appending /version after the main url, where version is any branch or tag. The version branches will only be updated in a backward compatible way, no breaking changes (see semantic versioning).

Append /1 to follow branch 1 which tracks version 1.y.z of the repo:

nix profile install github:abysssol/ollama-flake/1

Use an unchanging tagged version:

nix profile install github:abysssol/ollama-flake/1.3.1

Alternate packages can be specified as usual. From version 1, install the cpu package, which is built to only run on CPU:

nix profile install github:abysssol/ollama-flake/1#cpu

Other versions may be available:

nix profile install github:abysssol/ollama-flake/0
nix profile install github:abysssol/ollama-flake/1.1.0

Multiple packages are available for the different backend implementations supported by ollama.

The available options:

• cpu: fallback CPU implementation

  nix profile install github:abysssol/ollama-flake#cpu

• rocm: supported by modern AMD GPUs

  nix profile install github:abysssol/ollama-flake#rocm

• cuda: supported by modern NVIDIA GPUs; uses unfree licensed libraries

  nix profile install github:abysssol/ollama-flake#cuda

• gpu: build for both rocm and cuda, then dynamically load the relevant library at runtime

  nix profile install github:abysssol/ollama-flake#gpu

The default is gpu:

# both of these are the default package, and are equivalent
nix profile install github:abysssol/ollama-flake
nix profile install github:abysssol/ollama-flake#default
# both of the above are equivalent to the one below
nix profile install github:abysssol/ollama-flake#gpu

Find the index of the package to update:

nix profile list

Update the package at index:

nix profile upgrade $index

If nix is hesitant to download updates, force nix to download new files with --refresh:

nix profile upgrade --refresh $index

Find the index of the package to remove:

nix profile list

Remove the package at index:

nix profile remove $index

This software is dedicated to the public domain under the Creative Commons Zero. Read the CC0 in the LICENSE file or online.

Any contribution submitted for inclusion in the project is subject to the CC0; that is, it is released into the public domain and all copyright to it is relinquished.