GithubHelp home page GithubHelp logo

jahbini / ollama-node Goto Github PK

View Code? Open in Web Editor NEW

This project forked from technovangelist/ollama-node

0.0 1.0 0.0 127 KB

includes a fix for multiple chunk replies from Ollama

JavaScript 6.24% TypeScript 18.62% CSS 3.76% HTML 71.37%

ollama-node's Introduction

Simple JS library to work with Ollama

The simplest JavaScript library for the easiest way to run LLMs.

This is not the official library (we don't have one), but I am one of the maintainers of Ollama.

Get started

import { Ollama } from 'ollama-node';

const ollama = new Ollama();
await ollama.setModel("llama2");

// callback to print each word 
const print = (word: string) => {
  process.stdout.write(word);
}
await ollama.streamingGenerate("why is the sky blue", print);

Create the Ollama Object

const ollama = new Ollama();

Creates the Ollama object. All methods are called from this object.

Optional Parameter

Hostname - defaults to 127.0.0.1

Set Model

await ollama.setModel("llama2";)

Sets the model to use for Generation. Unless you override anything, it will use the template, parameters, and system prompt from the Modelfile.

Working with the Model

Set System Prompt

ollama.setSystemPrompt("You are an AI assistant.");

Sets the system prompt to use with this model. Overrides anything set in the Modelfile.

Set Template

ollama.setTemplate("this is a template")

Add a Parameter

ollama.addParameter("stop", "User:")

Delete a Parameter

ollama.deleteParameter("stop", "User:")

Delete a Parameter by Name

ollama.deleteParameterByName("stop");

Deletes all parameters with that name.

Delete All Parameters

ollama.deleteAllParameters();

Show All Parameters

const params = ollama.showParameters();

Show System Prompt

const sprompt = await ollama.showSystemPrompt()

Useful if you want to update the system prompt based on the existing one.

Show Template

const template = ollama.showTemplate();

Show Model

const model = ollama.showModel();

Shows the current model name

Show Model Info

const info = await ollama.showModelInfo();

Returns parameters, template, system prompt for the current model.

List All the Models Already Pulled

const models = await ollama.listModels();

Lists all local models already downloaded.

Generate (Ask a question and get an answer back)

const output = await ollama.generate("Why is the sky blue?");

This will run the generate command and return the output all at once. The output is an object with the output and the stats.

If you want the streaming version, see below.

Streaming Generate

This is a streaming version of generate, but you don't need to know anything about streaming. Just write a callback function that does what you want to happen.

const printword = (word: string) => {
  process.stdout.write(word);
}

const printline = (line: string)

await ollama.streamingGenerate("why is the sky blue", printword, null, printline)

There are four potential callbacks, all of which are optional, though their positions matter. Use null if you want a later one and not an earlier one.

The Callbacks are:

  • responseOutput: outputs just the token in the response.
  • contextOutput: outputs the context at the end.
  • fullResponseOutput: outputs the full response object.
  • statsOutput: outputs the stats object at the {% endif %}

Other functions I need to document

  • create
  • streamingCreate
  • streamingPull
  • streamingPush
  • copy

This is not in a finished state. It is absolutely a work in progress. I started putting this together and then later saw someone put out another library on npm. bummer. but cool that it's exciting for other folks.

I just want an easy way to consume this in some examples.

I'll flesh it out soon.

ollama-node's People

Contributors

technovangelist avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.