llm-tools / embedjs Goto Github PK

Hey guys, great job with embedJS. IMO it's the easiest way of adding LLM into a nodejs project right now, you guys are doing a great job!

I don't know if you guys have seen it, but one of the latest things AI community is talking is about Multi AI Agents. I think this would take embedJS to another level, since there aren't many nodejs libraries that does that

https://www.youtube.com/watch?v=sal78ACtGTc here's a talk about Multi AI Agents

I've found one NodeJS library that supports it: https://github.com/metaskills/experts

1. What is it
2. Format structure
3. User case example? name value pairs? single field with datalist?

What is best practice to initialize ragbuilder/ragapplication vars and add loaders/buid into lance

1. create a new rag builder and ragapplication var from it which is public as part of server initialization
2. I can add additional loaders via an api end point, set Lance and build each time
3. Should i be setting vector db once, then just calling builder after add loader each time?

What is the best way to build up incremental data in the session to answer questions reliably across web pages and PDF's mixed in?

Problem

when i add urls it seems to work, when i re-add they kind of work answer questions but i notice it becomes much less reliable. I tried adding 2 urls' then a PDF. initially the url's worked, answering questions, then when i added a PDF, that worked answered questions but the original two web pages started getting a lot more "I don't know the answer to the question" like it become much dumber.

Is this because the way I'm adding / building in Lance below or is it because Lance perhaps isn't the best semantic search rag I should be using as more data is added?

Code

initialization

let ragApplication;
let ragApplicationBuilder;

router.get('/test', async (req, res) => {
ragApplication = await new RAGApplicationBuilder()
.setTemperature(0)
res.status(200).send("test Done");
})

adding pages

router.post('/addURL', async (req, res) => { // Changed to POST
const { url } = req.body; // Extract URL from request body
if (!url) {
return res.status(400).send('URL is required');
}
try {
ragApplicationBuilder
.addLoader(new WebLoader({ url: url }))
ragApplication = await ragApplicationBuilder
.setVectorDb(new LanceDb({ path: './db' }))
.build();
res.status(200).send("updated web page");
} catch (error) {
console.error('Error adding URL:', error);
return res.status(500).send('Internal Server Error');
}
});

   const ragApplication = await new RAGApplicationBuilder()
      .setModel(new HuggingFace({ modelName: 'timpal0l/mdeberta-v3-base-squad2'}))
      .setQueryTemplate('')
      .setVectorDb(new PineconeDb({
        projectName: 'test',
        namespace: 'dev',
        indexSpec: {
          serverless: {
            cloud: 'aws',
            region: 'us-east-1',
          },
        },
      }))
      .setCache(new LmdbCache({ path: path.resolve('./cache') }))
      .setTemperature(0.01) // closer to 1 = more creative. Closer to 0 = more deterministic
      .setSearchResultCount(100) // hwow much to get from the db
      .build();

    const query = await ragApplication.query('What is Tesla?');

Given this model and many other models on HuggingFace the output error is

Error: Error in `inputs`: value is not a valid dict
    at request 

While I can see the error is from Hugging Face, it's unclear how I would provide the correct inputs in a case like this.

Any help is greatly appreciated!

When signing up for a new Pinecone account, a value for PINECONE_ENVIRONMENT is no longer provided:
https://github.com/pinecone-io/pinecone-ts-client/blob/main/v2-migration.md

Moving to version 2.0.1 of @pinecone-database/pinecone allows you to use the new keys provided, with a few minor changes to pinecone-db.ts.

Seems like an export is missing to use Pinecone DB

ts-node src/ai
Error: Package subpath './dist/vectorDb/pinecone-db' is not defined by "exports" in /

Same with cache

Error: Package subpath './dist/cache/lmdb-cache' is not defined by "exports" 

Code

    const ragApplication = await new RAGApplicationBuilder()
      .setModel(new HuggingFace({ modelName: 'mistralai/Mixtral-8x7B-v0.1' }))
      .setVectorDb(new PineconeDb({
        projectName: 'test',
        namespace: 'dev',
        indexSpec: {
          pod: {
            podType: 'p1.x1',
            environment: 'us-east1-gcp',
          },
        },
      }))
      .setCache(new LmdbCache({ path: path.resolve('./cache') }))

      .addLoader(new SitemapLoader({ url: 'https://tesla-info.com/sitemap.xml' }))
      .setTemperature(0.01)
      .setSearchResultCount(100)
      .build();

The imports are structured like this (auto imported)

import { HuggingFace, RAGApplicationBuilder, SitemapLoader } from '@llm-tools/embedjs';

import { PineconeDb } from '@llm-tools/embedjs/dist/vectorDb/pinecone-db';
import { LmdbCache } from '@llm-tools/embedjs/dist/cache/lmdb-cache'

Partial Package.json

  "dependencies": {
    "@llm-tools/embedjs": "^0.0.67",
    "@mistralai/mistralai": "^0.1.3",
    "@pinecone-database/pinecone": "^2.2.0",

Let me know if you need more information to reproduce!

Getting an error trying to import any of the classes. Simplest example:

import { LLMApplicationBuilder } from '@llm-tools/embedjs';

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '.../embedJsDemo/node_modules/@llm-tools/embedjs/dist/core/llm-application.js' imported from ...embedJsDemo/node_modules/@llm-tools/embedjs/dist/index.js

Likely related, starting with version 0.0.44, @llm-tools/embedjs/dist is missing everything but index.js and the package only contains 4 files.
https://www.npmjs.com/package/@llmembed/embedjs/v/0.0.44?activeTab=code

0.0.32 looks closer to expectation, with 75 files:
https://www.npmjs.com/package/@llmembed/embedjs/v/0.0.32?activeTab=code

this must be a bug?
export class OpenAi3LargeEmbeddings implements BaseEmbeddings {
private model: OpenAIEmbeddings;

constructor() {
    this.model = new OpenAIEmbeddings({ modelName: **'text-embedding-3-small',** maxConcurrency: 3, maxRetries: 5 });
}

also how do you import these models via import statement?

Is there anyway we can have another method to maybe tag the RAG (id), and enhance with meta data and resubmit. I think this would be a very powerful feature, to add meta data to individual media elements where its pulled for a source but you want to supplement it. Adding richer background context, and meta to say those webpages can really improve RAG a lot, it's performance, accuracy, and intelligence.

I've just been on some webinars today and this seemed to be a theme building up.

note : This is not the same as the initial prompt setting

I try to use Huggingface model like the following, but it shows error. Probably, I set the wrong config, please help me check it. Thanks in advance.

.env file

HUGGINGFACEHUB_API_KEY={hf_key}
AZURE_OPENAI_API_KEY=
OPENAI_API_KEY=

Code :

const { RAGApplicationBuilder, HuggingFace, TextLoader } = require("@llm-tools/embedjs");

const initEmbedding = async () =>{
const ragApplication = await new RAGApplicationBuilder()
.setModel(new HuggingFace({ modelName: 'google/gemma-7b' }))
.setSearchResultCount(10)
.addLoader(new TextLoader({ text: `Tesla is an American multinational automotive and clean energy company headquartered in Austin, Texas, which designs, manufactures and sells electric vehicles, stationary battery energy storage devices from home to grid-scale, solar panels and solar shingles, and related products and services.

Tesla was incorporated in July 2003 by Martin Eberhard and Marc Tarpenning as Tesla Motors. The company's name is a tribute to inventor and electrical engineer Nikola Tesla. In February 2004 Elon Musk joined as the company's largest shareholder and in 2008 he was named CEO. In 2008, the company began production of its first car model, the Roadster sports car, followed by the Model S sedan in 2012, the Model X SUV in 2015, the Model 3 sedan in 2017, the Model Y crossover in 2020, the Tesla Semi truck in 2022 and the Cybertruck pickup truck in 2023. The Model 3 is the all-time bestselling plug-in electric car worldwide, and in June 2021 became the first electric car to sell 1 million units globally.[5] In 2023, the Model Y was the best-selling vehicle, of any kind, globally.[2]

Tesla is one of the world's most valuable companies. In October 2021, Tesla's market capitalization temporarily reached $1 trillion, the sixth company to do so in U.S. history. As of 2023, it is the world's most valuable automaker. In 2022, the company led the battery electric vehicle market, with 18% share.

Tesla has been the subject of lawsuits, government scrutiny, and journalistic criticism, stemming from allegations of whistleblower retaliation, worker rights violations, product defects, and Musk's many controversial statements.` }))
.setCache(new MemoryCache())
.setVectorDb(new LanceDb({ path: '.db' }))
.build();

// get result
console.log(await ragApplication.query('What is Tesla?'));

}

initEmbedding();

Error :

node_modules/@langchain/openai/dist/embeddings.cjs:117
throw new Error("OpenAI or Azure OpenAI API key not found");

Too many errors on installation

At first i tryed with npm without success than tryed with yarn.

info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
PS C:\Users\L*****\embedJs-main> yarn install
yarn install v1.22.19
warning ..\..\..\package.json: No license field
info No lockfile found.
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
warning @qdrant/[email protected]: The engine "pnpm" appears to be invalid.
warning @qdrant/[email protected]: The engine "pnpm" appears to be invalid.
error @typescript-eslint/[email protected]: The engine "node" is incompatible with this module. Expected version "^18.18.0 || >=20.0.0". Got "18.14.0"
error Found incompatible module.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

I know all my libraries are installed, as they imported, other libraries are loading but lanceDb won't instantiate. DB folder exists relative to the script running.

I get ReferenceError: LanceDb is not defined

import {RAGApplicationBuilder, PdfLoader, YoutubeSearchLoader, SitemapLoader, WebLoader, TextLoader } from '@llm-tools/embedjs';

async function setupRagApplication() {
try {
const ragApplication = await new RAGApplicationBuilder()
.addLoader(new YoutubeSearchLoader({ searchString: 'Tesla cars' }))
.addLoader(new SitemapLoader({ url: 'https://tesla-info.com/sitemap.xml' }))
.addLoader(new WebLoader({ url: 'https://en.wikipedia.org/wiki/Tesla,_Inc.' }))
.setVectorDb(new LanceDb ({ path: path.resolve('/db') })) // Ensure path is correct
.build();
console.log('RAG application built successfully:', ragApplication);
} catch (error) {
console.error('Failed to build RAG application:', error);
}
}

setupRagApplication();

Are there / can there be any plans to support Amazon Titan in the near future?

I had a big long Discussion post outlining all of the steps I took and all the things that went wrong while trying to see "how easy it is to get started", particularly in regards to the incomplete documentation not saying anything about node.js needing

const { RAGApplicationBuilder, WebLoader, YoutubeLoader, SitemapLoader, SIMPLE_MODELS} = require('@llm-tools/embedjs');
const path = require('path');

Which is kind of important.

But while trying to figure it all out, discovered that the effort was fruitless anyway because different modules of the main project have conflicting dependencies.

Trying to install LanceDB:

npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @langchain/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/vectordb
npm WARN   vectordb@"*" from the root project
npm WARN   1 more (@llm-tools/embedjs)
npm WARN
npm WARN Could not resolve dependency:
npm WARN peerOptional vectordb@"^0.1.4" from @langchain/[email protected]
npm WARN node_modules/@langchain/community
npm WARN   @langchain/community@"^0.0.35" from @llm-tools/[email protected]
npm WARN   node_modules/@llm-tools/embedjs
npm WARN
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/vectordb
npm WARN   peerOptional vectordb@"^0.1.4" from @langchain/[email protected]
npm WARN   node_modules/@langchain/community
npm WARN     @langchain/community@"^0.0.35" from @llm-tools/[email protected]
npm WARN     node_modules/@llm-tools/embedjs

"@langchain/community" requires vectordb version 0.1.4
"@llm-tools/embedjs" requires vectordb version 0.1.19

So I scrapped it, started again and tried hnswlib-node

npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @langchain/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/hnswlib-node
npm WARN   hnswlib-node@"*" from the root project
npm WARN
npm WARN Could not resolve dependency:
npm WARN peerOptional hnswlib-node@"^1.4.2" from @langchain/[email protected]
npm WARN node_modules/@langchain/community
npm WARN   @langchain/community@"^0.0.35" from @llm-tools/[email protected]
npm WARN   node_modules/@llm-tools/embedjs
npm WARN
npm WARN Conflicting peer dependency: [email protected]
npm WARN node_modules/hnswlib-node
npm WARN   peerOptional hnswlib-node@"^1.4.2" from @langchain/[email protected]
npm WARN   node_modules/@langchain/community
npm WARN     @langchain/community@"^0.0.35" from @llm-tools/[email protected]
npm WARN     node_modules/@llm-tools/embedjs
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: @llm-tools/[email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/hnswlib-node
npm ERR!   hnswlib-node@"*" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peerOptional hnswlib-node@"^2.1.1" from @llm-tools/[email protected]
npm ERR! node_modules/@llm-tools/embedjs
npm ERR!   @llm-tools/embedjs@"^0.0.66" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: [email protected]
npm ERR! node_modules/hnswlib-node
npm ERR!   peerOptional hnswlib-node@"^2.1.1" from @llm-tools/[email protected]
npm ERR!   node_modules/@llm-tools/embedjs
npm ERR!     @llm-tools/embedjs@"^0.0.66" from the root project

"@langchain/community" is compatible with hnswlib-node 1.4.2
"@llm-tools/embedjs" is compatible with hnswlib-node 2.1.1

Pinecone installed.. but I really don't want to have to make another account on another site.

Apparently this can optimize the cost, performance whilst still out performing ada and small gpt models. Adding this will bake in flexibility for future.

See note below

https://platform.openai.com/docs/guides/embeddings/use-cases

If I want to clear out everything and rebuild all the embeddings what is the best approach?

For example a website may change over time and you may want to clear out the website embeddings or the entire thing and rebuild it weekly, monthly etc

I'm trying to use LmdbCache with Nextjs but i got error.

My route.ts file:

import {auth} from '@/lib/auth'
import {OpenAi, RAGApplication, RAGApplicationBuilder} from "@llm-tools/embedjs";
import * as path from "path";
const { QdrantDb } = require("@llm-tools/embedjs/vectorDb/qdrant");
const { LmdbCache } = require("@llm-tools/embedjs/cache/lmdb");

let ragApplication: RAGApplication;
const ragBuilder = new RAGApplicationBuilder()
    .setModel(new OpenAi({ modelName: 'gpt-3.5-turbo' }))
    .setVectorDb(
        new QdrantDb({
          apiKey: process.env.QDRANTDB_API_KEY || '',
          url: process.env.QDRANTDB_API_URL || '',
          clusterName: process.env.QDRANTDB_CLUSTER_NAME || '',
        })
    )
    .setLoaderInit(false);

export async function POST(request: Request) {
  const session = await auth()
  const { query } = await request.json()
  
  if (!session) {
    return new Response(null, {
      status: 401
    })
  }
  
  if (!ragApplication) {
    ragApplication = await ragBuilder.build();
  }
  
  const result = await ragApplication.query(query);
  
  return Response.json({ result })
}

Error:

Import trace for requested module:
./node_modules/.pnpm/@[email protected]_@[email protected][email protected]/node_modules/@llm-tools/embedjs/dist/cache/lmdb-cache.cjs
./app/api/chat/route.ts
 ⨯ ./node_modules/.pnpm/[email protected]/node_modules/lmdb/dist/index.cjs:3109:0
Module not found: Can't resolve 'cbor-x'

https://nextjs.org/docs/messages/module-not-found

If I install the cbor-x node package, I got this error:

 ⨯ node_modules/.pnpm/[email protected]/node_modules/node-gyp-build-optional-packages/index.js (77:0) @ load.path
 ⨯ Error: No native build was found for platform=darwin arch=arm64 runtime=node abi=108 uv=1 armv=8 libc=glibc node=18.20.2 webpack=true
    attempted loading from: .next/server/vendor-chunks and package: @/-darwin-arm64

it appears that ChromaDB is not exported within the NPM Module. also, think Ollama embeddings is missing unless i'm doing something wrong?

Module '"@llm-tools/embedjs"' has no exported member 'ChromaDb'.ts(2305)

Might this be a useful feature to keep things more private as the query is passed to GPT's and other LLM's where the model is more tuned one the query moves from semantic search/rag then passed to the LLM. An optional LLM model ID would be a useful feature.

Hey!

The request is about the following issue:

Currently, the Promise returned with .query consists of the result itself and the sources.
The sources itself represents the raw source string rather than a friendly name, making it rather messy when attempting to display them in a user interface in a readable manner. Especially for the text and file-based loaders (which get provided as base64 string in my case), the whole input / data URL is returned.

To solve this issue, I'd like to request the addition of a new parameter (possibly optional) that can be used during the addition of a new loader as well as being returned with the query itself.

Thanks a lot :)

If i need to switch a vector, i can switch dynamically based on the setVector and Path settingss and changing these to different values.

The question is, loaders, for example if i wanted to use a defined loader i used with a vector db, how to retrieve or remember it or reload it back when switching from one to other. I have a listLoaders api i wrote, to return me the list, types and id's but if you reload server, they all go. So it's about persisting without have to call the reloaders again such as re-adding web pages once they are in the vector db/llm cache.

If ti can be managed, this way you can potentially have multiple rags, and even build a graph type llm or a simpler router llm at the front of it later to decide which rag to to switch to.

should there be a loadVector/LoadEmbeddings function?

Any advice is appreciated. I think your doing an awesome job, I will try raise that FAISS PR this weekend!

No matter which managed vectorDB I try to use, I get timeout error - fetch failed. I've tried pinecone, qdrant. I get this error(for pinecone) -
error PineconeConnectionError: Request failed to reach Pinecone. This can occur for reasons such as network problems that prevent the request from being completed, or a Pinecone API outage. Check your network connection, and visit https://status.pinecone.io/ to see whether any outages are ongoing.

Is there something I need to look out for while using the client for nodejs?

before it would add one to the loader list, now in v0.78 it adds 3 other duplicates.

and on the debug console we get all this
1|platform | Error adding URL: [Error: lance error: Commit conflict for version 1195: There was a concurrent commit that conflicts with this one and it cannot be automatically resolved. Please rerun the operation off the latest version of the table.
1|platform | Transaction: Transaction { read_version: 1194, uuid: "b317b32c-fa93-4c90-aec7-c26663444642", operation: Delete { updated_fragments: [], deleted_fragment_ids: [598], predicate: "uniqueLoaderId = "WebLoader_7e57a7538098320500365556c0c96ea1"" }, tag: None }
1|platform | Conflicting Transaction: Some(Transaction { read_version: 1193, uuid: "eb1a0a1d-176e-4e0d-840c-8e8c949b02c1", operation: Append { fragments: [Fragment { id: 0, files: [DataFile { path: "8cccacb1-74c0-4190-bec8-d425d2736c52.lance", fields: [0, 1, 2, 3, 4, 5], column_indices: [], file_major_version: 0, file_minor_version: 0 }], deletion_file: None, physical_rows: Some(4) }] }, tag: None }), /home/build_user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lance-0.10.16/src/io/commit.rs:107:23]
1|platform | Error adding URL: [Error: lance error: Commit conflict for version 1336: There was a concurrent commit that conflicts with this one and it cannot be automatically resolved. Please rerun the operation off the latest version of the table.
1|platform | Transaction: Transaction { read_version: 1335, uuid: "036ad30e-4c63-4768-8984-4d74cf92f546", operation: Delete { updated_fragments: [], deleted_fragment_ids: [669], predicate: "uniqueLoaderId = "WebLoader_7e57a7538098320500365556c0c96ea1"" }, tag: None }
1|platform | Conflicting Transaction: Some(Transaction { read_version: 1334, uuid: "5ce59ff0-fc6d-4824-9068-d5131f6e6f36", operation: Append { fragments: [Fragment { id: 0, files: [DataFile { path: "fb294721-dfd9-4732-848f-92efcb7db637.lance", fields: [0, 1, 2, 3, 4, 5], column_indices: [], file_major_version: 0, file_minor_version: 0 }], deletion_file: None, physical_rows: Some(4) }] }, tag: None }), /home/build_user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lance-0.10.16/src/io/commit.rs:107:23]
^[[A^[[A

Hello,

I'm trying to use HuggingFace model, and I keep getting 429 error from OpenAPI My code:

`export class EmbedService implements OnModuleDestroy, OnModuleInit {
private ragApplication: RAGApplication;

public async query(query: string) {
    return this.ragApplication.query(query);
}

async onModuleInit() {

    this.ragApplication = await new RAGApplicationBuilder()
        .setModel(new HuggingFace({ modelName: "meta-llama/Llama-2-7b" }))
        .setTemperature(0.1)
        .addLoader(new YoutubeLoader({ videoIdOrUrl: 'https://www.youtube.com/watch?v=w2KbwC-s7pY' }))
        .addLoader(new WebLoader({ url: 'https://adhityan.com/' }))
        .setVectorDb(new ChromaDb({ url: 'http://localhost:8000' }))
        .build();
}

onModuleDestroy() {}

}

The error:

`InsufficientQuotaError: 429 You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.
at Object.defaultFailedAttemptHandler [as onFailedAttempt] (file:///home/leandro/code/CDEBot/node_modules/.pnpm/@langchain[email protected]/node_modules/@langchain/core/dist/utils/async_caller.js:33:21)
at RetryOperation._fn (/home/leandro/code/CDEBot/node_modules/.pnpm/[email protected]/node_modules/p-retry/index.js:67:20)
at processTicksAndRejections (node:internal/process/task_queues:95:5)

Important: I just can reach this point if Open AI env API key variable is set. If I don˜t set it,I got the error:

Error: OpenAI or Azure OpenAI API key not found

Thanks! :)

PS: Congratulations for the project, awesome work :)

very good LLM, and has a lot of support around it

• Grok is another one
• Amazon Bedrock services (like Hugging face)

I want to be able to add excel as a load, or upload it and answer questions from a reasonable structureds spreadsheet with product info, description column and pricing.

Is that possible? can this common pattern be added?

Great libary, and do you have a paypal or funding link for a coffee fund! Happy to buy a round of coffee's

not the same as add WebLoader, but LangChain has this hook to pass in a HTML page content, and some other settings. Under certain conditions this could be very userful to customize the HTML content or pre-process it before passing it to the loader

From the facebook team, and seen as good for research and scalable/fast

there is another git project from Microsoft for LLM Prompt Query compression thatr reduces query prompt by 20% and optimizes it but also reduces the LLM costing and speeds up performance.

https://github.com/microsoft/LLMLingua

Could this be something useful to hook in as an optimization on the LLM template and query?

import express from 'express';
import fs from 'fs';
import path from 'path';
import {RAGApplicationBuilder, RAGApplication,
PdfLoader, YoutubeSearchLoader, SitemapLoader, WebLoader, TextLoader } from '@llm-tools/embedjs';
import {LanceDb} from '@llm-tools/embedjs/vectorDb/lance';
import { OpenAI } from "openai";
let key = process.env.OPENAI_API_KEY; // Replace with your OpenAI API key

const openai = new OpenAI({
apiKey: key
});

router.get('/test', async (req, res) => {
const ragApplication = await new RAGApplicationBuilder()
.addLoader(new WebLoader({ url: 'https://en.wikipedia.org/wiki/Tesla,_Inc.' }))
.setVectorDb(new LanceDb({ path: './db' }));
res.status(200).send(await ragApplication.query('Tell me about the history of Tesla?'));

})

TypeError: ragApplication.query is not a function
Failed to build RAG application: TypeError: Cannot read properties of undefined (reading 'query')

However query is failing what imports do i need to specify for this?

Notice youtube issue's a fetch warning "experimental". I'm using node v18, moving to v20 soon.

To complete the MS office (although sharepoint would be good too)

It would be great if you could add Qdrant vector database to your project.

the docs incorrectly state
await new RAGApplicationBuilder()
.QueryTemplate('My own query template')

the correct setting is below
await new RAGApplicationBuilder()
.setQueryTemplate('My own query template')

The large embedded model constructor seems to be been fixed and this is now included in my project for build 0.71

However clearing vector and lmdb caches (deleting) and restart server, switching to the large declaration for gpt embedded model resulted in problems. A number of issues is happening. The code below works with small text embedded gpt model but switch this to large and add a single URL or multiple and it fails with sizing dimension issues immediately

Code
ragApplication = await ragApplicationBuilder
.setTemperature(0.2)
.setEmbeddingModel(new OpenAi3LargeEmbeddings())
.setVectorDb(new LanceDb({ path: './db' }))
.setCache(new LmdbCache({ path: './llmcache'}))
.build();

1|platform | Error adding URL: Error: Invalid argument error: Values length 13824 is less than the length (3072) multiplied by the value size (3072) for FixedSizeList(Field { name: "item", data_type: Float32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, 3072)
1|platform | at LocalTable.add (/opt/bitnami/projects/platform/node_modules/vectordb/dist/index.js:209:14)
1|platform | at async LanceDb.insertChunks (file:///opt/bitnami/projects/platform/htdocs/node_modules/@llm-tools/embedjs/dist/vectorDb/lance-db.js:61:9)
1|platform | at async RAGApplication.batchLoadChunks (file:///opt/bitnami/projects/platform/htdocs/node_modules/@llm-tools/embedjs/dist/core/rag-application.js:130:23)

Any way to use hugging face embedding models currently?

thanks!

Be able to set the text split size for the text splitter server

What is the right procedure to remove the item from the loader (or all items)

I can remove the embedding using deleteEmbeddingsFromLoader and this works but the loader reference from a previous add is still hanging around and present in the loaders list

    **await ragApplication.deleteEmbeddingsFromLoader(uniqueLoaderId, true); /* works */**

    **ragApplicationBuilder.deleteLoader(uniqueLoaderId);  /* ? something like this to follow on? */**

I have been considering this for a while now. The more loaders, LLMs and embedding models we add, the question becomes more and more important. Currently the library already separates out the dependencies for the vector databases using clever package.json config, leveraging peer and optional dependencies.

But this does not scale well but is not a problem today. At the same time, it's getting important that we really break out the various parts even more. I don't want a lot of dependencies being added in by default. It's also problematic to address vulnerabilities. Ultimately people will only use some of the choices and they should only have those relevant modules.

Here are our options -

1. Continue with the current package.json based solution. We will have more optional dependencies and we will address this by more detailed documentation.

2. Switch to a monorepo. A core package (embedJs) and several sub-packages (like embedjs-openai, embedjs-slack-loader, etc)

Clearly option 2 is the long term direction but might be more work than necessaty today.

What do you all think?

Since I don't have an API key to try embedding with OpenAI, perhaps there is a free or open-source option available.

Hey guys, I want to use streaming for OpenAI Models. How can I do that? Can you add this feature?

Should be quick and easy to add from Langchain with wrapper with following parameter, file, delimiter, quoteChat, field name string array of strings.

loader = CSVLoader(file_path='./example_data/mlb_teams_2012.csv', csv_args={
'delimiter': ',',
'quotechar': '"',
'fieldnames': ['MLB Team', 'Payroll in millions', 'Wins']
})

data = loader.load()

use for logging/tracking/thresholds and bill costings

I believe in openai you can now do this?

{ "type": "json_object"

as part of the query, i think it would be sensible to have text and non-text options. So

.setFormat('json')

1. Add Excel file to loader
2. Query loaders
3. Add same Excel file to loader (repeat process)
4. Query loaders

In v0.77, two entires Duplicate ID's

I would expect it would at the very least to create a new different Loader ID or if its able to determine the spreadsheet is same as before (filename matched) it will not add it and return a "could not add, as named resource already exists"

Probably not the best issue, but..!

We are simply testing the Cohere example on Windows, but it throws Error: OpenAI or Azure OpenAI API key not found.

Now, this looked like Cohere would do the embeddings, etc. - so is OpenAI really needed?

Just to make sure, since there is no .env.example, this is all set through global variables, right?

In other words:

set COHERE_API_KEY=abc

There should be support for audio, video and image inputs in addition to text

          OK so we should be using?

   ragApplication.deleteLoader(uniqueLoaderId, true);???

Originally posted by @converseKarl in #59 (comment)

I am interested in support for Gemini models, preferably on VertexAI by Google Cloud Platform.
The Gemini 1.5 Pro supports up to 1M Context, which could open a bunch of possibilities for grounding and RAG. Further more it is multimodal (text, images, audio, PDFs, code, videos).

Resources for the integration:

LLM and Embedding models on VertexAI:

• https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models

Langchain LLM support for VertexAI and all Gemini models:

• https://js.langchain.com/docs/integrations/llms/google_vertex_ai

Langchain Embeddings on VertexAI (Expirimental):
Supports both textembedding-gecko and multimodalembedding@001

• https://js.langchain.com/docs/modules/data_connection/experimental/multimodal_embeddings/google_vertex_ai

I'm currently working on the integration, it looks pretty straight forward.
Will also submit a PR, if the authors are interested.

P.S. Thanks for creating this library, well structured and easy to use.

Error: Cannot find module '@llmembed/embedjs/vectorDb/pinecone-db.js'
Can't find the path to import it in my nodejs project. Tried the above path but it's not found on it for some reason. No solution in readme or issues.