kagisearch / pyllms Goto Github PK

Hey,
I have noticed that there is a heavy dependency accelerate included in the requirements.
accelerate has a dependency on torch, which results in a pretty big total size (I believe torch is about 600mb). I traced down the dependency to a PR that was adding support for local LLM runs.

Have you considered adding extras_require to setup.py so these local run dependencies are only installed if the user wants to run the LLM locally?

I am happy to open up a PR with a suggestion.

HuggingFace max_tokens parameter name in complete function is "max _tokens" instead of "max_length"

Please add the feature: custom the LLM api_base_url to the llms.init

Hi all,

I´m getting the following exception when initializing the aleph alpha model:

Found luminous-extended in AlephAlphaProvider
127.0.0.1 - - [11/Sep/2023 21:20:51] "POST /init HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/home/cw/LLM_BACKEND/LLMB_backend.py", line 164, in init
    pyllmsModel=llms.init(alephalpha_api_key=aleph_alpha_key, model=model)
  File "/home/cw/.local/lib/python3.10/site-packages/llms/__init__.py", line 8, in init
    return LLMS(*args, **kwargs)
  File "/home/cw/.local/lib/python3.10/site-packages/llms/llms.py", line 70, in __init__
    self._providers.append(provider.provider(api_key=provider.api_key, model=single_model))
  File "/home/cw/.local/lib/python3.10/site-packages/llms/providers/aleph.py", line 28, in __init__
    self.async_client = AsyncClient(api_key)
  File "/home/cw/.local/lib/python3.10/site-packages/aleph_alpha_client/aleph_alpha_client.py", line 618, in __init__
    self.session = RetryClient(
  File "/home/cw/.local/lib/python3.10/site-packages/aiohttp_retry/client.py", line 193, in __init__
    client = ClientSession(*args, **kwargs)
  File "/home/cw/.local/lib/python3.10/site-packages/aiohttp/client.py", line 228, in __init__
    loop = get_running_loop(loop)
  File "/home/cw/.local/lib/python3.10/site-packages/aiohttp/helpers.py", line 288, in get_running_loop
    loop = asyncio.get_event_loop()
  File "/usr/lib/python3.10/asyncio/events.py", line 656, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'Thread-20 (process_request_thread)'.

is that issue known?
I don´t face this problem mit openai models, like gpt-3.5-turbo
Thanks!

Hi,

I am wandering if the following feature is considered to be added in this project, and/or if potential PR in that direction might be welcomed ?

• Adding the support for embedding creation with the various models when they support it (and perhaps evaluation too)

I am encountringi this error ModuleNotFoundError: No module named 'anthropic.api'. I just installed it and tried checking out the model list

import llms

model = llms.init()
model.list()

In Google you are calculating the tokens by the length of the prompt and completion but it should be done with google count_message_tokens function/endpoint

I've tried bechmark with default problems and a custom problem. In both cases its giving an error - KeyError: 0
From the logs, it is generating response for all the problems using all the LLMs models in the list, and then failing.