when multi-thread, "runtime error: already borrowed"occurred about keybert

[MaartenGr]

Could you share the code you have been using? This allows me to create a reproducible example and hopefully fix the issue you are having.

[heartedness81]

Excuse me, I tried two ways to use keybert: Fast and Flask. (1) When I use keybert by Fast(multi-thread on a computer), The error "runtime error: already borrowed" did not occurrred. But error "runtime error Default CPUAllocator: not enough memory: you tried to allocate 196608 byte" (my computer has 256M memory, two thread run on it). (2) When I use keybert by Flask (multi-thread on a computer), The error "runtime error: already borrowed" occurred. But the error about memory allocator did not occurred. The following is my code. Thank you very much.
(1) Fast
from fastapi import Depends, FastAPI
from pydantic import BaseModel

app = FastAPI()

kw_model=KeyBERT()

class KeywordRequest(BaseModel):
text: str
num: Optional[int] = 10
ngram_lo: Optional[int] = 1
ngram_hi: Optional[int] = 2
diversity: Optional[float] = 0.6

@app.post('/keywords')
def index(
keyword_request: KeywordRequest,
):
keywords = kw_model.extract_keywords(keyword_request.text,keyphrase_ngram_range=(1,2), stop_words='english',top_n=keyword_request.num)
result={'isSuc':'true', 'errMsg':'null'}
stringindex=[]
for x in keywords:
detail_res_dic = {}
detail_res_dic['phrase']=x[0]
detail_res_dic['weight']=x[1]
stringindex.append(detail_res_dic)
result['res']=stringindex
return json.dumps(result,ensure_ascii=False)
(2)Flask
import flask
import json
from keybert import KeyBERT

server = flask.Flask(name)

strs='123456'
print("加载。。。")
kw_model=KeyBERT()
@server.route('/index',methods=['get','post'])
def index():
text=flask.request.values.get('text')
num_str= flask.request.values.get('num')
num=int(num_str)

keywords = kw_model.extract_keywords(text,keyphrase_ngram_range=(1,2), stop_words='english',top_n=num)
result={'isSuc':'true', 'errMsg':'null'}
stringindex=[]
for x in keywords:
	detail_res_dic = {}
	detail_res_dic['phrase']=x[0]
	detail_res_dic['weight']=x[1]
	stringindex.append(detail_res_dic)
result['res']=stringindex
return json.dumps(result,ensure_ascii=False)

server.run(port=7777,debug=True,host='0.0.0.0')

[MaartenGr]

How did you exactly call both servers? In other words, could you share exactly how you started the servers and how you did the POST? For example, did you start the FastAPI server with uvicorn main:app --reload? And how did you do your POST method, was it through Postman or something else?

However, these models are not meant to be multi-threaded which can explain the issues you are having. Extracting keywords that are generated through embedding models is difficult to multi-thread/parallelize which can explain your issues.

[heartedness81]

I do my POST method just on my computer. I use the localhost as server. I start the FastAPI server with
uvicorn --hoset 0.0.0.0 --port 5000 main:app
The two servers is the same computer. I first use it by FAST, When I got the error, I stop and try Flask to test.

[MaartenGr]

I just tried it out in a fresh environment and had no issues with the following code:

from fastapi import FastAPI
from keybert import KeyBERT

app = FastAPI()
kw_model=KeyBERT()

@app.get('/keywords')
def index(text: str):
	keywords = kw_model.extract_keywords(text, keyphrase_ngram_range=(1,2), stop_words='english')
	return keywords	

Using that code in main.py I ran the following both in the browser and in Postman:
http://127.0.0.1:8000/keywords/?text="This is a piece of text that has some interesting values."

Try using the code above and see if that works. However, it might be interesting to look into using an embedder that uses less memory since you have very little memory that you can use. These models typically require a bit more.

[MaartenGr]

I will close this for now seeing as there hasn't been an update. However, feel free to re-open it if you are still experiencing this issue!