zejunwang1 / bert4vec

一个基于预训练的句向量生成工具

• 👋 Hi, I’m @zejunwang1
• 👀 I’m interested in Natural Language Processing and Machine Learning.
• 🌱 I’m currently learning Pre-trained Language Model / Chinese Text Correction.
• 📫 How to reach me: [email protected]
• 💞️ WeChat official account: CNLP
• 😄 WeChat: autonlp

bert4vec's Issues

使用自己训练过的模型是否效果更好

1目前是直接加载预训练模型，不是自己训练过的,在预训练模型上面训练之后再生成句向量是否更好
2怎么训练roformer模型，它的数据格式是怎样的

同一个模型输出不同

为什么使用 mode='roformer-sim-base' 和 model_name_or_path='WangZeJun/roformer-sim-base-chinese' 和本地下载的 model_name_or_path='./roformer-sim-base-chinese' 输出的结果均不同

执行model.encode出错

代码：

coding: utf-8

import traceback
import faulthandler
faulthandler.enable()

from tqdm import *

import pandas as pd
from bert4vec import Bert4Vec

model = Bert4Vec(mode='simbert-base', model_name_or_path="/Users/ruiliu/Downloads/simbert-base-chinese")

model = Bert4Vec(mode='roformer-sim-base', model_name_or_path="/Users/ruiliu/Downloads/roformer-sim-base-chinese")

def load_excel(excel_path, sheet_name=0):
data_list = []
data = pd.read_csv(excel_path, keep_default_na=False)
for i in data.index.values:
data_list.append(data.loc[i].to_dict())
return data_list

if name == "main":
try:
data_list = load_excel("data/交互平台问答数据-ALL-CODE.csv")
o_file = open("data/simbert_no_normal_vector.txt", "w", encoding="utf-8")
for data in data_list:
sentences = [data["标准问题*"]]
vectors = model.encode(sentences, batch_size=64, convert_to_numpy=True, normalize_to_unit=False)
# print(vectors)
# vector = [str(float(i)) for i in vectors[0]]
# print(vector)
# o_file.writelines(sentences[0] + "--" + "--".join(vector) + "\n")
o_file.close()
except Exception as e:
print(traceback.format_exc())

如何输出更小的维度

您好！请问怎么让模型输出更小的维度呢(比如64维)？768维在实际中占用空间太大，而且计算也比较耗时

关于模型转换的问题

您好，我看到原始的 SimBERT 和 RoFormer-Sim 模型是基于TensorFlow的，作者使用的什么方法做的？