bpyap / bert-wsd Goto Github PK
View Code? Open in Web Editor NEW[EMNLP-Findings 2020] Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences
Home Page: https://arxiv.org/abs/2009.11795
[EMNLP-Findings 2020] Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences
Home Page: https://arxiv.org/abs/2009.11795
demo_model.py
works well with transformers v2.3.0. However when I try a more recent version, I get the embedding error "IndexError: index out of range in self".
I localized the error to the value of tokenizer.additional_special_tokens_ids
, which is [100] in v2.3.0 and [30523] in v4.2.2. If I replace all occurrences of 30523 with 100, I am able to reproduce the results. But this is just a dirty fix. Is there any elegant solution? Am I missing anything with respect to the tokenizer?
Sir, can you explain what should i do to implement WSD using BERT for any other language?
When trying to run the demo, I've run into three issues.
Firstly, when I run demo_model.py with any pre-trained model (in this case, bert_large-batch_size=128...), I get the following warning:
Some weights of the model checkpoint at DIR were not used when initializing BertWSD: ['similarity_loss_factor', 'ranking_loss_factor', 'similarity_linear.weight', 'similarity_linear.bias']
- This IS expected if you are initializing BertWSD from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertWSD from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Special tokens have been added in the vocabulary, make sure the associated word embedding are fine-tuned or trained.
Then, when I run it with my CPU using the test sentence "He caught a [TGT] bass [TGT] yesterday.", it throws an error:
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self
Finally, if I run it with my GPU (2080 Ti), it throws a ton of errors reading " Assertion srcIndex < srcSelectDimSize
failed." It ends with this error: "RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasCreate(handle)
"
Was wondering if you had any fixes to get it working. Thanks a ton!
Edit: Running just "bert-large-uncased" worked. Is there a way to still use the pre-trained models included?
How do I use models that aren't in the specified list?
I would like to use this model:
https://huggingface.co/dccuchile/bert-base-spanish-wwm-uncased
How do I go about doing this?
Regards,
Jamie
Hi,
I'm trying to use BERT-Base, Multilingual Cased
from google-research
But it's taking very long to load. I suspect an infinite loop.
The files in the model folder have different names than the models available from BERT-WSD.
How can I make this other model compatible? I need multilingual support.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.