Comments (7)
Hi, this seems because the program run out of the GPU memory.
If possible, could you try a GPU with a larger memory? (Our experiments are based on a NVIDIA 2080Ti GPU, which has 11GB memory.) Also, try to train the model without adding cross-sentence context first, i.e., setting --context_window 0
.
from pure.
Of course, I'll try tomorrow. Also, is my setting for task correct: --task ace05? Also, will your code accepts my relations:
"relations": [[[0, 0, 10, 11, "Founder"], [0, 0, 13, 14, "Founder"]], [[33, 33, 39, 39, "CEO"]], [[46, 46, 51, 54, "Located in"]]]}
Thanks again.
from pure.
The ACE05 doesn't include these relations. You can define relation types for new task in shared/const.py and add your task name in the entity model and the relation model.
from pure.
I have altered the files as you suggested, I am using this command to train the model:
python run_entity.py --do_train --learning_rate=1e-5 --task_learning_rate=5e-4 --train_batch_size=16 --context_window 0 --data_dir data_dir --model bert-base-uncased --output_dir output_dir --task inception
using RTX 3060 with 12 GB RAM, NVIDIA-SMI 460.91.03, and cuda==11.1
The command is working but it is taking a very long time, it seems to be stuck on "Moving to CUDA...". I know the GPU is working, the fan is at 78% when running the code. Is it supposed to take long?
My train.json, test.json and dev.json files are all the same. Perhaps it is because they all have the same doc_key? The content is as follows:
{"doc_key": "Test_001", "sentences": [["Google", "was", "founded", "on", "September", "4", ",", "1998", ",", "by", "Larry", "Page", "and", "Sergey", "Brin", "while", "they", "were", "Ph.D", ".", "students", "at", "Stanford", "University", "in", "California", "."], ["On", "December", "3", ",", "2019", ",", "Pichai", "also", "became", "the", "CEO", "of", "Google", "."], ["In", "March", "1999", ",", "the", "Google", "moved", "its", "offices", "to", "Palo", "Alto", ",", "California", "."]], "ner": [[[0, 0, "ORG"], [10, 11, "PER"], [13, 14, "PER"]], [[33, 33, "PER"], [39, 39, "ORG"]], [[46, 46, "ORG"], [51, 54, "LOC"]]], "relations": [[[0, 0, 10, 11, "Founder"], [0, 0, 13, 14, "Founder"]], [[33, 33, 39, 39, "CEO"]], [[46, 46, 51, 54, "Located in"]]]}
from pure.
It is weird that the code gets stuck on "Moving to CUDA". Did you run the code on the datasets we used (e.g., SciERC) for a sanity check?
Also, please use different doc_key, as we use the doc_key when generating the final predictions.
from pure.
Hi, I ran the code on the datasets you used and I am still running into the same issue, the following is showing up on my screen:
09/15/2021 19:03:28 - INFO - root - Moving to CUDA...
I have some questions:
1.) What is the time required to wait for this command to execute?
2.) I do not have any clusters defined in the train, dev and test JSON files. Would this make any difference?
3.) I am using --model bert-base-uncased. Is this for relation extraction? I only need the model for relation extractions.
Just to clarify, my task is to make a custom relation extraction model from the pretrained model using my domain data and domain relations. Thank you very much for all your help.
from pure.
Hi, I think I have resolved this issue. It takes 10 minutes but it moves on from "Moving from CUDA..." to the next stage. Thank you very much.
from pure.
Related Issues (20)
- Multiple issues HOT 2
- different F1 with the same seed HOT 2
- tensorflow版本 HOT 1
- About the relation in datasets HOT 1
- [Paper] What are "gold" entity and relationship types? HOT 2
- Provide full environment
- Input Data Format HOT 5
- How to load models into Python HOT 2
- some code problems reguarding run_relation_approx(get_features_from_file) HOT 2
- where is the code of Efficient Batch Computations
- Approximation Model Training & Inference HOT 1
- entity is S or O ?
- Further question of f1 and e2e_f1
- 版本库问题 HOT 1
- 版本库问题
- ACE dataset
- Training a model on a dataset that is not ace04, ace05, or scierc HOT 1
- training model for WLP -- stuck in suboptimal solution
- Input data format question for custom dataset !
- cuda out of memory
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pure.