ds3lab / cognival-cli Goto Github PK

I was trying to import cognival to run it within a python script.
After these steps:
conda create --name cognival python=3.7
conda activate cognival
pip install -r requirements.txt
python setup.py install
and in python: from cognival import cog_evaluate I got an ModuleNotFoundError.

Hello :)

after creating a new conda env with conda create --name cognival python=3.7 and installing the requirements with pip install -r requirements.txt I got an error saying that tensorflow is missing. In the requirements.txt history I saw that 1.15.2 was the latest used version. Maybe we should add it? I tried 2.5.0 out of curiosity but got some problems with missing losses in the model history object.

After installing cognival python setup.py install I got the error ArrayField.empty_field: return type None is not a <class allennlp.data.fields.field.Field> from allennlp. Suggested by this thread I installed pip install overrides==3.1.0 which solved the error.

Hi,
When I was trying to reproduce the results in Appendix B from the paper, however, I failed to reproduce the BERT word embeddings with cognitive data sources in all three modalities.

I was using the standard_vocab.txt (~/cognival-cli/user/resource/standard_vocab.txt) with the BERT-base/large models that downloaded from embedding_registry.json to extract the hidden states of the second to last of 12/24 output layers as the representation for every single token.

Assuming something is wrong when I was extracting the features from the BERT models, and would you be so kind as to provide more details about how did you generate the BERT word vectors or share the bert_base.txt and bert_large.txt which you used to generate the results in Appendix B from the paper if it is doable?

Btw, I m adding a screenshot of my BERT feature-extracting script to this issue for a better description.

Many thanks,
Jack

Hi :)

I run into a small performance problem where I have multiple embeddings with the same dimensionality.
First, I created random embeddings for each of them, but this lead to quite some amount of training runs.
Then I wanted to activate random embeddings for only one of them but got an error that random embeddings are missing for the other when I run the experiment.

This is caused by https://github.com/DS3Lab/cognival-cli/blob/master/cognival/lib_nubia/commands/process.py#L575.
I see the reason behind the warning but it might be also nice to instead set a None random embedding in the config, to decrease the training time. For me this decreased the number of jobs from ~6000 to ~1800.