cbaziotis / datastories-semeval2017-task4

Deep-learning model presented in "DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis".

datastories-semeval2017-task4's Issues

`semeval_data_loader.py` missing

Thumbs up for your code! Trying to reproduce your SemEval 2017 results for Subtask A, I realised that semeval_data_loader.py is missing in the dataset module. It's explicitly excluded from the repository via .gitignore.

If I'm not mistaken, there's no way to train a model without the SemEvalDataLoader class from semeval_data_loader.py. Would you mind sharing it?

Error running model_target.py

Hello,
I am trying to recreate the results of subtask B, 2 way classification of the tweets, and I have taken the following steps:
change the TASK = 'BD' in model_target.py

When I try running model_target.py, I am through until the model being trained, and then I get an error
TypeError: Using a tf.Tensor as a Python bool is not allowed. Use if t is not None: instead of if t: to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.

I have tried installing an older version of Tensorflow, but still get the same result.

Here is the full error:---------
TypeError Traceback (most recent call last)
in ()
101 attention="context",
102 clipnorm=.1,
--> 103 classes=len(classes))
104
105 plot(nn_model, show_layer_names=True, show_shapes=True, to_file="model_task4_sub{}.png".format(TASK))

/models/neural/keras_models.py in target_RNN(wv, tweet_max_length, aspect_max_length, classes, **kwargs)
175 h_aspects = shared_RNN(aspects_emb)
176 h_aspects = Dropout(drop_target_rnn)(h_aspects)
--> 177 h_aspects = MeanOverTime()(h_aspects)
178 h_aspects = RepeatVector(tweet_max_length)(h_aspects)
179

~/anaconda2/envs/py35/lib/python3.5/site-packages/keras/engine/topology.py in call(self, x, mask)
570 if inbound_layers:
571 # This will call layer.build() if necessary.
--> 572 self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
573 # Outputs were already computed when calling self.add_inbound_node.
574 outputs = self.inbound_nodes[-1].output_tensors

~/anaconda2/envs/py35/lib/python3.5/site-packages/keras/engine/topology.py in add_inbound_node(self, inbound_layers, node_indices, tensor_indices)
633 # creating the node automatically updates self.inbound_nodes
634 # as well as outbound_nodes on inbound layers.
--> 635 Node.create_node(self, inbound_layers, node_indices, tensor_indices)
636
637 def get_output_shape_for(self, input_shape):

~/anaconda2/envs/py35/lib/python3.5/site-packages/keras/engine/topology.py in create_node(cls, outbound_layer, inbound_layers, node_indices, tensor_indices)
164
165 if len(input_tensors) == 1:
--> 166 output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
167 output_masks = to_list(outbound_layer.compute_mask(input_tensors[0], input_masks[0]))
168 # TODO: try to auto-infer shape

~/anaconda2/envs/py35/lib/python3.5/site-packages/kutilities/layers.py in call(self, x, mask)
21 if mask is not None:
22 mask = K.cast(mask, 'float32')
---> 23 if not K.any(mask):
24 return K.mean(x, axis=1)
25 else:

~/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in bool(self)
612 TypeError.
613 """
--> 614 raise TypeError("Using a tf.Tensor as a Python bool is not allowed. "
615 "Use if t is not None: instead of if t: to test if a "
616 "tensor is defined, and use TensorFlow ops such as "

TypeError: Using a tf.Tensor as a Python bool is not allowed. Use if t is not None: instead of if t: to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.

Problem with "tensorflow_backend"

I get the below massage when I try to run model_message.py for subtask A;

Traceback

(most recent call last):

File "/Users/datastories-semeval2017-task4-master/models/neural/model_message.py", line 90, in
dropout_rnn_U=0.3)
File "/Users/datastories-semeval2017-task4-master/models/neural/keras_models.py", line 104, in build_attention_RNN
model.add(Attention())
File "/usr/local/lib/python3.6/site-packages/keras/models.py", line 332, in add
output_tensor = layer(self.outputs[0])
File "/usr/local/lib/python3.6/site-packages/keras/engine/topology.py", line 572, in call
self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python3.6/site-packages/keras/engine/topology.py", line 635, in add_inbound_node
Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python3.6/site-packages/keras/engine/topology.py", line 166, in create_node
output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
File "/usr/local/lib/python3.6/site-packages/kutilities/layers.py", line 94, in call
eij = K.dot(x, self.W)
File "/usr/local/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 819, in dot
y_permute_dim = [y_permute_dim.pop(-2)] + y_permute_dim
IndexError: pop index out of range

I wonder if you help me with this problem!

kutilities

in keras 2 when import kutilities following error printed:
cannot import name 'initializations'

in keras2 the initializations changed to initializers

Pre-processing slow for Subtask A on a dataset of 30,000 entries

Hi!

First, thanks for sharing the code!

I am trying to run the Subtask A on a different language (Latvian) dataset of ~30,000 entries, but the pre-processing takes now more than a day (and is stuck at 89% for over 8 hours). It seems that the disk usage is at 9% (why does it require so much disk I/O?), but the RAM usage is only at 1.5-3GB out of 16GB (i.e., there is plenty left). In terms of CPU, only one thread (out of 12) is used at 100%. I have my own word embedding file (from FastText). That is ~1.5GB large. Apart from that I just removed everything from the download directory, placed there the 30,000 dataset file (in the same format as the other files and also named as one of the training files), and replaced the contents of the test and gold files with a dataset of 1000 entries.

I have two questions:

1. Does the fact that there is English hard-coded in the (pre-processing) code mean that the solution won't support other languages out-of-the-box (i.e., without re-writing the pre-processing part)? I have pre-processed the data externally using my own tools. So ... all that I would expect to be done internally, would be splitting tokens on spaces.
2. What is the part that slows everything down so much and do you have an idea of whether it is reasonable to remove the part that is so slow?

save model

Dear cbaziotis,

I was able to run all your code for the subtask A and then I saved the nn_model in a h5 file.

However, when I tried to load it with the following code:

from keras.models import load_model
model = load_model('my_model2.h5')

I get "ValueError: Invalid layer: Attention"

If by chance you know how to solve this issue, I would really really appreciate

Thank you in advance

Matt

Word embeddings download link is invalid

Good job！ Thank your nice code. However, the word embeddings download link is invalid now. May
you provide a new link ?

Word statistics files not found!

Using TensorFlow backend.
hello
Word statistics files not found!
Downloading... Traceback (most recent call last):

Mismatch in Hyper Parameters provided in code and those specified in the paper

Not really an issue, more of a question.

I am carrying out some research on sentiment analysis and I was looking to reproduce the results of Subtask A, specified in the paper here. Now, I should add that with the source code as is provided, and with TensorFlow backend, I get very similar results. However, upon closer inspection, I noticed some inconsistincies between what's in the code and in the paper.

For example, the paper states that:

We train all of our networks to minimize the
cross-entropy loss, using back-propagation with
stochastic gradient descent and mini-batches of
size 128.

However, in the code, the batch size is 50:

history = nn_model.fit(training[0], training[1], validation_data=validation if not FINAL else testing, epochs=50, batch_size=50, class_weight=class_weights, callbacks=_callbacks)

There are few other inconsistencies as well. My question is, should I change the hyper parameters to the ones specified in the paper in order to exactly replicate the results? Or is exact replication not possible? I should also add that when I run the system multiple times, I get similar, but slightly different results for each epoch. Maybe this has something to with using TensorFlow as a backend? I've been reading that some people have had issues getting the same results when using TensorFlow backend.

How to use the model for prediction?

First of all thank you for making the code public!
I have run models/nn_task_message.py with PERSIST=True but don't see any saved model. How can I save the model and use it to generate predictions for my own set of Tweets?

2 "Quick-start" questions

What is the recommended version of tensorflow/theano to work with?
Will un-commenting the line that uses Attention-RNN model and use it instead of cnn-multi-filters should work out-of-the box (model_message.py)?

python version

which version of python should i use which should be compatible with your requirements.txt file?

PreProcessing... so slowly

can I make it quickly please ， it‘s so slow，PreProcessing...: 12%|█▏ | 5540/44613 [1:33:41<11:00:46, 1.01s/it]
is it normal？

short text topic modeling

hello,
Do you think this model performs good on short text topic modeling,?

what is 'input_aspect'

Hi
Can I ask that what is input_aspect ?
Is the aspect like query terms? If so , then what is training[1] in 'history = nn_model.fit(training[0], training[1]' of model_target.py ?