soroushv / tweet2vec

Code for the implementation of Tweet2Vec

Tweet2Vec

Please Cite: Soroush Vosoughi, Prashanth Vijayaraghavan and Deb Roy. (2016). Tweet2Vec: Learning Tweet Embeddings using Character-level CNN-LSTM Encoder-Decoder. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2016). Pisa, Italy. ##Requirements

1. numpy
2. Theano
3. joblib

##Training: Create subdirectories to store logs and models. However you can specify your own dir from command line: logs/ all_data/

Use the following command to see all the options & hyperparameters for training that can be specified from command line:

THEANO_FLAGS=mode=FAST_RUN,device=gpu1,floatX=float32 PYTHONPATH=. python model/train.py -h

Using default parameters, training can be performed by : THEANO_FLAGS=mode=FAST_RUN,device=gpu1,floatX=float32 PYTHONPATH=. python model/train.py -t -i ./train_file.json -m "new_model" -E 3

Training data format is provided in sample.json. However, the tweet1 & tweet2 need to be a modified version. It could either be replies to tweets which have similar meaning or just synonym modified version of the tweet.

NOTE: Training takes a long time! The cost is the total sum of the log probabilities across each batch, timestep and decoder. Note that the Cost will fluxuate a lot. You could experiment with other loss functions as in loss.py. It takes more than a week to get good vectors.

##Embeddings

Use the following command to see all the options & parameters for getting embeddings that can be specified from command line:

THEANO_FLAGS=mode=FAST_RUN,device=gpu1,floatX=float32 PYTHONPATH=. python model/embeddings.py -h

Get embedding using the following command: THEANO_FLAGS=mode=FAST_RUN,device=gpu1,floatX=float32 PYTHONPATH=. python model/embeddings.py -R -i /res/tweets.json -m "new_model" -e 3 -o "/res/emb.npy"

tweet2vec's Issues

Can not retrive encodings

I specified output file to be test1, but I get error No such file or directory 10//encoding_10_test1.npy

Problem running example

Running example sample.json.

[paulo@wormhole Tweet2Vec-master]$ THEANO_FLAGS=mode=FAST_RUN,floatX=float32 PYTHONPATH=. python model/train.py -t -i sample.json -m "new_model" -E 3
Using TensorFlow backend.
2017-06-12 18:59:47,981 - utils - INFO - Loading Data from: sample.json
2017-06-12 18:59:47,982 - utils - INFO - Loaded Data from: sample.json
Creating Training Function...
2017-06-12 19:00:29,165 - main - INFO - Starting epoch 1
2017-06-12 19:00:29,166 - main - INFO - Training with the Adam update function
2017-06-12 19:00:29,166 - main - INFO - Training epoch 1.
2017-06-12 19:00:29,166 - main - INFO - Epoch 1 results
Traceback (most recent call last):
File "model/train.py", line 145, in
trainer.train_enc_dec(train_file=args.train_file, model_name=args.model_name, num_epochs=args.epochs,out_dir=args.out_dir)
File "model/train.py", line 98, in train_enc_dec
self.logger.info("Saving model parameters for epoch {0} at path {1}".format(e, ))
IndexError: tuple index out of range

Switch to other loss function

I'm not quite familiar with theano. Can anyone help me with how to change to other loss function? A modified version of code is very appreciated.

Char-level one-hot encoding+CNN-LSTM encoding/decoding vs standard word2vec+CNN-LSTM encoding/decoding

If I am not wrong this work is build on char-level one-hot encoding+CNN-LSTM encoding/decoding.

Any commends on using char-level one-hot encoding+CNN-LSTM encoding/decoding compare to word2vec+CNN-LSTM encoding/decoding ?

get_embeddings, and output vector size ?

Hi, thanks for sharing the code with community. I want to know if the the output vector size is 256 ?

I also got one problem when I try to run the code.

in embeddings.py

Line 61

for i,xx,yy in data_manager.batch_generator_test(tweets,tweets,batch_size=256):

The function 'batch_generator_test' gives 4 return values instead of three.