midas-research / fast-eacl

FAST: Financial news and tweet based time Aware network for Stock Trading

This codebase contains the python scripts for FAST: Financial News and Tweet Based Time Aware Network for Stock Trading.

EACL - The 16th Conference of the European Chapter of the Association for Computational Linguistics 2021 paper coming soon

Environment & Installation Steps

Create an environment having Python 3.6 and install the following dependencies

pip install -r requirements.txt

Contents

1. train.py consists of the training, validation, and testing scripts.
2. model.py comprises the definition of FAST and its components.
3. evaluator.py contains the script to evaluate model performance.
4. processed_data comprises the processed data in .npy files.

Data

Find the US S&P 500 data here, and the China & Hong Kong data here.

Training

Execute the following command in the same environment to train and test FAST:

python3 train.py

Processed data

Each .npy file corresponds to data about one lookback period. FAST takes the following data as inputs for training and evaluation:

1. Text: Text embeddings for each day in the lookback for all stocks in the dataset. Dimensions: (number of stocks, length of lookback period, number of texts, embedding size)
2. Timestamps: Inverse of time intervals between release of consecutive texts in a day, for all days, for all stocks. Dimensions: (number of stocks, length of lookback period, number of texts, 1)
3. Mask: A single value for each stock, set to 0 if the daily return ratio is small, or 1 otherwise. Dimensions: (number of stocks, 1)
4. Price: A single value for each stock, indicating the normalized adjusted closing price across all stocks. Dimensions: (number of stocks, 1)
5. Ground Truth: A single value for each stock, indicating the trading day's actual normalized return ratio. Dimensions: (number of stocks, 1) In the folder titled processed_data, we provide some sample data for clarity.

Preprocessing

To encode the texts, we use the 768-dimensional embedding obtained per news item or tweet by averaging the token-level outputs from the final layer of BERT. However, FAST is compatible with any and all 1-D text embeddings. To extract the timestamp input, we obtain the time interval (in minutes) between the release of two consecutive texts and compute its inverse. For masking to 0, we keep the return ratio threshold of 0.05. We normalize the adjusted closing prices and the return ratios across all stocks in the dataset.

Cite

If our work was helpful for your research, please kindly cite our work.

fast-eacl's Issues

About NDCG in evaluator.py

I have some questions about NDCG in evaluator.py. Why the parameters y_true and y_score in sklearn.metrics.ndcg_score() are gt_top5 and pre_top5, respectively, which are the index of group_truth and prediction?

Ref: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ndcg_score.html

`no_stocks` is undefined

I ran the your training script, but got the following error.

python train.py 
Traceback (most recent call last):
  File "train.py", line 210, in <module>
    model = FAST().to('cuda')
  File "/home/ubuntu/fast-eacl/model.py", line 74, in __init__
    self.text_lstm = [nn.LSTM(768,64) for _ in range(no_stocks)]
NameError: name 'no_stocks' is not defined

It seems that no_stocks is not defined in train.py. What does no_stocks represent?

would you mind sharing pre-processed data sets or pre-processed scripts?

Thank you for your contribution. would you mind sharing pre-processed data sets or pre-processed scripts?
You can contact me by email.
[email protected]

Best wishes.