cvxtz / time_series_forecasting

time_series_forcasting

Code for https://towardsdatascience.com/how-to-use-transformer-networks-to-build-a-forecasting-model-297f9270e630

Install (GPU)

conda create -n py38 python=3.8
conda activate py38
conda install pytorch torchvision cudatoolkit=11.1 -c pytorch -c conda-forge
conda install -c conda-forge jupyterlab
conda install -c conda-forge matplotlib
git clone https://github.com/CVxTz/time_series_forecasting
cd time_series_forecasting
pip install .

Run

bash pipeline.sh

time_series_forecasting's Issues

visualization.json

Have no visualization.json file!

有个 疑问

想问一下 代码怎么应用到多变量时间系列预测

Missing config.js file

Hello, I am using your code. But now I have a problem, I need to use your config.js file. You can provide it, or talk about how to write this code？

Skip connection in encoder and decoder

Hi, thanks for sharing this wonderful project. I notice that in the transformer encoder, there is a skip connection between the encoder output and input, i.e. src = self.encoder(src) + src_start.
It seem different from the original transformer model? Decoder also has the same connection.

def encode_src(self, src):
    src_start = self.input_projection(src).permute(1, 0, 2)

    in_sequence_len, batch_size = src_start.size(0), src_start.size(1)
    pos_encoder = (
        torch.arange(0, in_sequence_len, device=src.device)
        .unsqueeze(0)
        .repeat(batch_size, 1)
    )

    pos_encoder = self.input_pos_embedding(pos_encoder).permute(1, 0, 2)

    src = src_start + pos_encoder

    src = self.encoder(src) + src_start

    return src

How to use on another dataset where we dont want to do group by

Hi Please suggest how can i use this where I don't have categorical col to group by...like simple timestamp and say like temperature.

Predictions

Hi Mate

Enjoying trying out a few of your projects really helping me on my deep learning journey, I'm currently doing your Transformer time series forecast project. I've got to the part where the model has trained and saved to transformer_ts.pth. How do I make the future predictions? I'm a noob to transformers lol

Hope for a reply
G

Question regarding the data shape.

Hi, first, thanks for sharing the code for the transformer network application in time series data. So I read your code and notice that your source, tgt_in, and tgt_out all are a matrix with 3 dimensions. I started to wonder what those dimensions mean. For instance, in LSTM applications, the 3 dimensions would mean, respectively: batch, timestamps, and features.

With all that in mind, I want to know if you follow the same data format of the LSTM network (batch, timestamps, features). Thanks in advance!!