This is the original pytorch implementation for the following paper: [FECAM: Frequency Enhanced Channel Attention Mechanism for Time Series Forecasting](https://arxiv.org/abs/2212.01209).
If you find this repository useful for your research work, please consider citing it as follows:
@misc{jiang2022fecam,
title={FECAM: Frequency Enhanced Channel Attention Mechanism for Time Series Forecasting},
author={Maowei Jiang and Pengyu Zeng and Kai Wang and Huan Liu and Wenbo Chen and Haoran Liu},
year={2022},
eprint={2212.01209},
archivePrefix={arXiv},
primaryClass={cs.AI}
}
- [2022-12-01] FECAM v1.0 is released
- Support Six popular time-series forecasting datasets, namely Electricity Transformer Temperature (ETTh1, ETTh2 and ETTm1,ETTm2) , Traffic, National Illness, Electricity and Exchange Rate , ranging from power, energy, finance,illness and traffic domains.
- We generalize FECAM into a module which can be flexibly and easily applied into any deep learning models within just few code lines.
- Provide all training logs.
- Integrate FECAM into other mainstream models(eg:Pyraformer,Bi-lstm,etc.) for better performance and higher efficiency on real-world time series.
- Validate FECAM on more spatial-temporal time series datasets.
- As a sequence modelling module,we believe it can work fine on NLP tasks too,like Machine Translation and Name Entity Recognization.Further more,as a frequency enhanced module it can theoretically work in any deep-learning models like Resnet.
Stay tuned!
- Install the required package first(Mainly including Python 3.8, PyTorch 1.9.0):
cd FECAM
conda create -n fecam python=3.8
conda activate fecam
pip install -r requirements.txt
- Download data. You can obtain all the six benchmarks from Tsinghua Cloud or Google Drive. All the datasets are well pre-processed and can be used easily.
- Train the model. We provide the experiment scripts of all benchmarks under the folder
./scripts
. You can reproduce the experiment results by:
bash ./scripts/ETT_script/FECAM_ETTm2.sh
bash ./scripts/ECL_script/FECAM.sh
bash ./scripts/Exchange_script/FECAM.sh
bash ./scripts/Traffic_script/FECAM.sh
bash ./scripts/Weather_script/FECAM.sh
bash ./scripts/ILI_script/FECAM.sh
FECAM outperforms all transformer-based methods by a large margin.
Compared to vanilla models, only a few parameters are increased by applying our method (See Table 4), and thereby their computationalcomplexities can be preserved.Forecasting visualization:Visualization of ETTm2 and Exchange predictions given by different models.
FECAM visualization:Visualization of frequency enhanced channel attention and output tensor of encoder layer of transformer.x-axis represents channels,y-axis represents frequency from low to high,performing on datasets weather and exchange.
We conduct the experiments on 6 popular time-series datasets, namely Electricity Transformer Temperature (ETTh1, ETTh2 and ETTm1) and Traffic, Weather,Illness, Electricity and Exchange Rate, ranging from power, energy, finance , health care and traffic domains.
Datasets | Variants | Timesteps | Granularity | Start time | Task Type |
---|---|---|---|---|---|
ETTh1 | 7 | 17,420 | 1hour | 7/1/2016 | Multi-step |
ETTh2 | 7 | 17,420 | 1hour | 7/1/2016 | Multi-step |
ETTm1 | 7 | 69,680 | 15min | 7/1/2016 | Multi-step |
ETTm2 | 7 | 69,680 | 15min | 7/1/2016 | Multi-step&Single-step |
ILI | 7 | 966 | 1hour | 1/1/2002 | Multi-step |
Exchange-Rate | 8 | 7,588 | 1hour | 1/1/1990 | Multi-step&Single-step |
Electricity | 321 | 26,304 | 1hour | 1/1/2012 | Multi-step-step |
Traffic | 862 | 17,544 | 1hour | 1/1/2015 | Multi-step-step |
Weather | 21 | 52,695 | 10min | 1/1/2020 | Multi-step-step |
Download data. You can obtain all the six benchmarks from Tsinghua Cloud or Google Drive. All the datasets are well pre-processed and can be used easily.(We thanks Author of Autoformer ,Haixu Wu for sorting datasets and public sharing them.)
The data directory structure is shown as follows.
./
└── datasets/
├── electricity
│ └── electricity.csv
├── ETT-small
│ ├── ETTh1.csv
│ ├── ETTh2.csv
│ ├── ETTm1.csv
│ └── ETTm2.csv
├── exchange_rate
│ └── exchange_rate.csv
├── illness
│ └── national_illness.csv
├── traffic
│ └── traffic.csv
└── weather
└── weather.csv
If you have any questions, feel free to contact us or post github issues. Pull requests are highly welcomed!
Maowei Jiang: [email protected]
Thank you all for your attention to our work!
This code uses (Autoformer,Informer, Reformer, Transformer, LSTM,N-HiTS, N-BEATS, Pyraformer, ARIMA) as baseline methods for comparison and further improvement.
We appreciate the following github repos a lot for their valuable code base or datasets:
https://github.com/zhouhaoyi/Informer2020
https://github.com/thuml/Autoformer
https://github.com/cure-lab/LTSF-Linear