haounsw / pisa

PromptCast: A New Forecasting Paradigm

Introduction

This repository is the reporisity of PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting (TKDE2023). PISA is a large-scale dataset including three real-world forecasting scenarios (three sub-sets) with 311,932 data instances in total. It is designed to support and facilitate the novel PromptCast task proposed in the paper.

Numerical Time Series Forecasting vs. PromptCast

Exisiting numerical-based forecasting VS. Prompt-based forecasting

PromptCast Evaluation Metrics

• RMSE
• MAE
• Missing Rate: whether the numerical forecasting target can be decoded (via string parsing) from the generated output prompts.

PISA Dataset

Forecasting Scenarios

The proposed PISA dataset contrains three real-world forecasting scenarios:

• CT: city temperature forecasting
• ECL: electricity consumption forecasting
• SG: humana mobility visitor flow forecasting

Details of three sub-sets

Folder Structure (see Dataset)

Dataset
|── PISA-Prompt
    │── CT
        │-- train_x_prompt.txt
        │-- train_y_prompt.txt
        │-- val_x_prompt.txt
        │-- val_y_prompt.txt
        │-- test_x_prompt.txt
        │-- test_y_prompt.txt
    │── ECL
        │-- train_x_prompt.txt
        │-- train_y_prompt.txt
        │-- val_x_prompt.txt
        │-- val_y_prompt.txt
        │-- test_x_prompt.txt
        │-- test_y_prompt.txt  
    │── SG
        │-- train_x_prompt.txt
        │-- train_y_prompt.txt
        │-- val_x_prompt.txt
        │-- val_y_prompt.txt
        │-- test_x_prompt.txt
        │-- test_y_prompt.txt   

Benchmark Results

Please check Benchmark folder for the implementations of benchmarked methods.

RMSE and MAE performance

Missing Rate results

Results under train-from-scratch and cross-scenario zero-shot settings

If you think this repo is useful, please cite our papers

@ARTICLE{xue2023promptcast,
  author={Xue, Hao and Salim, Flora D.},
  journal={IEEE Transactions on Knowledge and Data Engineering}, 
  title={PromptCast: A New Prompt-Based Learning Paradigm for Time Series Forecasting}, 
  year={2023},
  volume={},
  number={},
  pages={1-14},
  doi={10.1109/TKDE.2023.3342137}}

@inproceedings{xue2022translating, 
  title={Translating human mobility forecasting through natural language generation}, 
  author={Xue, Hao and Salim, Flora D and Ren, Yongli and Clarke, Charles LA}, 
  booktitle={Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining}, 
  pages={1224--1233}, 
  year={2022} 
} 

@inproceedings{xue2022leveraging, 
  title={Leveraging language foundation models for human mobility forecasting}, 
  author={Xue, Hao and Voutharoja, Bhanu Prakash and Salim, Flora D}, 
  booktitle={Proceedings of the 30th International Conference on Advances in Geographic Information Systems}, 
  pages={1--9}, 
  year={2022} 
} 

pisa's Issues

Instructions to try Promptcast

Hello, this is nice work. I want to know if the instructions to try it locally are available to try on real world problems. Thanks,

question

Hi, could I ask if the author has provided all the code in the paper?

how can i do the numerical part?

@xuehaouwa
have you upload the npy files?
how can i do the numerical part? The data you present are for prompt

Datasets: how can i find the data xxx.npy

The data in such function:

def generate_numerical(raw_folder, save_path, mode="test", obs_length=15):
    raw_data = np.load(os.path.join(raw_folder, "sg_raw_" + mode + ".npy"))
    data_x = []
    data_y = []
    for i in range(len(raw_data)):
        data = raw_data[i]
        number_of_instance = len(data) - obs_length
        for j in range(number_of_instance):
            y = data[obs_length + j]
            x = data[j: obs_length + j]
            data_x.append(x)
            data_y.append(y)

    data_x = np.reshape(data_x, [-1, obs_length])
    np.save(os.path.join(save_path, mode + "_" + str(obs_length) + "_x.npy"), data_x)
    data_y = np.reshape(data_y, [-1, 1])
    np.save(os.path.join(save_path, mode + "_" + str(obs_length) + "_y.npy"), data_y)

named xxx.npy

Code to generate prompts from Time Series

Hi, I would like to do a similar work to the one presented in PISA but with a custom dataset. I tried to find the code to generate the prompts from the time series data but I did not find it in this repository. Is it available here? If not, are you planning to make it available any time soon? Thank you!

Hello! Asking for other data prompts

Thanks for your splendid work, but I have only discovered val_x_prompt and val_y_promt in "PISA-Prompt", could you please provide train and test prompt.txt ~, thanks for providing such insight in your paper, nice work, I enjoy watching the limitation and future work discussion, it's very interesting and perspective.
Dataset

|── PISA-Prompt
│── CT
│-- train_x_prompt.txt
│-- train_y_prompt.txt
│-- val_x_prompt.txt
│-- val_y_prompt.txt
│-- test_x_prompt.txt
│-- test_y_prompt.txt
│── ECL
│-- train_x_prompt.txt
│-- train_y_prompt.txt
│-- val_x_prompt.txt
│-- val_y_prompt.txt
│-- test_x_prompt.txt
│-- test_y_prompt.txt
│── SG
│-- train_x_prompt.txt
│-- train_y_prompt.txt
│-- val_x_prompt.txt
│-- val_y_prompt.txt
│-- test_x_prompt.txt
│-- test_y_prompt.txt

源码运行遇到一些问题

$ python cmd_inference.py
Traceback (most recent call last):
File "cmd_inference.py", line 7, in
from calculate_bleu import metric_with_missing_rate
ModuleNotFoundError: No module named 'calculate_bleu'

请问calculate_bleu是哪个包

Question about case study

I tried the case study, copied the prompt to ChatGPT, below is the output, I would like to double confirm, is this the same as how you get the output? It's just average of all numbers.