edapinenut / gntpp

This is the official codebase of `Exploring Generative Neural Temporal Point Process' (Accepted by TMLR).

Exploring Generative Neural Temporal Point Proces (GNTPP)

This is a PyTorch implementation of GNTPP in the paper GNTPP: `Exploring Generative Neural Temporal Point Process' (TMLR).

Requirements

• torch>=1.7.0
• ticks
• xarray
• torchdiffeq

Dependency can be installed using the following command:

conda env create --file env_gntpp.yaml
conda activate GNTPP_env

Training the Model

Run the following commands to train the model.

#  Training the Model
python tpp_train.py --dataset_dir ./data/{dataset}/ --hist_enc {encoder} --prob_dec {decoder}

{dataset} can be replaced by [mooc, retweet, stackoverflow, yelp, synthetic_n5_c0.2].

{encoder} can be replaced by [LSTM, Attention].

{decoder} can be replaced by [CNF, Diffusion, GAN, ScoreMatch, VAE, LogNorm, Gompt, Gaussian, Weibull, FNN, THP, SAHP].

(NOTE: The provided THP and SAHP use different type-modeling methods (type-wise intensity modelling), while others model all the types in a single sequence. So the final metric evaluation will be in a different protocol.)

Some of the datasets are larger than allowed, so we provided all these datasets file at Google Drive.

Preparing Your Own Datasets

If you want to use your own dataset, please prepare it in a Dict, where the keys include timestamps, types, lengths, intervals, t_max and event_type_num. (Refer to Line.88 in ./datasets/tpp_loader.py).

Building up Your Own Neural TPP

If you want to build up your own model, please refer to ./test.py, to see how the different modules in our paper constitute the TPP models.

Relation Visualization

For learned model, Trainer.plot_similarity method could provide the relations among events learned by the model. A similarity matrix will be generated, which looks like the following:

If the repository is helpful to your research, please cite the following:

@article{
lin2022GNTPP,
title={Exploring Generative Neural Temporal Point Process},
author={Haitao Lin and Lirong Wu and Guojiang Zhao and Pai Liu and Stan Z. Li},
journal={Transactions on Machine Learning Research},
year={2022},
url={https://openreview.net/forum?id=NPfS5N3jbL},
note={}
}

or

@misc{lin2022GNTPP,
  doi = {10.48550/ARXIV.2208.01874},
  
  url = {https://arxiv.org/abs/2208.01874},
  
  author={Haitao Lin and Lirong Wu and Guojiang Zhao and Pai Liu and Stan Z. Li},

  keywords = {Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
  
  title = {Exploring Generative Neural Temporal Point Process},
  
  publisher = {arXiv},
  
  year = {2022},
  
  copyright = {Creative Commons Attribution 4.0 International}
}

gntpp's Issues

Missing retweet data

Hi, Nice work, and thanks for open sourcing the codes and data.
a quick note: I do not see retweet data under data/, could you please upload it?

question about predict_start_from_noise

"sqrt_recip_alphas_cumprod", to_torch(np.sqrt(1.0 / alphas_cumprod)
"sqrt_recipm1_alphas_cumprod", to_torch(np.sqrt(1-alphas_cumprod/alphas_cumprod ))

    def predict_start_from_noise(self, x_t, t, noise):
        return (
            extract(self.sqrt_recip_alphas_cumprod, t, x_t.shape) * x_t
            - extract(self.sqrt_recipm1_alphas_cumprod, t, x_t.shape) * noise
        )

in this code it means:

$$ x_0=\frac{x_t}{\sqrt{\bar{a}_t}}-\frac{1}{\sqrt{1-\bar{a}}}\varepsilon $$

but in diffusion models we know this

$$ x_t(x_0,\varepsilon)=\sqrt{\bar{a}_t}x_0+\sqrt{1-\bar{a}}\varepsilon -----> x_0=\frac{x_t}{\sqrt{\bar{a}_t}}-\frac{\sqrt{1-\bar{a}_t}}{\sqrt{\bar{a}_t}}\varepsilon $$

so how to understand this problem?

Questions on *.pkl raw files

Hello, thanks for sharing your codes and dataset for this nice paper.

Could you please upload Python code for building raw *.pkl files in data/ folder?

Coding error in GNTPP/models/prob_decoders/vae_modules /vae.py

There is an error in file vae.py. At the version on May 6, 2023, the bug is in line 131, where KL-divergence term is computed. It reads as follows:
kl_loss = (- 0.5 * torch.sum(1 + log_var - mean.pow(2) - log_var.exp())) * seq_onehots
Note that the "dim" parameter is passed to torch.sum(), which means this operation is summing over the whole tensor "1 + log_var - mean.pow(2) - log_var.exp()", yielding a scalar! No exception is raised, because multiplying this scalar to the onehot mask is still valid and it preserves the tensor shape. But this is not what we wanna do to compute the loss. This line should be refactored as:
kl_loss = (- 0.5 * torch.sum(1 + log_var - mean.pow(2) - log_var.exp(), dim=-1)) * seq_onehots

Please check it out.

Wrong splitting of embeddings.

https://github.com/BIRD-TAO/GNTPP/blob/2d67284d638882555a69f4e74d8dd90fd10cca5f/models/tpp_warper.py#L76

I think you mixed up the indexing of the type embedding.

Coding error in GNTPP/models/prob_decoders/diffusion_modules /gaussian_diffusion.py

There might be an error in GaussianDiffusion.p_losses(self, x_start, t, noise=None), where the loss of the form $\lVert \epsilon-\epsilon_\theta(\sqrt{\bar{\alpha}_t}x_0+\sqrt{1-\bar{\alpha}_t}\epsilon,t) \rVert^2 $ is computed. The returned value of self.denoise_fn(x_noisy, t, cond=cond), i.e. x_recon, is actually the predicted noise $\epsilon_\theta(\sqrt{\bar{\alpha}_t}x_0+\sqrt{1-\bar{\alpha}_t}\epsilon,t)$.

The code currently reads as:
loss = torch.square(x_noisy - x_recon).sum()

Since x_noisy is the corrupted sample, i.e. $x_t$, and I don't think $x_t-\epsilon_\theta$ makes any sense. So, in line 273, I think the correct code should be:
loss = torch.square(noise- x_recon).sum()

Please check it out.