yuliarubanova / latent_ode Goto Github PK

Hello!

I am trying to use latent ODE functionality to interpolate a set of points with over 5k irregularly sampled timesteps. When using the current version and python3 run_models.py --niters 500 -n 1000 -s 50 -l 10 --dataset periodic --latent-ode --noise-weight 0.01 --max-t 6000 I get an error in on line 275 : assert(not torch.isnan(inc).any()). A short-term solution for me is to just scale my time series down to ranges used in paper/examples. I am curious if you can also reproduce this error and if it is expected behavior. I am happy to follow up with any information needed to reproduce.

Also... this does not occur when I set extrapolate to True...perhaps that is pointing to some issue with how the series is sampled.

by the way I am using:
torchdiffeq 0.2.1
torch 1.8.0
Python 3.7.2

Hi @YuliaRubanova, @duvenaud, @rtqichen and other users,

I tried to run this line:
python3 run_models.py --niters 300 -n 10000 -l 15 --dataset hopper --latent-ode --rec-dims 30 --gru-units 100 --units 300 --gen-layers 3 --rec-layers 3

And after some epochs I always receiving such an error:

Other examples seem to work fine, but I'm interested in this one. Can you look into it?
Thanks in advance.

Hello, @YuliaRubanova!
I found a small bug here. This line
idx_not_nan = 1 - torch.isnan(mortality_label)
causes RuntimeError with PyTorch 1.4.0 because torch.isnan returns torch.BoolTensor, which cannot be subtracted from int.
Everything works if substitute this line with the following:
idx_not_nan = ~torch.isnan(mortality_label)

Hi Yulia,

  Thanks a lot for providing the open-source code. Currently the experiment Latent ODE with RNN encoder has the following error. 
 latent_ode/lib/encoder_decoder.py", line 135, in forward
if mask is not None:

UnboundLocalError: local variable 'mask' referenced before assignment

 Could you provide some suggestions? 

 Highly appreciate your time and help! 

Hi @YuliaRubanova
Really interested in the code and looking at building jupyter notebooks with alternate data sets.
I was just wondering how best to use the code on say a pandas data set?
Can this be used on a multivariate series?
Many thanks and best,
Andrew

So I load this checkpoint path from the trained latent ODE model:
ckpt_path = '/experiments/experiment_11387.ckpt'
checkpt = torch.load(ckpt_path)

If I want to feed a new unseen time-series into the model and visualize the forecast, should I use LatentODE().get_reconstruct? If so, how do I initialize the model from the weights from the checkpoint path (i.e. the model I just trained)?

I'm trying to use the Visualize functions but they seem to need a data_dict and I can't find where you make this or what components it should have.

Hi Yulia,

Please add requirements.txt file to the repo so that dependencies can be installed.

Hi. According to the author paper http://papers.nips.cc/paper/8773-latent-ordinary-differential-equations-for-irregularly-sampled-time-series.pdf, page 6, you said you used scheduled sampling in the extrapolation mode by feeding previous predictions instead of observed data. But how could you do this in latent ODE?, since the latent dimension can be different from observed data dimension and the ODE decoder receive no other inputs (whether observed data x or its own prediction x_hat) other than its own internal latent state z.

Please suggest.
Thank you.

Hi, my time series have many features and each of them is observed at different times. How can I use latent-ode here to learn from such data? It seems the current collate_fn can be used for time series with different observation times but in each time, it assumes all the features are observed (but in my case, some features are missing while others are observed in one time series).

Can I just use the predicted values of unobserved features from the ode forward (i.e. x(t) = ODEsolver(neural_net, y0, time at the next observation) and set x_unobserved = x[index of unobserved features] ), and update the state by GRU using x(t) = [x_observe at t, x_unobserved at t] as the input to the GRU.

Please suggest.

Thank you.

Hi,

   Thanks a lot for your open source code. There is some error for line 233 of likelihood_eval.py. res = compute_masked_likelihood(mu, data, mask, mse).  No mse has been defined before. Thanks a lot.

Hi there, thanks for sharing your code.

C:\Users\admin\latent_ode_updated\run_models.py
run_models.py --niters 100 -n 8000 -l 20 --dataset physionet --latent-ode --rec-dims 40 --rec-layers 3 --gen-layers 3 --units 50 --gru-units 50 --quantization 0.016 --classif

AttributeError Traceback (most recent call last)
~\latent_ode_updated\run_models.py in
262
263 batch_dict = utils.get_next_batch(data_obj["train_dataloader"])
--> 264 train_res = model.compute_all_losses(batch_dict, n_traj_samples = 3, kl_coef = kl_coef)
265 train_res["loss"].backward()
266 optimizer.step()

~\latent_ode_updated\lib\base_models.py in compute_all_losses(self, batch_dict, n_traj_samples, kl_coef)
257 batch_dict["observed_data"], batch_dict["observed_tp"],
258 mask = batch_dict["observed_mask"], n_traj_samples = n_traj_samples,
--> 259 mode = batch_dict["mode"])
260
261 #print("get_reconstruction done -- computing likelihood")

~\latent_ode_updated\lib\latent_ode.py in get_reconstruction(self, time_steps_to_predict, truth, truth_time_steps, mask, n_traj_samples, run_backwards, mode)
86
87 # Shape of sol_y [n_traj_samples, n_samples, n_timepoints, n_latents]
---> 88 sol_y = self.diffeq_solver(first_point_enc_aug, time_steps_to_predict)
89
90 if self.use_poisson_proc:

~\Anaconda3\envs\gpu-supported\lib\site-packages\torch\nn\modules\module.py in call(self, *input, **kwargs)
487 result = self._slow_forward(*input, **kwargs)
488 else:
--> 489 result = self.forward(*input, **kwargs)
490 for hook in self._forward_hooks.values():
491 hook_result = hook(self, input, result)

~\latent_ode_updated\lib\diffeq_solver.py in forward(self, first_point, time_steps_to_predict, backwards)
40 pred_y = odeint(self.ode_func, first_point, time_steps_to_predict,
41 rtol=self.odeint_rtol, atol=self.odeint_atol, method = self.ode_method)
---> 42 pred_y = pred_y.permute(1,2,0,3)
43
44 assert(torch.mean(pred_y[:, :, 0, :] - first_point) < 0.001)

AttributeError: 'tuple' object has no attribute 'permute'

I see that in ODEFunc you have implemented ODEFunc_w_Poisson while in the run_model.py I found that you haven't implemented any RNN-ODE with Poisson likelihood yet. How should I use Poisson Likelihood function? Can you give an example? Thanks a lot!

I was reading your paper and there was mention of supplement document,can you please share the same so that we can understand on how number of latent state, hyper-parameters were selected.