arkadaw9 / pgnn

Physics-guided Neural Networks (PGNN) : An Application In Lake Temperature Modelling

Physics-guided Neural Networks (PGNN) : An Application In Lake Temperature Modelling

This repository provides code for the PGNN paper. If you're using the code in your research please cite the paper https://arxiv.org/abs/1710.11431.

Abstract:

This paper introduces a novel framework for combining scientific knowledge of physics-based models with neural networks to advance scientific discovery. This framework, termed as physics-guided neural network (PGNN), leverages the output of physics-based model simulations along with observational features to generate predictions using a neural network architecture. Further, this paper presents a novel framework for using physics-based loss functions in the learning objective of neural networks, to ensure that the model predictions not only show lower errors on the training set but are also scientifically consistent with the known physics on the unlabeled set. We illustrate the effectiveness of PGNN for the problem of lake temperature modeling, where physical relationships between the temperature, density, and depth of water are used to design a physics-based loss function. By using scientific knowledge to guide the construction and learning of neural networks, we are able to show that the proposed framework ensures better generalizability as well as scientific consistency of results.

Datasets :

This paper considers the following two example lakes to demonstrate the effectiveness of PGNN framework.

1. Lake Mille Lacs in Minnesota, USA
2. Lake Mendota in Wisconsin, USA

Please note that the paper provides an semi-supervised approach framework, where the mean squared error on the temperature predictions are computed using a labeled dataset whereas the physics based loss can be computed from an unlabeled dataset. The labelled and unlabeled datasets can be found in the 'datasets\' directory under the name '[lake].mat' and '[lake]_sampled.mat' respectively. [lake] should be replaced by 'mendota' for Lake Mendota and 'mille_lacs' for Lake Mille Lacs.

Dependencies :

• Python 3.7.3
• Keras 2.2.5
• Tensorflow 1.14.0

Using the code :

The repository contains code and datasets needed for training and testing the PGNN framework described in the paper.

1. To save the models and the results after training please create a '\results\' directory.

mkdir results

2. Then run the script '\models\PGNN.py'.

cd models
python PGNN.py

The hyperparameters and the datasets for the PGNN framework can be changed from the script.

Note: An alternative version of the PGNN script with parser options can be used too. Example for Lake Mille Lacs with Adam Optimizer (other hyperparameters can be tuned accordingly):

cd hybrid
python hpd.py --dataset mille_lacs --optimizer_val 2 --data_dir ../datasets/ --use_YPhy 1 --lambda 100.0

Using the code for Hybrid Modeling:

The HPD model is same as the PGNN0, i.e., the physics-loss is not used $\lambda_{PHY}=0$. To run the HPD model use the following script '\hybrid\hpd.py' Example for Lake Mille Lacs with Adam Optimizer:

cd hybrid
python hpd.py --dataset mille_lacs --optimizer_val 2 --data_dir ../datasets/ --use_YPhy 1 --lambda 0.0

To use the Residual (Res) model, without the use of the addition YPhy input use the script '\hybrid\res_nn.py' Example for Lake Mille Lacs with Adam Optimizer:

cd hybrid
python res_nn.py --dataset mille_lacs --optimizer_val 2 --data_dir ../datasets/ --use_YPhy 0 --lambda 0.0

To run the Hybrid-Residual (HPD-Res) model, use the following:

cd hybrid
python res_nn.py --dataset mille_lacs --optimizer_val 2 --data_dir ../datasets/ --use_YPhy 1 --lambda 0.0

pgnn's Issues

Error while running the code on Colab

Hello I am trying to run your code on colab.
But I am getting the following error I can't see how I may fix it. Any ideas?

Traceback (most recent call last):
  File "PGNN.py", line 148, in <module>
    PGNN_train_test(optimizer_name, optimizer_val, drop_frac, use_YPhy, iteration, n_layers, n_nodes, tr_size, lamda, lake_name)
  File "PGNN.py", line 110, in PGNN_train_test
    spio.savemat(results_name, {'train_loss_1':history.history['loss_1'], 'val_loss_1':history.history['val_loss_1'], 'train_rmse':history.history['root_mean_squared_error'], 'val_rmse':history.history['val_root_mean_squared_error'], 'test_rmse':test_score[2]})
KeyError: 'loss_1'

What are the names of the columns in Xc and uX1,2 ?

Hello,
I have tried running your code following your article (https://arxiv.org/abs/1710.11431).
It runs fine but I don't understand how each of the 12 columns in the Xc and uX1,2 arrays relate to the names of the input drivers listed in table 1 of the article (p.6).
Which column is the 'Day of Year', 'Depth', etc ?
Is there any way to find the original dataset, before any transformation (standardization) is performed on the data ?
Thanks,
Best regards,
Clément

The phyloss doesn't pass any gradient to the model parameters

Hello, I have tried running your code following your article.
However, I found that tensorflow only calculates the gradients based on the MSE part in the combined_loss, meanwhile neglecting the phy_loss term.

For instance, I tried training the model only with the phy_loss: model.compile(loss=phyloss, ...), it will return "ValueError: No gradients provided for any variable, XXXXX".

I also tried tf.GradientTape() to calculated the gradients. The gradients calculated using the phyloss is a list of None value (i.e., [None, None, None, None, None, None, None, None])

with tf.GradientTape() as tape:
    Y_pred = model(trainX)
    #loss = mean_squared_error(trainY, Y_pred)
    #loss = totloss(trainY, Y_pred)
    loss = phyloss(trainY, Y_pred)
grads = tape.gradient(loss, model.trainable_variables)

Loading mat-files not possible

Hi,

I am having the problem, that i can't open the mat-files:

mat = sio.loadmat("mendota.mat", squeeze_me=True,
    variable_names=['Y','Xc_doy','Modeled_temp']) 

leads to:
ValueError: Unknown mat file type, version 97, 116

Keras v2.6.0
Scipy v1.8.0
Tensorflow 2.6.0

What is the actual data of Xc_doy1 and Xc_doy2?

I am trying to use your approach for another Physics-guided problem and have problems replicating it. I have backtracked your physical loss to Xc_doy1 and Xc_doy2, which are in the MatLab files with dimensions 649723x12. I have trouble understanding what these values actually stand for. Are these 649723 temperature samples at 12 different depths in the lake?