solving simple ode about neurodiffeq

[shuheng-liu]

You need to change the np.exp(3t) to torch.exp(3*t). Because x and t are both tensors.
In addition, 5x should be 5*x

[shuheng-liu]

As with all deep learning processes, you can try more extensive hyper-parameter tuning.

For example, if you change the activation from torch.nn.Tanh to neurodiffeq.networks.Swish

from neurodiffeq.networks import Swish
fcnn = FCNN(......, actv=Swish)

The result gets much better.

In addition, if you try to increase the number of hidden units (currently you only have 5 for each hidden layer), the number of points sampled per epoch (currently N and N2 are only 5), or the number of epochs (currently 1500 + 1000), the result will be even better than it.

[shuheng-liu]

The exponential function is a simple case, and you can have decent solutions with very few parameters and very few sampled points. If you try to solve some more complicated (e.g. nonlinear, higher-order) ODE systems, you might want to use more points.

Personally, I use 1k ~ 16k points every epoch for the problem I solve for. My network architecture is also more complicated with a single hidden layer of 4k hidden units. If you have a GPU instance (such as Google Colab), I would recommend filling as much GPU memory as possible.

[Arup-nit]

thank you, sir. for your kind help. could u send me your google scholar id? I want to follow ur research

[Arup-nit]

ImportError: cannot import name 'Swish' from 'neurodiffeq.networks' (C:\Users\USER\anaconda3\lib\site-packages\neurodiffeq\networks.py)
#opps it is showing error

[shuheng-liu]

Sorry, I forgot that the Swish activation function is included in a future v0.3.0 release.

If you want to use it, try one of the following

1. run pip uninstall neurodiffeq to remove the current installation; and follow the Manual Install section in the README.md to install the pre-release. You need to have git installed for the manual installation process.

2. Alternatively, just define the class somewhere in your code. Then you'll be able to use the Swish activation.

import torch
import torch.nn as nn


class Swish(nn.Module):
    def __init__(self, beta=1.0, trainable=False):
        super(Swish, self).__init__()

        beta = float(beta)
        self.trainable = trainable
        if trainable:
            self.beta = nn.Parameter(torch.tensor(beta))
        else:
            self.beta = beta

    def forward(self, x):
        return x * torch.sigmoid(self.beta * x)

I'm still a student and don't have a google scholar id, but you're welcome to follow the work at Institute for Applied Computational Science (IACS), Harvard.

[Arup-nit]

without using swish I just increase no of epoch and N2 layer it gives a better result .
I have just one query if I want to find ANN solution on a particular point like t= 3.345, what will be code for it ?

[Arup-nit]

If I want to find an ANN solution on a particular point like t= 1.345, what will be the code for it?

You need to change the np.exp(3t) to torch.exp(3*t). Because x and t are both tensors.
In addition, 5x should be 5*x

ya it works ty brother

[shuheng-liu]

Just pass a tensor of shape (n_points, 1) to the returned solution. If you are only interested in one particular time point, n_points is equal to 1, and the shape is (1, 1).

solution, loss_history = solve(...)

t0 = 1.345
t = torch.ones(1, 1, requires_grad=True) * t0

output_tensor = solution(t)  # this is a PyTorch tensor of shape (1, 1)
output_float = output_tensor.item()  # this is a Python built-in float
output_array = output_tensor.detach().cpu().numpy()  # this is a numpy.ndarray of shape (1, 1)

Note that the requires_grad=True is optional. You only need it if you want to compute something like:

derivative = diff(output_tensor, t)

[Arup-nit]

Dear Liu, just tell me Can I use these ANN results in my research?
do neurodiffeq follow the algorithm of solving ODE by Lagaris. et. al. in paper doi- 10.1109/72.712178 ?

[shuheng-liu]

Yes, it does follow the method in this paper with more customizable features.

The parameterization might be different from the paper, I recommend you check out our documentation for the neurodiffeq.conditions.IVP class.

[Arup-nit]

Yes, it does follow the method in this paper with more customizable features.

The parameterization might be different from the paper, I recommend you check out our documentation for the neurodiffeq.conditions.IVP class.

I found your paper on neurodiffeq. will follow this , and cite it. this package result has better convergence.

[Arup-nit]

Dear Liu, just tell me Can I use my ann result using neurodiffeq in my research?

[shuheng-liu]

Dear Liu, just tell me Can I use my ann result using neurodiffeq in my research?

Yes. Neurodiffeq is open-source software and you're welcome to use it in your research.