nico-curti / numpynet

Neural Networks library in pure numpy

Hi there, I'm Nico! 👋

Hi, I'm Nico Curti, a PhD at the Dept. of Physics and Astronomy of Bologna University (Italy).

Please feel free to clone projects, raise issues and submit PRs if you think something could be better.

• 💬 Ask me about anything
• 📫 You can contact me here

Happy Coding! 😊

Languages and Tools

Github groups

Latest pubblications

• Implementation and optimization of algorithms in Biomedical Big Data Analytics.

• A network approach for low-dimensional signatures from high-throughput data.

• Prediction of vascular ageing based on smartphone acquired PPG signals.

• rFBP: Replicated Focusing Belief Propagation algorithm.

• plasticity: Unsupervised Neural Networks with biological-inspired learning rules.

• COVID-19 Lung Segmentation.

• Automated Prediction of the Response to Neoadjuvant Chemoradiotherapy in Patients Affected by Rectal Cancer.

• Effectiveness of semi-supervised active learning in automated wound image segmentation.

• Automated wound image segmentation: transfer learning from human to pet via active semi-supervised learning.

• Characterization of pupillary light response features for the classification of patients with Optic Neuritis.

Top collaborators

Gianluca Carlini	Riccardo Biondi	Daniele Buschi	Daniele Dall'Olio	Enrico Giampieri	Alessandro Fabbri	Stefano Sinigardi

Current works

numpynet's Issues

Error in network.py

I get an error running the MNIST example:

Traceback (most recent call last):
  File "MNIST.py", line 107, in <module>
    model.compile(optimizer=Adam(), metrics=[accuracy])
  File "/Users/bblais/opt/anaconda3/lib/python3.7/site-packages/NumPyNet-1.0.0-py3.7.egg/NumPyNet/network.py", line 287, in compile
    self._check_metrics(metrics)
  File "/Users/bblais/opt/anaconda3/lib/python3.7/site-packages/NumPyNet-1.0.0-py3.7.egg/NumPyNet/network.py", line 299, in _check_metrics
    if len(infos.args) - len(infos.defaults) != 2:
TypeError: object of type 'NoneType' has no len()

It's caused by the fact that infos.defaults is None (not zero) if there are no defaults, so there is no len(). a fix like,

     infos = inspect.getfullargspec(func)
      if infos.defaults:
        num_defaults=len(infos.defaults)
      else:
        num_defaults=0

      if len(infos.args) - num_defaults != 2:

solves the problem.

Sequential/step mode

Is there a preferred way to do a sequential mode for training? I think I can just set batch=1, the input vector to a 1-sample matrix, etc... not sure if this will cause issues if I try.

(trying to use the neural network to do deep-q learning, which updates 1-input vector at a time).

Simple Linear Regression Network not working

So I'm just testing out some simple networks, like the one below for doing linear regression. Perhaps I'm missing something, but when I call the model.fit() it only prints Epoch 1/1000 and doesn't run anything.

# make some fake data
m_0, m_1 = 1.1, 2.1
intercept = bias = 3.1
coefficients = np.array([[m_0, m_1]])

x_obs=20*rand(200,2)-10
y_obs = (x_obs @ coefficients.T) + bias  + randn(*y_obs.shape)*0.5


X,y=x_obs,y_obs
num_samples, input_size = X.shape


# Reshape the data according to a 4D tensor
X = X.reshape(num_samples, 1, 1, input_size)
y = y.reshape(num_samples,1,1,-1)
batch=num_samples


# build the model
model = Network(batch=input_size, 
                input_shape=(1,1,input_size))

model.add(Connected_layer(outputs=1, activation='Linear'))
model.add(Cost_layer(cost_type='mse'))
model.compile(optimizer=Adam(lr=0.001), 
            metrics=[mean_accuracy_score])
model.batch=batch

model.fit(X=X, y=y, max_iter=1000,verbose=True)

Computation seems to be worng with BatchNorm

In Group Normalization, it says that the Batchnorm is computed along the channel dimension. However in this implementation, the forward of the BN is just computed regardless of the channel dimension:

self.x = inpt.copy()
self.mean = self.x.mean(axis=0)                             # shape = (w, h, c)
self.var = 1. / np.sqrt((self.x.var(axis=0)) + self.epsil)  # shape = (w, h, c)

I think it should be like this:

mean = np.mean(X, axis=(0, 2, 3), keepdims=True)
variance = np.var(X, axis=(0, 2, 3), keepdims=True)

How to save & load models

Hello,

Forgive me as I am still learning, but I am looking for some assistance please, once I have trained a CNN model how do I then save/load the model and make predictions on images? I try to use the model.save_model and model.save_weights, but how should the network initialisation be? Any examples of doing this once a model has been trained would be much appreciated.

thanks

No module named 'parser' error

`File ~/Documents/Git/NumPyNet/NumPyNet/NumPyNet/init.py:18
15 from .image_utils import image_utils
16 from .network import Network
---> 18 import parser
19 from . import rnn_utils
20 from .utils import print_statistics

ModuleNotFoundError: No module named 'parser'`

Seems like a simple local-import problem? I'll see if I can solve it myself, but if you happen to fix it before me so much the better. :)

Output of a network to a single input vector after being trained in batches

Not quite an issue, but I didn't see an example of this. I train a network in batches, say batch size of 10 like below. Then I want the output of each layer to a single input pattern. Not sure of the recommended way of doing this. My various attempts have failed. I copy one below, for a simple case.

I made a model like:

model_dict={
    'input':9,               # number of inputs
    'hidden':[(5,'relu'),],
    'output':(9,'linear'),  # number of moves
    'cost':'mse',
}
_model = Network(batch=10, input_shape=(1,1,model_dict['input']))

hidden=model_dict.get('hidden',[])
for n,typ in hidden:
    _model.add(Connected_layer(outputs=n, activation=typ))

n,typ=model_dict['output']
_model.add(Connected_layer(outputs=n, activation=typ))
_model.add(Cost_layer(cost_type=model_dict['cost']))

_model.compile(optimizer=Adam(lr=0.1))

_model.summary()

layer       filters  size              input                output
   0 input                    10 x   1 x   1 x   9   ->    10 x   1 x   1 x   9
   1 connected                10 x   1 x   1 x   9   ->    10 x   5
   2 connected                10 x   1 x   1 x   5   ->    10 x   9
   3 cost                     10 x   1 x   1 x   9   ->    10 x   1 x   1 x   9

This works to train a batch of 10 on dummy input vectors and target values:

X=np.zeros((10,9))
y=np.ones((10,9))

# Reshape the data according to a 4D tensor
num_samples_X, size_X = X.shape
num_samples_y, size_y = y.shape

assert num_samples_X==num_samples_y

X = X.reshape(num_samples_X, 1, 1, size_X)
y = y.reshape(num_samples_y, 1, 1, size_y)

_model.fit(X=X, y=y, max_iter=10,verbose=False)

Now I want to get the model predictions for a single vector (and also get access to the outputs of each layer, but I think I know how to do that part):

Xp=np.random.randn(1,9)
size_X=9
Xp = Xp.reshape(1, 1, 1, size_X)  # reshape to a single 4d tensor

_model.predict(Xp)

yields the error:

ValueError                                Traceback (most recent call last)
Cell In[52], line 5
      2 size_X=9
      3 Xp = Xp.reshape(1, 1, 1, size_X)
----> 5 _model.predict(Xp)

File ~/anaconda3/lib/python3.11/site-packages/NumPyNet-1.0.0-py3.11.egg/NumPyNet/network.py:656, in Network.predict(self, X, truth, verbose)
    653 num_data = len(X)
    654 _truth = None
--> 656 batches = np.array_split(range(num_data), indices_or_sections=num_data // self.batch)
    658 begin = now()
    659 start = begin

File <__array_function__ internals>:200, in array_split(*args, **kwargs)

File ~/anaconda3/lib/python3.11/site-packages/numpy/lib/shape_base.py:778, in array_split(ary, indices_or_sections, axis)
    776 Nsections = int(indices_or_sections)
    777 if Nsections <= 0:
--> 778     raise ValueError('number sections must be larger than 0.') from None
    779 Neach_section, extras = divmod(Ntotal, Nsections)
    780 section_sizes = ([0] +
    781                  extras * [Neach_section+1] +
    782                  (Nsections-extras) * [Neach_section])

ValueError: number sections must be larger than 0.

Display in jupyter notebook

In the Network.fit() method, there is no way to either shut off the printing (e.g. verbose flag) or a way to display properly in a jupyter notebook. The first is easy to implement, the second might need something like:

    try:
        from IPython.core.display import clear_output
        have_ipython = True
    except ImportError:
        have_ipython = False

and then

      try:
          clear_output(wait=True)
      except Exception:
          # terminal IPython has no clear_output
          pass

perhaps a better way would be to use a package like tqdm which works in either notebook or console mode.