duguyue100 / conceptors

A Python implementation of Conceptors.

Conceptors

This repo aims to provide a python implementation of H. Jaeger's conceptor network. The original work can be found from here.

Requirements

This package is developed under following system and packages:

• Ubuntu 14.04
• python 2.7.9
• numpy 1.9.1
• scipy 0.14.0
• matplotlib u'1.4.0'

These packages can be found and installed by using Anaconda 2.1.0.

Updates

• Structure sketch [DONE]
• First working version [2015-02-26]
• Refined version of conceptor network [2015-02-27]
• Autoconceptors [2015-03-05]
• Random Feature Conceptors [UNTESTED 2015-03-06]
• Conceptor visualization [2015-02-27] [A test is added in the code, not in a function, visualization seems fine]
• Conceptor I/O: saving and loding conceptors in file [TODO]
• Now the naive conceptor can accept multidimensional input instead of 1-d input [2014-02-28]
• Japanese Vowels recognition test [TODO]
• Conceptors network fixed [2015-03-05]
• Update logical operators (AND, OR, NOT) [UNTESTED, 2015-03-05]
• Input simulation matrix D has problem of simulating input [TODO]

Notes

• Based on my reading so far, there is really no training for the network. The readout weights and target weights are calculated analytically.

• I did a test of using two patterns, then it's able to recall barely. (this implementation is purly based on the test.m of the file) [2015-02-26] [This is not true anymore, the tests I ran is a messy recall, it's supposed to be like that way. the new version refined the test results]

• The final objective is to realize Random Feature Conceptor network, this network gives a biological plausible solution to realize conceptors.

• I was testing the conceptor network and this time, I used a 2-d signal instead of 1-d. The signal is made up by two sine waves that have different frequencies. Turns out the network output can almost match the first dimension, and it's failed to reconstruct the second dimension. (This problem is fixed, I mis-calculated one equation in the updating function).

• The reconstruction of autoconceptor is not as expected. However, if we change the input simulation matrix D to input signal, then the reconstruction is improved.

Contacts

Yuhuang Hu
Advanced Robotic Lab
Department of Artificial Intelligence
Faculty of Computer Science & IT
University of Malaya
Email:[email protected]

conceptors's Issues

Values converge to 1.0 then RuntimeWarning

I tried plugging in the CIFAR10 data normalized by reproducing your util.load_CIFAR10 function (which is not in the repository), and after one step of the inner training loop x converges to 1.0 which throws a warning:

conceptors/conceptors/net.py:325: RuntimeWarning: divide by zero encountered in arctanh
  self.W_targets=np.arctanh(self.all_train_args)-numpy.matlib.repmat(self.W_bias, 1, self.num_pattern*self.learn_length);

At this stage, self.all_train_args has minimum value 1.0 and maximum value 1.0. Could you either commit your modified utils module, or let me know how to fix this?

Thanks!