vab9 / pir-project2017 Goto Github PK

Surprise, it's sometimes even getting more stupid in each iteration. There are a number of things that don't quite add up:

• Why do we so often have all test data sets correctly identified right at the beginning with random weights and biases?
• When we do have a value between 0 and the test set size why does it often get lower than the original recognition rate only to go slowly back to the original value? Why does it reapproach the original value at all?
• Why does it seem to never exceed the original recognition rate?
• The activation levels seem to be extraordinarily high, suspiciously so.

Need to verify that

• evaluate actually does what it's supposed to do
• check nablas
• check weights and biases and how they change over the different epochs
• check with the python code

Reads the data set (for example from a text file), parses it and returns the Data in a suiting format. It might be necessary to progress the data chunk by chunk.

saves the networks matrix.

• serialisation
• most likely possible with some crate

blas-sys is a BLAS implementation in Rust that uses cblas.
blas-sys isn't documented properly so i will link the cblas documentation.
https://software.intel.com/en-us/node/520730 //documentation of cblas
https://crates.io/crates/blas-sys //link to the crate
needs probably build-essential and gfortran packages installed

@johann162 already did some work on that and tried to test the library.
Also See #2
We will need this rather sooner than later, would be great if we could work this out quickly.

We need a Struct to simplify the use of the datasets for the NN learning
#22

Meaning how to store the neurons and data. What does the actual net need to look like?

use clap to parse arguments from commandline

• Prepare Backpropagation which will use Gradient Descent
• Implement Gradient Descent

Implement easy logging for our Project and put usefull logs in our Project.

Current naming is wrong, rename "classifier" to "class" in the structs thingy

Cargo project should be the root directory, this seems to be standard for projects hosted on crates.io...

Alle einverstanden, wenn ich das jetzt mache, oder wollen wir auf @toschneider 's #9 warten?

Someone but me needs to work through this stuff and check if the code does what it is supposed to do. I got confused in the end with all the lists of vectors of slices of dvectors. Urgent.

(Note: Actual backprop not yet implemented)

Currently everything is cramped in there. At least the flower stuff should be separated in it's own module.

enable travis for this project

Make sure the network can feedforward activation levels of neuron layers.

Maybe something that implements BLAS. Don't know yet what we will need exactly.

It might be that I'm just stupid, but I don't find a way around this: Whenever I want to add or multiply two vectors or matrices or whatever using nalgebra it tells me that the ops are not defined to work on references (mind you, errors are being thrown for right-hand-side arguments).

Since I obviously can't take ownership each time and then return it I have used a clone() call everywhere where this is an issue. This is bad, and I guess it will come back and bite us when we try to let the network learn (performance).

This needs to go rather sooner than later. I'm rather certain that I just overlooked something obvious or forgot-how-to-rust because otherwhise nalgebra wouldn't be usable.

As of now we can work with blas_sys (yay!) but we need to install a c-compiler and gfortran before we do that (boho:( ). So It would be good if someone could work a way around that or at least put the necessary stuff into a script so that installation works automated.

As discussed in #13 we should switch to this blas crate. The transition should not be too hard (especially since we didn't really use it yet), only the parameter types need to be adjusted.

Need to update the wiki and our example (probably @toschneider will have to do that), then this issue can be closed, wrote it mainly for visibility.

TODO

Probably done by experimenting once everything else is good to go

The net, layers, and related functions (everything you would write into it's impl block)

• Solve #3
• Solve #6

All the configuration for the learning progress (nn-layout, learning rate, ...) will be stored in a separate file which we will need to parse. Suggestion: Use XML and find a crate that does all the work for you.