Add support for a second weight that will be used as bias to the Convolution layer.

If clippy runs into docstring errors then it will report those errors and bail. This might give the impression that the code is pretty sound since the only issues are docstring things. Well, that wouldn't be correct! If we fix these 36 docstring issues then a cascade of >450 errors on the actual code is the result.

For the lints that refer to a wiki page we can use the following grep:

$ cargo build --features=clippy 2>&1 | tee build.log
$ cat build.log | grep http | cut -f2 -d'#' | sort | uniq -c
   1 assign_op_pattern
  31 cast_possible_truncation
   1 cast_possible_wrap
   8 cast_precision_loss
   2 cast_sign_loss
   1 explicit_iter_loop
 116 indexing_slicing
   3 map_clone
   2 match_bool
   3 match_ref_pats
   1 match_same_arms
   2 mut_mut
   7 needless_borrow
  37 option_unwrap_used
 194 result_unwrap_used
   2 shadow_reuse
   1 shadow_unrelated
   7 single_match_else
   4 too_many_arguments
   9 toplevel_ref_arg
  10 type_complexity
  13 use_debug
   3 used_underscore_binding
   3 useless_vec

This is 461 errors, but there are 481 errors in all. The ones that don't refer to a page are mostly about unneeded use of & when calling functions and using match on boolean expressions (clippy prefers if/else).

Obviously not all of these need to fixed in a humungous PR, but what should be done?

FWIW, here's a patch that fixes the docstring issues:
docstring.patch.zip

Patch is a zip since github doesn't want any files with .patch as a suffix. 😞

NB: this required updating clippy to 0.0.67 in Cargo.toml.

At the moment the Network stores the layers and metadata about the layers in two different arrays with matching indices. This is Caffe legacy.

We would like to take the metadata about the layer into the layer struct as this cleanes up the structure and makes the scope of layers clearer.

What do you think about removing ArcLocks from Leaf codebase? There is a lot of .read().unwrap(), Arc::new(RwLock::new(..)) and then let inputs_locked: Vec<_> = inputs.iter().map(|b| b.read().unwrap()).collect() that make code harder to read. Users of Leaf also have to wrap inputs in ArcLocks.

I think when autumnai/collenchyma#37 is implemented, there would be less problems with borrowing, and ArcLocks can be removed (but I'm not completely sure yet).

Or are there plans to introduce multithreading that require all inputs/outputs/weights to be wrapped in this way?

This issue was automatically generated. Feel free to close without ceremony if
you do not agree with re-licensing or if it is not possible for other reasons.
Respond to @cmr with any questions or concerns, or pop over to
#rust-offtopic on IRC to discuss.

You're receiving this because someone (perhaps the project maintainer)
published a crates.io package with the license as "MIT" xor "Apache-2.0" and
the repository field pointing here.

TL;DR the Rust ecosystem is largely Apache-2.0. Being available under that
license is good for interoperation. The MIT license as an add-on can be nice
for GPLv2 projects to use your code.

Why?

The MIT license requires reproducing countless copies of the same copyright
header with different names in the copyright field, for every MIT library in
use. The Apache license does not have this drawback. However, this is not the
primary motivation for me creating these issues. The Apache license also has
protections from patent trolls and an explicit contribution licensing clause.
However, the Apache license is incompatible with GPLv2. This is why Rust is
dual-licensed as MIT/Apache (the "primary" license being Apache, MIT only for
GPLv2 compat), and doing so would be wise for this project. This also makes
this crate suitable for inclusion and unrestricted sharing in the Rust
standard distribution and other projects using dual MIT/Apache, such as my
personal ulterior motive, the Robigalia project.

Some ask, "Does this really apply to binary redistributions? Does MIT really
require reproducing the whole thing?" I'm not a lawyer, and I can't give legal
advice, but some Google Android apps include open source attributions using
this interpretation. Others also agree with
it.
But, again, the copyright notice redistribution is not the primary motivation
for the dual-licensing. It's stronger protections to licensees and better
interoperation with the wider Rust ecosystem.

How?

To do this, get explicit approval from each contributor of copyrightable work
(as not all contributions qualify for copyright, due to not being a "creative
work", e.g. a typo fix) and then add the following to your README:

## License

Licensed under either of

 * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
 * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)

at your option.

### Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any
additional terms or conditions.

and in your license headers, if you have them, use the following boilerplate
(based on that used in Rust):

// Copyright 2016 leaf developers
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.

It's commonly asked whether license headers are required. I'm not comfortable
making an official recommendation either way, but the Apache license
recommends it in their appendix on how to use the license.

Be sure to add the relevant LICENSE-{MIT,APACHE} files. You can copy these
from the Rust repo for a plain-text
version.

And don't forget to update the license metadata in your Cargo.toml to:

license = "MIT/Apache-2.0"

I'll be going through projects which agree to be relicensed and have approval
by the necessary contributors and doing this changes, so feel free to leave
the heavy lifting to me!

Contributor checkoff

To agree to relicensing, comment with :

I license past and future contributions under the dual MIT/Apache-2.0 license, allowing licensees to chose either at their option.

Or, if you're a contributor, you can check the box in this repo next to your
name. My scripts will pick this exact phrase up and check your checkbox, but
I'll come through and manually review this issue later as well.

• @hobofan
• @MichaelHirn
• @homu
• @kschmit90
• @killercup

As pointed out in #79, the link to the contribution guidelines is broken since it links to the develop branch.

The correct link should be:

https://github.com/autumnai/leaf/blob/master/CONTRIBUTING.md#git-commit-guidelines

Recently multiple issues were created suggesting relicensing of multiple projects related to Leaf to a dual MIT/Apache-2.0 license.

@MichaelHirn and I discussed the issue and are both in favor of relicensing the concerned projects, however we don't inted to include license headers for the Apache-2.0 license in every file.

List of the issues for reference, and status of signatures collected:

• #38
• autumnai/cuticula#20
• autumnai/collenchyma#34
• autumnai/collenchyma-blas#4
• autumnai/collenchyma-nn#10
• autumnai/rust-cudnn#10

We will also have to ask permission to relicense for any contributors of outstanding PRs before merging them.

The following layers have a high priority:

• Convolution
• Inner Product (FullyConnected)
• Recurrent / Long Short-Term Memory #73

Other common layers that are interesting for implementation:

• Pooling
• Dropout

To provide state-of-the-art performance for training and running DNNs and comparing benchmarks for the first time with other Deep Learning Frameworks, we will need GPU support.

A not exhaustive list of actions I am about to take in order to make this happen.

• Digesting more information about GPU programming
• Preparing Phloem for GPU/CPU support with Arrayfire.
• abstracting mathematical operations from Leaf and putting it all back together.

From this post:

Which is why Max and I will suspend the development of Leaf and focus on new ventures.

Is this accurate? If so, should the README mention so?

Autumn is also still listed on http://rust-lang.org/friends.html, is Autumn still a thing? If so, will the company continue to use Rust?

After cloning the repository, running the example (cargo run --release --example benchmarks) is failing at the point of compiling leaf. I had installed openssl via brew and had updated gcc as well....

yangtheman:~/projects/leaf(master)$ cargo clean
yangtheman:~/projects/leaf(master)$ cargo run --release --example benchmarks
   Compiling libc v0.2.8
   Compiling winapi v0.2.5
   Compiling rustc-serialize v0.3.18
   Compiling bitflags v0.3.3
   Compiling utf8-ranges v0.1.3
   Compiling winapi-build v0.1.1
   Compiling regex-syntax v0.2.5
   Compiling log v0.3.5
   Compiling kernel32-sys v0.2.1
   Compiling memchr v0.1.10
   Compiling time v0.1.34
   Compiling rand v0.3.14
   Compiling timeit v0.1.2
   Compiling byteorder v0.4.2
   Compiling rblas v0.0.11
   Compiling linear-map v0.0.4
   Compiling pkg-config v0.3.8
   Compiling lazy_static v0.1.15
   Compiling aho-corasick v0.5.1
   Compiling cudnn-sys v0.0.3
   Compiling cublas-sys v0.1.0
   Compiling regex v0.1.55
   Compiling cublas v0.2.0
   Compiling cudnn v1.3.1
   Compiling num v0.1.31
   Compiling env_logger v0.3.2
   Compiling enum_primitive v0.1.0
   Compiling collenchyma v0.0.8
   Compiling collenchyma-nn v0.3.4
   Compiling collenchyma-blas v0.2.0
   Compiling leaf v0.2.0 (file:///Users/yangtheman/projects/leaf)
error: linking with `cc` failed: exit code: 1
note: "cc" "-m64" "-L" "/usr/local/lib/rustlib/x86_64-apple-darwin/lib" "/Users/yangtheman/projects/leaf/target/release/examples/benchmarks.0.o" "-o" "/Users/yangtheman/projects/leaf/target/release/examples/benchmarks" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/Users/yangtheman/projects/leaf/target/release" "-L" "/Users/yangtheman/projects/leaf/target/release/deps" "-L" "/usr/local/lib/rustlib/x86_64-apple-darwin/lib" "/Users/yangtheman/projects/leaf/target/release/deps/libenv_logger-2fedde90a22290a6.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libregex-ea979451af017568.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libutf8_ranges-afda04de2a52d0c6.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libaho_corasick-eea45523c8483aac.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libmemchr-545f7c18e4fb5174.rlib" "/Users/yangtheman/projects/leaf/target/release/libleaf.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libtimeit-32b1006bf5998cae.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libtime-22c21fe32894ddad.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libcollenchyma_blas-a8b9d5dd0a4541d1.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/librblas-f688441f4b40cb58.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libcublas-75b68f8b15390ba0.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libcublas_sys-c067fced6fd7f628.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libregex_syntax-46f1c6091c03f7ce.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libcollenchyma_nn-f77eb8f370d7400a.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/liblog-0b1f2bad29d31333.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libcollenchyma-9c720fde0214e273.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libenum_primitive-2dc0d7f5a40461af.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/liblinear_map-9d19437a952d430c.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/liblazy_static-007034d2ad8108ce.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libnum-d3b232b2ec3de579.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/librand-6fab17fc670a2bc8.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libbyteorder-bff578a6925be5fe.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/librustc_serialize-79a17eda1cd94e46.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libcudnn-854d347fd7afab90.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libcudnn_sys-9f4c28c0b836a8f7.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/liblibc-c1044b0a546bbfd6.rlib" "/Users/yangtheman/projects/leaf/target/release/deps/libbitflags-646076c1f4684754.rlib" "/usr/local/lib/rustlib/x86_64-apple-darwin/lib/libstd-6a154fe0.rlib" "/usr/local/lib/rustlib/x86_64-apple-darwin/lib/libcollections-6a154fe0.rlib" "/usr/local/lib/rustlib/x86_64-apple-darwin/lib/librustc_unicode-6a154fe0.rlib" "/usr/local/lib/rustlib/x86_64-apple-darwin/lib/librand-6a154fe0.rlib" "/usr/local/lib/rustlib/x86_64-apple-darwin/lib/liballoc-6a154fe0.rlib" "/usr/local/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-6a154fe0.rlib" "/usr/local/lib/rustlib/x86_64-apple-darwin/lib/liblibc-6a154fe0.rlib" "/usr/local/lib/rustlib/x86_64-apple-darwin/lib/libcore-6a154fe0.rlib" "-l" "blas" "-l" "cublas" "-l" "cublas" "-l" "OpenCL" "-l" "cuda" "-l" "cudnn" "-l" "System" "-l" "pthread" "-l" "c" "-l" "m" "-l" "compiler-rt"
note: ld: library not found for -lcublas
clang: error: linker command failed with exit code 1 (use -v to see invocation)

error: aborting due to previous error
Could not compile `leaf`.

To learn more, run the command again with --verbose.
yangtheman:~/projects/leaf(master)$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.3.0
Thread model: posix

The only way for the moment to create Networks is by specifying a new Network with a NetworkConfig, which means the Network gets initialized from scratch.

Resuming to already trained models and keeping the learning state-less with loading from raw weights would open new usage scenarios.

We are looking forward to use Cap'n Proto for serialization, as it allows for state-of-the-art performance.

I am all new to rust so I apologize if this is useless:

# rustc -V
rustc 1.7.0-beta.3 (36237fc61 2016-02-11)

I tried this with beta 1.7 and stable 1.6, both yield the very same result

# cargo build
   Compiling byteorder v0.4.2
   Compiling rustc-serialize v0.3.18
   Compiling libc v0.1.12
   Compiling bitflags v0.3.3
   Compiling libc v0.2.7
   Compiling rblas v0.0.10
/home/bernhard/.cargo/registry/src/github.com-88ac128001ac3a9a/libc-0.1.12/rust/src/liblibc/lib.rs:81:21: 81:39 warning: lint raw_pointer_derive has been removed: using derive with raw pointers is ok
/home/bernhard/.cargo/registry/src/github.com-88ac128001ac3a9a/libc-0.1.12/rust/src/liblibc/lib.rs:81 #![allow(bad_style, raw_pointer_derive)]
                                                                                                                          ^~~~~~~~~~~~~~~~~~
   Compiling rand v0.3.14
   Compiling log v0.3.5
   Compiling num v0.1.31
   Compiling enum_primitive v0.1.0
/home/bernhard/.cargo/registry/src/github.com-88ac128001ac3a9a/rblas-0.0.10/src/lib.rs:4:1: 4:27 error: #[feature] may not be used on the beta release channel
/home/bernhard/.cargo/registry/src/github.com-88ac128001ac3a9a/rblas-0.0.10/src/lib.rs:4 #![feature(concat_idents)]
                                                                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to previous error
Could not compile `rblas`.

To learn more, run the command again with --verbose.

Is this a known issue?

I'm trying to fiddle with leaf-examples using the latest source; with the fix for history tensors.

The main error I'm getting right now is:

`ConvolutionConfig` does not name a structure
...
`PoolingConfig` does not name a structure

It looks like this is because the module exports for leaf::layers::common::convolution and leaf::layers::common::pooling are feature gated to only be compiled when using cuda without native.

I don't understand what this is about? You'd think you'd always need native if compiling with cuda to be able to sync the memory back and read results?

Need to rename:
\doc\book\http:\autumnai.github.io\leaf\leaf
or else you build a program depending on Leaf 0.2.1 on Windows:

unable to get packages from source

Caused by:
failed to unpack package leaf v0.2.1

Caused by:
failed to create C:\Users\abc\.cargo\registry\src\github.com-88ac128001ac3a9a\leaf-0.2.1\doc\book\http:\autumnai.github.io\leaf\leaf

Caused by:
The filename, directory name, or volume label syntax is incorrect. (os error 123)

The issue is the colon in the name

I came across some inconsistencies in our documentation - especially in its quality. On the one side, we are not documenting the complex parts, which makes it hard to contribute, because one isn't really getting what the code does and on the other side, we are documenting quite obvious things.

A lot of the other Deep/Machine Learning Frameworks aren't really better at documentation, but I think it is quite important to make documentation a num.1 priority, because the underlying technology is not really broadly understood and under heavy change. Meaning that we have to address two personas with our documentation:

• those who want to use Leaf
• those who would like to develop on Leaf

I experienced those problems myself when I was working on the conv layers and think it is quite important to remove the obstacles to contribution and usage due to unsufficient documentation right now - as they become only harder to remove the further on we move.

I will express my thoughts and propose a solution to that problem later in the day.

As @dirvine pointed out (#47 (comment)) an example of a proper commit message would be helpful in the contribution guidelines, since currently it is only explained in an abstract way.

Add support for a second weight that will be used as bias to the Linear layer. (See also #69)

L1 regularization is not a priority since it is not that commonly used and requires more math functions to be implemented.

The convolution layer currently only supports 2D convolutions.

Supporting 3D and higher dimensional convolutions might just require relaxing the num_spatial_dims bounds to work, but there also needs to be some testing for that.

Leaf is currently providing the raw markdown files of the Leaf book, as well as the compiled assets, generated with mdbook. This introduces unnecessary complexity (see e.g. #98) and causes bugs on Windows machines (#100)

As pointed out by @mitchmindtree on #40, Leaf doesn't build under stable, yet. I don't think that any important feature currently prevents us from supporting Rust stable (@hobofan might comment on this in more detail). Supporting Rust stable on Leaf, would increase its accessibility, dramatically.

Hi!

I'm very interested in the ML field (though quite inexperienced). After having watched leaf from afar for a while, I thought I'd clone the repo and have a play!

The first thing that I noticed was that it currently relies on some unstable features. I'm curious: is there a certain set of heavily required unstable features that you're waiting on the stabilisation of before you plan to move to the stable channel? Or do you plan to follow the nightly in general, without minding the use of unstable features?

I'd love to contribute to leaf for a couple reasons: primarily as a way of learning more about ML, but also as I anticipate having a use-case for a strong rust ML library in the future, and from what I can tell, leaf looks like the most promising candidate for this so far 😸 I'm not super interested in tracking nightly rust however (I started using rust around 0.9 and keeping up with the instability across all my projects really took the wind out of me.. haven't been able to bring myself to leave the stable channel since heheh).

It would be great to provide the docker image and Dockerfile for newbie developers.

We can also use that container to run leaf in the cloud much more easily.

I have got build error with rustc 1.12 on debian stretch.

$ RUST_BACKTRACE=1 cargo build
   Compiling leaf v0.2.1 (file:///home/<user>/work/leaf)
   Compiling num-rational v0.1.35
Build failed, waiting for other jobs to finish...
error: failed to run custom build command for `leaf v0.2.1 (file:///home/<user>/work/leaf)`
Process didn't exit successfully: `/home/<user>/work/leaf/target/debug/build/leaf-87f43ad88e5c8bb2/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: Failed, description: "Error while trying to execute `capnp compile`: Failed: No such file or directory (os error 2).  Please verify that version 0.5.2 or higher of the capnp executable is installed on your system. See https://capnproto.org/install.html" }', src/libcore/result.rs:788
stack backtrace:
   1:     0x55e338350a6b - std::sys::backtrace::tracing::imp::write::h00e948915d1e4c72
   2:     0x55e338355ebf - std::panicking::default_hook::_{{closure}}::h7b8a142818383fb8
   3:     0x55e33835511d - std::panicking::default_hook::h41cf296f654245d7
   4:     0x55e3383557d5 - std::panicking::rust_panic_with_hook::h4cbd7ca63ce1aee9
   5:     0x55e338355622 - std::panicking::begin_panic::h93672d0313d5e8e9
   6:     0x55e338355591 - std::panicking::begin_panic_fmt::hd0daa02942245d81
   7:     0x55e338355511 - rust_begin_unwind
   8:     0x55e33838c97f - core::panicking::panic_fmt::hbfc935564d134c1b
   9:     0x55e3382950c2 - core::result::unwrap_failed::h14ae321fa6665b8d
                        at /build/rustc-1.12.0+dfsg1/src/libcore/result.rs:29
  10:     0x55e338294251 - _<core..result..Result<T, E>>::unwrap::hbe110bf6bea72ca6
                        at /build/rustc-1.12.0+dfsg1/src/libcore/result.rs:726
  11:     0x55e338297306 - build_script_build::main::he5071e1b00fcfe0e
                        at /home/<user>/work/leaf/build.rs:4
  12:     0x55e33835d896 - __rust_maybe_catch_panic
  13:     0x55e33835480d - std::rt::lang_start::h53bf99b0829cc03c
  14:     0x55e3382974e3 - main
  15:     0x7fa0f6ae52b0 - __libc_start_main
  16:     0x55e3382935e9 - _start
  17:                0x0 - <unknown>

What should I see to make this work?

Most important:

• Momentum

To achieve feature parity with Caffe we need to implement the following solvers:

• Adagrad
• Adadelta
• Adam
• Nesterov
• RMSprop

I feel like, the entrance barrier for (Rust) Developers to engage with Leaf and Machine Learning, for contributions and hacking-away purposes, is still far too high. Partly because the concepts of Machine Learning (Deep Learning) are not yet widely known and partly because not many are familiar with the general design of a Machine Learning framework - compared e.g. to the general design of a Web framework.

The Leaf Book, should provide a practical introduction to Deep Learning for developers. Explain the easy Leaf API and provide examples for popular use-cases like adding a new Layer, Machine Learning across multiple devices and co. After reading it, a developer should feel comfortable hacking on Leaf, even if she has no prior knowledge about Deep Learning ( Deep Learning is really easy).

@hobofan pointed out in #45 interactive documentation for Layers. I am not sure to what extent they can be provided here with the Leaf Book. For the interactive layer documentation, I have something with Jupyter in mind, which would require a Rust kernel first, though. But other options for interactive layer documentation are welcome.

For the book I am trying mdBook as it gives a nice layout and allows it to place the book inside the leaf project. Feedback on the choice is welcome.

Hi All,
I am new here. I have cuda and cuda nn installed with paths setup. I cannot run the example files for some reason.
Am I missing something?
System is Ubuntu 16.04
`akhileshsk@akhileshsk-home:$ cd rust_leaf_tutorials/leaf
akhileshsk@akhileshsk-home:/rust_leaf_tutorials/leaf$ cargo run --release --no-default-features --features cuda --example benchmarks alexnet
Compiling linear-map v0.0.4
Compiling num-traits v0.1.37
Compiling pkg-config v0.3.9
Compiling winapi v0.2.8
Compiling byteorder v0.4.2
Compiling lazy_static v0.1.16
Compiling rustc-serialize v0.3.24
Compiling utf8-ranges v0.1.3
Compiling libc v0.2.23

Compiling log v0.3.7
Compiling rblas v0.0.11
Compiling capnp v0.6.2
Compiling winapi-build v0.1.1
Compiling regex-syntax v0.3.9
Compiling bitflags v0.3.3
Compiling num-integer v0.1.34
Compiling enum_primitive v0.1.1
Compiling rand v0.3.15
Compiling kernel32-sys v0.2.2
Compiling time v0.1.37
Compiling memchr v0.1.11
Compiling num-iter v0.1.33
Compiling cublas-sys v0.1.0
Compiling cudnn-sys v0.0.3
Compiling thread-id v2.0.0
Compiling thread_local v0.2.7
Compiling aho-corasick v0.5.3
Compiling cudnn v1.3.1
Compiling cublas v0.2.0
Compiling timeit v0.1.2
Compiling capnpc v0.6.2
Compiling num-bigint v0.1.37
Compiling num-complex v0.1.37
Compiling regex v0.1.80
Compiling num-rational v0.1.36
Compiling num v0.1.37
Compiling collenchyma v0.0.8
Compiling collenchyma-blas v0.2.0
Compiling collenchyma-nn v0.3.4
Compiling leaf v0.2.1 (file:///home/akhileshsk/rust_leaf_tutorials/leaf)
Compiling leaf v0.2.1
error[E0004]: non-exhaustive patterns: &mut Cuda(_) not covered
--> /home/akhileshsk/.cargo/registry/src/github.com-1ecc6299db9ec823/leaf-0.2.1/src/util.rs:28:11
|
28 | match mem {
| ^^^ pattern &mut Cuda(_) not covered

error[E0004]: non-exhaustive patterns: &Cuda(_) not covered
--> /home/akhileshsk/.cargo/registry/src/github.com-1ecc6299db9ec823/leaf-0.2.1/src/solvers/mod.rs:77:24
|
77 | match result.get(native.device()).unwrap() {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ pattern &Cuda(_) not covered

error: aborting due to 2 previous errors

error: Could not compile leaf.
Build failed, waiting for other jobs to finish...
error: build failed
akhileshsk@akhileshsk-home:/rust_leaf_tutorials/leaf$
akhileshsk@akhileshsk-home:/rust_leaf_tutorials/leaf$ ^C
akhileshsk@akhileshsk-home:~/rust_leaf_tutorials/leaf$ `

 cargo build --features=opencl
   Compiling leaf v0.2.1
   Compiling collenchyma v0.0.8
   Compiling collenchyma-nn v0.3.4
   Compiling collenchyma-blas v0.2.0
/home/bernhard/.cargo/registry/src/github.com-88ac128001ac3a9a/leaf-0.2.1/src/layers/container/sequential.rs:8:21: 8:29 error: unresolved import `util::LayerOps`. There is no `LayerOps` in `util` [E0432]
/home/bernhard/.cargo/registry/src/github.com-88ac128001ac3a9a/leaf-0.2.1/src/layers/container/sequential.rs:8 use util::{ArcLock, LayerOps};
# yada yada

Is this due to a limitation that only cuda supports something or is this a different issue?

Save the time I wasted; Yes, the potential for AI in Rust is far beyond the potential for it in Python.
I understand that much, Unfortunately, the authors stopped and gave up on the infrastructure;
and without so much as mentioning it on the repository.

Alas I had to find out here.. after weeks of wondering wtf, why doesn't this work in any environment.
https://www.bountysource.com/teams/autumnai/issues?tracker_ids=24995999

Apparently important project updates, go on blogs now, and not the project README's..
https://medium.com/@mjhirn/tensorflow-wins-89b78b29aafb

Maybe someone update the repo and call it abandonware.

Maybe someone can figure out why no one can get past the errors:
: undefined reference to cudnnSetConvolutionNdDescriptor_v3' : undefined reference to cudnnConvolutionBackwardFilter_v3'
: undefined reference to `cudnnConvolutionBackwardData_v3'

Maybe someone whose interested in the technology could fork it.
I dunno, I find it really frustrating.. (╯°□°）╯︵ ┻━┻

As part of our network glossary we will want to have a section explaining the basics of feedforward and backpropagation through the network.

The following activation layers have a high priority:

• Sigmoid
• ReLU

Other activation layers that are interesting for implementation:

• TanH

Hi leaf folks,

I'm wondering what's the best way for someone new to the project to get involved and start contributing? Are you looking for new contributors? Are there some low hanging fruits I can try and tackle? For context, I am somewhere between a beginner and intermediate with regards to both Rust and ML.

This issue makes part of #20 more concrete.

Recurrent Neural Networks, became an effective Neural Network architecture, that we would like to implement in Leaf as well. The operations could probably be composed of BLAS operations with collenchyma-blas. Differentiation for backpropagation is another topic, that might better be solved with Auto Differentiation.

Hi, all
I try to compile and run leaf example with the following command,
cargo run --release --no-default-features --features cuda --example benchmarks alexnet
but got errors like this

In function `convolution_descriptor::ConvolutionDescriptor::new::hbe3406e228523108khb':
cudnn.0.rs:(.text._ZN22convolution_descriptor21ConvolutionDescriptor3new20hbe3406e228523108khbE+0x29c): undefined reference to `cudnnSetConvolutionNdDescriptor_v3'
cudnn.0.rs:(.text._ZN22convolution_descriptor21ConvolutionDescriptor3new20hbe3406e228523108khbE+0x346): undefined reference to `cudnnSetConvolutionNdDescriptor_v3'
cudnn.0.rs:(.text._ZN22convolution_descriptor21ConvolutionDescriptor3new20hbe3406e228523108khbE+0x3a2): undefined reference to `cudnnSetConvolutionNdDescriptor_v3'

I am using rustc 1.8.0 with cargo 0.10.0.

please give me some hints.

thanks

Currently weight updates are calculated on Native backend. Profiling shows that about 40% of CPU time is spent doing corresponding BLAS operations. Another 40% are in an area without debug info, quite likely that's nvidia driver doing i/o. In the same time according to nvidia-smi GPU load is about 20% even on my relatively slow GTX 960.

I think it's possible to get 3x-5x speedup if weight updates are implemented on GPU. It should be quite easy since update is a simple BLAS operation y = a * x + b * y where a and b are scalars, x and y are tensors of equal dimensions.

For using convolutional networks, we need some more layer types

• Convolutional Layer
• ReLu Layer
• Pooling Layer
• Dropout Layer

Rust "impl specialization" should land Rust 1.9(?) (PR here: rust-lang/rust#30652).

That feature should hopefully allow us to better determine the capabilities of the different frameworks that are compiled in and use more performant operations if possible. Currently we handle this via ugly feature attributes which rely on our knowledge of the implemented operations rather than the type system. The current system of feature flags also requires us to "dumb down" the backends to the capabilities they all have in common in order to retain the backend portability.

When naively trying to build an example project, one gets this:

thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: Failed, description: "Error while trying to execute `capnp compile`: Failed: No such file or directory (os error 2).  Please verify that version 0.5.2 or higher of the capnp executable is installed on your system. See https://capnproto.org/install.html" }', src/libcore/result.rs:746
note: Run with `RUST_BACKTRACE=1` for a backtrace.

You should mention that you require capnproto to be installed in the system beforehand.

The following loss layers have a high priority:

• Euclidean
• SoftMax

Other loss layers that are interesting for implementation:

• Hinge
• Sigmoid Cross Entropy

The most common learning policies (Fixed, Step, Exp) were implemented with edfbbdf.

We still need:

• Multistep (also requires additional SolverConfig param)
• Inv
• Poly
• Sigmoid

We want to have as unambigous naming as possible.
Porting code from Caffe also brought some ambigous naming with it, especially around the word "parameter"/"param" which to make things worse already has a meaning in programming languages.

Where "parameter"/"param" is used to mean the synaptic weight, we use weight.

Where "parameter/param" is used to mean "hyper-paramter" we use config (e.g. SolverParameter -> SolverConfig).

This issue should be closed once we finish porting code from Caffe.

Hi guys, I'm a little confused. I'm a beginner in neural networks but some docs/library use neuron/preceptron/etc (like http://synaptic.juancazala.com/ ) some using layer/solver (leaf). What's the difference? Can I write a feedforward network in leaf? (like http://synaptic.juancazala.com/#/paint-an-image )

In order to load a network with weights from file, we also would need to save one to a file.

referencing: #14

I also noticed that the collenchyma crate version seemed to be a little behind - I figure you must be busy with other work at the moment, but I just thought I'd ask what your plans are for updating. Are you currently waiting on developments upstream? Or has this simply not been updated yet due to lack of time?

Caffe makes a great case for using layers to module Networks. Those layers - which have different functionality and behaviors are grouped into the same namespace. To help development, understanding, code-structure and documentation, we will provide a clear module for those layers.

This is basically a restructuring tasks of files and modules.