frjnn / bhtsne Goto Github PK

Dear Francesco,

Thanks for this implementation. I assume it is faster than the C++ original version right considering the rayon parallel efficiency. I will have some tests later on but just want to know whether you have some benchmarks.

Thanks,

Jianshu

I just saw your post on Reddit, awesome work! I'm the maintainer of linfa and thought about implementing t-SNE as a transformative dimensionality reduction technique in the past, but never came to it. This crate can take off a lot of work for us. We would implement a wrapper which adepts your algorithm by:

• implementing builder style pattern for configuration
• using datasets for input/output
• implementing transform trait for the algorithm

Sounds good? I just quickly glanced at the source code and three things stood out which could be improved:

• make csv dependency optional, sometimes it's not necessary to pull that in
• make algorithm generic for num_traits::Float
• what is about error handling? Can any part of your algorithm fail, especially what happens if there are NaNs in your data or parameters are mis-configured (e.g. perplexity negative)

When I run cargo test for the first time I get the following:

running 10 tests
test test::set_embedding_dim ... ok
test test::set_epochs ... ok
test test::exact_tsne ... FAILED
test test::set_final_momentum ... ok
test test::set_learning_rate ... ok
test test::set_momentum ... ok
test test::set_momentum_switch_epoch ... ok
test test::set_perplexity ... ok
test test::set_stop_lying_epoch ... ok
test test::barnes_hut_tsne ... FAILED

failures:

---- test::exact_tsne stdout ----
thread 'test::exact_tsne' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/test.rs:77:87

---- test::barnes_hut_tsne stdout ----
thread 'test::barnes_hut_tsne' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/test.rs:107:87
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    test::barnes_hut_tsne
    test::exact_tsne

It looks like the tests expect several CSV data files, but they're not there.

In the near future we'll use rayon's parallel routines to perform the embarrassingly parallel parts of the computation.

It would be nice to add the possibility to run the algorithm with custom user-defined metrics.

First, thanks so much for providing this crate! I've thought about implementing t-SNE in Rust before, but never took on the challenge. Kudos!

I think you have a typo in one of your function names; bhtsne::wite_csv should probably be write_csv. Just wanted to let you know.

Thanks, again!