manzilzaheer / covertree Goto Github PK
View Code? Open in Web Editor NEWCover Tree implementation in C++ for k-Nearest Neighbours and range search
License: Apache License 2.0
Cover Tree implementation in C++ for k-Nearest Neighbours and range search
License: Apache License 2.0
Hello, I am finding that the nearestNeighbour
queries are not giving the correct result.
In the attached test case, i have three 2D points in the cover-tree, and for two test points, I do get the correct kNearestneighbours
(k=3
) with correct distances. But the nearest neighbor is incorrect.
I will try to look into the code in detail to figure out the cause. But if you have any ideas as to why this could be happening, I would appreciate the fix for this.
The attached program produces the following output.
Number of OpenMP threads: 1
adding 3 points to cover tree!
Entered case 1: 3.31402 1 0
Requesting global lock!
testing for nearest neighbors!
query point 0 : (-0.944485 0.116473)
doing 3-nearest neighbors using direct computation!
nearest 0 : 1, 1.438655
nearest 1 : 2, 1.651407
nearest 2 : 0, 2.003489
doing 3-nearest neighbors using cover_tree!
cover_tree 0 : 2, 1.438655
cover_tree 1 : 0, 1.651407
cover_tree 2 : 0, 2.003489
nearest :: direct = (1, 1.438655) cover_tree = (0, 1.651407)
---------------------------------------------------------------------> mismatch
query point 1 : (-0.931471 0.781848)
doing 3-nearest neighbors using direct computation!
nearest 0 : 1, 1.537557
nearest 1 : 2, 2.114969
nearest 2 : 0, 2.256513
doing 3-nearest neighbors using cover_tree!
cover_tree 0 : 2, 1.537557
cover_tree 1 : 0, 2.114969
cover_tree 2 : 0, 2.256513
nearest :: direct = (1, 1.537557) cover_tree = (0, 2.114969)
---------------------------------------------------------------------> mismatch
Hi,
I try to use this demo to have a test, and I run like this:
dist/cover_tree data/train_100d_1000k_1000.dat data/test_100d_1000k_10.dat
data/train_100d_1000k_1000.dat
data/test_100d_1000k_10.dat
Number of OpenMP threads: 8
Number of points: 1000000
Number of dims : 100
56.5687 85.2284 -26.9832
Build time: 62972
Number of OpenMP threads: 8
Number of points: 10000
Number of dims : 100
99.7609 -40.0263 99.2302
Querying serially
Quering parallely
k-NN serially
range serially
Query time: 4394231
sh: 1: pause: not found
How to check the output of the covertree? e.g. indices distances
Thank you.
When will support for removal of points become available?
I cloned the repository and installed the python module using setup.py.
When I tried to import the library in a script using
import covertree
I get the following
File "../knn/agents.py", line 3, in
import covertree
File "/home/joe/anaconda3/envs/knnenv/lib/python2.7/site-packages/covertree/init.py", line 1, in
from covertree import CoverTree
File "/home/joe/anaconda3/envs/knnenv/lib/python2.7/site-packages/covertree/covertree.py", line 1, in
import covertreec
ImportError: /home/joe/anaconda3/envs/knnenv/lib/python2.7/site-packages/covertreec.so: undefined symbol: _ZTINSt6thread6_StateE
I'm getting a segfault when using the Python API (very grateful for that addition by the way =D), am I using it correctly? The API reference PDF doesn't list the constructor. I'm using gcc/g++7.1.0 and Python 2.7, see the gdb trace below.
>>> ct = CoverTree.from_matrix(document_embeddings)
Faster Cover Tree with base 1.3
Max distance: 65.5565
Scale chosen: 18
100% [==================================================]
[New Thread 0x7ffff13fe700 (LWP 11853)]
[New Thread 0x7fffe8bfd700 (LWP 11854)]
Duplicate entry!!!Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe8bfd700 (LWP 11854)]
run (src=..., dst=...) at lib/Eigen/src/Core/Assign.h:410
410 dst.template copyPacket<Derived2, dstAlignment, srcAlignment>(index, src);
I found that in bool CoverTree::insert(const pointType& p)
,
several statement about setting the maxdistUB have been commented. And I wonder how does it work without setting the maxdistUB properly.
you hardcode euclidean distance and Eigen::VectorXd
points. i would prefer to have the options of
i.e. iโd like to have
template<Point>
class CoverTree {
...
}
with Point
needing to specify distance
and operator==
, e.g.:
template<class Distance, class Vector>
class IndexedPoint {
private:
Vector _vec;
size_t _idx;
public:
IndexedPoint(Vector v, size_t i) : _vec(v), _idx(i) {}
const Vector& vec() const { return this->_vec; }
size_t idx() const { return this->_idx; }
bool operator==(const IndexedPoint<Distance>& p) {
return is_true(all(this->_vec == p.vec()));
};
double distance(const IndexedPoint<Distance>& p) const {
return Distance::distance(*this, p);
};
};
class CosineDistance {
public:
static double distance(const IndexedPoint<CosineDistance>& p1, const IndexedPoint<CosineDistance>& p2) {
return 1 - cor(p1.vec(), p2.vec());
}
};
class EuclideanDistance { ... }
then i can do:
CoverTree<IndexedPoint<EuclideanDistance>> ct;
Hey, can you share the analysis for query time as value of K increases. We tried K=200 but it's taking
long time to query. Test points are 21 million.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.