Investigate into Fixed Size Arrays / dim parameterization about nearestneighbors.jl

[KristofferC]

Some basic testing on my laptop shows that hardcoding the loop count for dim = 3 is about 2 times faster than the general function.

@code_llvm also shows that the loop is unrolled.

So this is something that probably should be done. Not sure how to create the trees in a type stable way though...

[KristofferC]

Although on full knn search the timing is kind of irrelevant. It looks like we are bottled necked by memory accesses.

[c42f]

Perhaps one way to do this if you're sticking with matrices would be to pass the dimension along as either N (for large N) or Val{N} for very small N. With the dimension encoded in a type rather than a value, the compiler might have a chance of inlining it and unrolling the loop. I'm not sure if that's a good idea or it's trying to write C++ in julia. This would force things like knn_kernel! to be compiled once for each fixed dimension, probably ok if the Val wrapping is restricted to very small N.

Does the memory access bottleneck depend on the spatial access pattern? It's probably possible to arrange for spatially coherent queries in a lot of cases (though that's up to the user).

[KristofferC]

You are right that this would cause a recompile for all runs with different dimensional data. However, how often is it that one needs to run a lot of queries on data with widely different dimensions? My expectation is that one wants to do a lot of run on a large data set of a fixed dimension and want that to run as fast as possible.

There are two ways to do it:

• Force the user to give the dimension as a Val{N}. This would make things type stable but be awakward for the user.
• Just read of the dimension of the input data and create the correctly parameterized tree. This would be type unstable but I'm not sure how much it matters since unless one is querying with very mall data sets the overhead from the instability should be insignificant.

[c42f]

Yes, a fixed dimension is certainly my use case.

I wasn't imagining that the tree itself be parameterized on the dimension, simply that knn() would dynamically determine the dimension of the data in the tree, construct a Val{N} from that and pass it through the layers into knn_kernel!(). That should involve a single layer of dynamic dispatch to get from knn() into the guts, and all function calls from then in would be parameterized on the dim type.

[KristofferC]

Ah! That sounds like a much better solution indeed!

[c42f]

Hmm, creating the Val on the fly would probably kill performance unless the compiler optimized out any allocation... but you can probably get around that by just storing the Type{Val{N}} in the tree itself as a generic Any.

Anyway, all of this is a bit academic unless it really improves things :-) It would be unfortunate if the extra layers of dispatch just made the code more obscure but barely faster...

[KristofferC]

Wouldn't the Val be created only once before the loop over all query points?

[c42f]

Yeah, that would probably work. It'd be nice if the single point query interface wasn't penalized too much though.

Come to think of it, your second option seems roughly equivalent to what I'm suggesting, but actually simpler and cleaner to implement. Both should involve a single layer of dynamic dispatch to get into the guts of things. It's just a question of whether it makes sense to parameterize the tree type on the dimension or not.

[KristofferC]

It is possible that for smallish trees that fits in the cache that there could be a noticeable change. Gotta benchmark though zzz... :)

[andyferris]

Hi guys,

I've been thinking about this in the context of StaticArrays. I would really like to use a Vector{SVector{3,Float64}} (for instance) to describe my points, rather than a matrix. I'm wondering if this would make NearestNeighbors a bit faster? Moving to vectors of points, rather than matrices, will really help in generalizing the package.

Perhaps one way to do this if you're sticking with matrices would be to pass the dimension along as either N (for large N) or Val{N} for very small N.

Some ideas: you can either build the dimension directly into the tree parameters, or rely on size() and length() being defined on the type for static arrays (so length(eltype(::Vector{SVector{3,Float64}})) = 3), and use that as a compile-time constant (or in a @generated function if that is faster).

[KristofferC]

I think just putting the vector dimension as a parameter is the easiest. However, parameterizing on the actual vector type is more general and probably better. The type for a nearest neighbor tree could move to something like abstract NNTree{T <: AbstractFloat, P <: Metric, V <: AbstractVector} and then all vectors should be supported.

We then just need to make the choice of what to do when the user gives a set of points as a matrix since I don't want to break that. The easiest would probable be to just add a dependency on StaticArrays and then reinterpret the input matrix A to a Vector{SVector{size(A, 1), T}}(size(A, 2)). I'm not sure howStaticArraysscale with dimensions but when using trees like this you mostly use it indim < 20` and that should be fine.

[KristofferC]

Closed by #29