refactor NN graph building (included in #43)

[naspert]

No description provided.

[mdeff]

Thanks for your contribution @naspert :)

This refactoring was much needed and I like it. Note that we are still in early development and so we don't care too much about breaking the API (e.g. for the distances' names).

Some general concerns:

• Shouldn't 'scipy-pdist' be the default? For sure the default should be from scipy (so users won't have to install any additional library), but I wonder how good the scipy KDTree is. I tried the LSHForest once and it wasn't great...
• If you have some intuitions about which backend to use when, it would be nice to share it in the documentation. ;)
• Travis is failing, please fix it.
• For now the tests are simply running the code, without any check. (As it was before I know...) It would be great to check the approximation error in the tests, with 'scipy-pdist' as the ground truth.
• While refactoring, I would use the opportunity to rename NNtype to type, dist_type to metric, symmetrize_type to symmetrization for consistency.
• The knn and radius branches should certainly share more code. At least the computation of the Gaussian kernel should be factored out.

Thanks again, and feel free to disagree with the above. ;)

[mdeff]

you miss a space after the colon

[coveralls]

Coverage increased (+0.6%) to 82.579% when pulling a562896 on naspert:nn_refactor into e696d9f on epfl-lts2:master.

[mdeff]

Thanks for merging the knn and radius matrix build 👍

[bricaud]

This might be an interesting alternative to FLANN:
https://github.com/spotify/annoy

[mdeff]

Thanks @bricaud :) There's actually a whole lot of libraries, see e.g. this benchmark. We currently support the following backends (thanks to @naspert work):

• scipy pdist (brute force), KDTree and cKDTree, because scipy is a hard requirement and will be installed for all users
• nmslib, because it's the fastest according to the benchmark
• flann, because it was there before and people know it

@bricaud: which kNN libraries do you have experience with? Would you recommend any?
@naspert: do you see any reason to use KDTree over cKDTree?

[naspert]

cKDTree is the same as kdtree, with a C backend. It generates the same results, much faster so no good reason to prefer kdtree over ckdtree. If you are looking into having a dataset with many dimensions, nmslib will be faster.