Comments (9)
Oh it also fails on Ubuntu ppc64el. Here are the logs:
from opentsne.
That is interesting. Unfortunately, I don't know how I could debug this, as I do not have access to these platforms. Are there actually any visual differences when looking at these embeddings? I know this is a test run with no visuals, but have you tried looking at the difference between passing in a precomputed matrix and or letting openTSNE take care of that?
from opentsne.
Hello, I have uploaded the 1.0.0 version in Debian and yes the problem is still present.
https://buildd.debian.org/status/logs.php?pkg=opentsne&ver=1.0.0-1&suite=sid
This is really strange arm 32bit arch are passing the tests. This seems really i386 related...
There is specific code for x86 cpu ?
I removed the notiv flag during the compilation since this is a violation of the Debian policy.
on i386 we do not support SSE instructions.
https://wiki.debian.org/ArchitectureSpecificsMemo#i386-1
Cheers
from opentsne.
This could potentially have something to do with the Annoy library. I believe it has some AVX instructions and such, howver, I really don't know this code at all and wouldn't even know where to begin trying to fix it.
However, it seems that maybe they have taken some care to add i386: https://github.com/spotify/annoy/pull/207/files
So maybe it's not that? I generally tried to copy their setup pipeline into https://github.com/pavlin-policar/openTSNE/blob/master/setup.py as closely as possible, but perhaps I've messed something up?
As I've said, I have no way to reproduce this and I am not familiar with the platform-specific compiler directives, so any help on this would be greatly appreciated.
from opentsne.
Just to clarify: is this the ONLY test that fails? I don't think this rest relies on Annoy: it uses the Iris data which is so small that openTSNE should use exact NN.
from opentsne.
Sorry, I totally missed the context. Only one test is failing! I have no idea what could be causing this.
Could it be due to accumulating rounding errors or something? The actual embedding coordinates do seem to be ranked similarly. Or could it be that scipy computes distances a bit differently to scikit-learn on i386?
from opentsne.
Or could it be that scipy computes distances a bit differently to scikit-learn on i386?
No idea, but it does seem like the problem may be upstream. Would be interesting to compare sklearn NearestNeighbors results on X vs on pdist(X) with metric=precomputed, to see if there is indeed any difference. Is there any way to run this on i386 in a virtual machine? Otherwise, @utkarsh2102 @picca do you want to help debugging this?
from opentsne.
from opentsne.
@picca I was just wondering if you could run the following snippet in your Python environment on i386:
import numpy as np
from sklearn.datasets import load_iris
from sklearn.neighbors import NearestNeighbors
from scipy.spatial.distance import pdist, squareform
X = load_iris()['data']
np.random.seed(42)
X += np.random.randn(*X.shape) / 10
nn = NearestNeighbors(metric="euclidean").fit(X)
distances1, indices1 = nn.kneighbors(n_neighbors=90)
nn = NearestNeighbors(metric="precomputed").fit(squareform(pdist(X)))
distances2, indices2 = nn.kneighbors(n_neighbors=90)
assert(np.allclose(distances1, distances2))
assert(np.all(indices1 == indices2))
In my environment this passes.
from opentsne.
Related Issues (20)
- Failed to install from source HOT 3
- Extend openTSNE to specific purposes HOT 4
- Barnes-Hut optimization with the default learning rate collapses on small datasets HOT 4
- Tests fail: ImportError: attempted relative import with no known parent package HOT 8
- Negative reported KL divergence for dof>1 HOT 4
- Unable to use custom callable metric HOT 2
- process crashes when /tmp gets full HOT 2
- Question about SGD method used HOT 2
- [Windows] saving/loading TSNEEmbedding objects to pickle, Directory error HOT 6
- Cannot install on Mac M1 HOT 1
- `utils` import error in example notebooks HOT 1
- Problem with data from CSV file HOT 7
- Question on initialization HOT 4
- import errer HOT 1
- Allow KNNIndex classes to take **kwargs arguments and pass them to KNN library HOT 2
- oldest-supported-numpy depricated HOT 3
- Advice on how to use this Repo with sparse data HOT 5
- Pickling/unpickling embeddings object does not work on Windows HOT 2
- missing hardening flags on Debian building HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opentsne.