Comments (2)
Reference implementation which is unfortunately available only as MATLAB p-files (encrypted).
from astartes.
Thanks for sharing this paper! To summarize our preliminary discussion, my initial thought is that I see no reason to prefer this method over Kennard-Stone (KS) or random sampling (RS):
- KS enforces interpolation since it takes the points furthest away from each other in the X space and places them in the training set, which causes the testing set to be entirely contained within the space of the training set. However, this rigorous enforcement comes at the cost of scaling as O(N^2) since that’s simply the cost of computing a distance matrix. It's up to users to decide if this cost is worth it.
- RS often results in similar interpolation splits since random sampling often causes the training and testing set to have similar distributions, especially if the original dataset is large enough. Importantly the computational cost is dramatically lower.
Conceptually, I’m currently not convinced that blending them offers any advantage. After all, why would we eat the O(N^2) cost to enforce interpolation but then use random splitting to undo that rigorous assignment? If the hypothesis is that MLM leads to better testing set performance, this seems at odds with the results presented in this paper. Table 1 shows that MLM doesn’t help for datasets 2, 3, 4, and 5. This is often true when looking at Table 2 as well. Reporting the mean +- standard deviation from the cross-validation would have also helped in interpreting these results.
It would be interesting to read this paper more closely to better understand. It would also be useful to perform additional analysis with their code and datasets. For example, it would be interesting to compare their implementation to 2 function calls from astartes: first call KS then pass the resulting indices to RS.
from astartes.
Related Issues (20)
- [BUG]: train_test_split get KeyError when X is pd.DataFrame HOT 6
- [FEATURE]: Return pd.DataFrame if X is DataFrame after split HOT 5
- [FEATURE]: Mahalanobis Distance Kennard-Stone (MDKS) Sampler HOT 2
- [FEATURE]: M-SPXY HOT 1
- [FEATURE]: SWNW Sampler HOT 1
- [BUG]: `tabulate` should be in default requirements
- [BUG]: `sklearn` v1.3.0 introduces backwards-incompatible changes in KMeans HOT 2
- Pre-Submission Checklist
- pyOpenSci Initial Review Cleanup HOT 2
- [FEATURE]: Split based on functional groups
- [FEATURE]: Szymanski Sampling
- 3.12 support blocked by RDKit, tracking
- Update Citation to JOSS Paper HOT 1
- [QUESTION]: It appears that scaffold splits always return the same splits HOT 8
- [FEATURE]: Optionally add sklearn's `TimeSeriesSplit` HOT 3
- [FEATURE]: split by atom presence/absence HOT 3
- [FEATURE]: split by property value HOT 1
- [DOCS]: Add an '`astartes` in the literature' Section to the README
- [BUG]: ImperfectSplittingWarning rounding HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from astartes.