Classification and Regression Performance Metrics library
unixjunkie / cpmlib Goto Github PK
View Code? Open in Web Editor NEWClassification and Regression Performance Metrics library
Classification and Regression Performance Metrics library
Generic versions of those two functions could come in a Utils module.
let shuffle_then_cut seed p train_fn =
match Utls.lines_of_file train_fn with
| [] | [_] -> assert(false) (* no lines or header line only?! *)
| (csv_header :: csv_payload) ->
let rng = BatRandom.State.make [|seed|] in
let rand_lines = L.shuffle ~state:rng csv_payload in
let train, test = Utls.train_test_split p rand_lines in
train_test_dump csv_header train test
let shuffle_then_nfolds seed n train_fn =
match Utls.lines_of_file train_fn with
| [] | [_] -> assert(false) (* no lines or header line only?! *)
| (csv_header :: csv_payload) ->
let rng = BatRandom.State.make [|seed|] in
let rand_lines = L.shuffle ~state:rng csv_payload in
let train_tests = Utls.cv_folds n rand_lines in
L.rev_map (fun (x, y) -> train_test_dump csv_header x y) train_tests
we already have ROC AUC but...
Can be used to compare the ranking power of methods.
Currently there is a hack using gnuplot.
If we had an OCaml implementation of Platt scaling, this would remove the dependency to gnuplot.
Regression Error Characteristic Curve
Bi, J. and Bennett, K.P., 2003. Regression error characteristic curves. In Proceedings of the 20th international conference on machine learning (ICML-03) (pp. 43-50).
$LogAUC_\lambda=\frac{\sum_{i}^{where~x_i\ge\lambda} (\log_{10} x_{i+1} - \log_{10} x_i)(\frac{y_{i+1}+y_i}{2})}{\log_{10}\frac{1}{\lambda}}$
RIE = S/<S>
S = sum_over_i (exp (-rank(active_i) / a))
<S> = average S for 1000 trials where molecules (actives and inactives) are randomly sorted.
a << N.
for example, with a of 50, S almost vanishes after 300.
This is a kind of early EF measure.
by the one from RanKers.
Write a test suite for it.
instead of a list
after tests
also, it will be useful to version constrained to the previous major
version software that were already using this library
cf. latest tag
NRI = p(up/event) - p(down/event) + p(down/nonevent) -p(up/nonevent)
For the layman: actives should go up, inactives should go down.
If they do, bonus points. If they don't, malus points.
This allows to compare two classifiers based on their ranking performance; not weighted more
towards early recovery.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.