Comments (3)
I agree this would be cool. Do you have a reference on how this is implemented in the context of ROC curves?
from proc.
The curve would be based on the weighted versions of sensitivity and specificity.
library(tidymodels)
#> Registered S3 method overwritten by 'tune':
#> method from
#> required_pkgs.model_spec parsnip
data(pathology)
str(pathology)
#> 'data.frame': 344 obs. of 2 variables:
#> $ pathology: Factor w/ 2 levels "abnorm","norm": 1 1 1 1 1 1 1 1 1 1 ...
#> $ scan : Factor w/ 2 levels "abnorm","norm": 1 1 1 1 1 1 1 1 1 1 ...
set.seed(1)
pathology$weights <- runif(nrow(pathology))
event <- "abnorm"
unweighted <-
sum(pathology$pathology == event & pathology$scan == event) /
sum(pathology$pathology == event)
unweighted
#> [1] 0.8953488
# via yardstick:
sensitivity(pathology, pathology, scan)
#> # A tibble: 1 × 3
#> .metric .estimator .estimate
#> <chr> <chr> <dbl>
#> 1 sens binary 0.895
weighted <-
sum( pathology$weights * (pathology$pathology == event & pathology$scan == event) ) /
sum( pathology$weights * (pathology$pathology == event) )
weighted
#> [1] 0.9013333
Created on 2021-09-13 by the reprex package (v2.0.0)
@DavisVaughan has the start of changes that we will be making to yardstick
here
from proc.
I think I see. The easiest would be to directly update the roc.utils.perfs.all.fast to calculate TP/FP taking the weights into account:
tp <- cumsum(response.sorted==1 * weights.sorted)
fp <- cumsum(response.sorted==0 * weights.sorted)
A few thought on the implementation:
- The number of cases and controls might become fractional because of this change. I'm not sure what side-effects this could have.
- There's a C++ algorithm that will need to be updated too. It's a loop so it should be quite straightforward. Alternatively it could be a good time to get rid of alternative algorithms and simplify the code.
- It will be necessary to modify the
roc
objects and store the weights there, so that bootstrap functions re-use the weights appropriately. - At this point I'm not sure how much changes will be required in those bootstrapping functions. They've needed major refactoring for a long time but I never found the time to do so.
- Issue #70 will get in the way. There's quite a lot of redundancy as pROC has several functions that build ROC curves under the hood (ie
auc
,ci
, etc), which will have to be updated.
from proc.
Related Issues (20)
- DeLong test with the same ROC curve crashes R HOT 2
- Print issue of roc-class object when a formula object is used for creating roc. HOT 4
- likelihood ratios (and associated CIs) from pROC HOT 2
- OutDec options HOT 5
- pRoc does not react on options(OutDec=",") in plot() HOT 1
- not enough details in coord() "best" method HOT 3
- ggroc.list parameter legacy.axes break HOT 2
- One-sided CIs for AUCs HOT 2
- ggroc on smooth.roc objects HOT 3
- Partial match warnings HOT 3
- Suggesting vdiffr but not using it conditionally HOT 1
- Submit pROC 1.17.0 to CRAN HOT 3
- More inputs in coords
- Venkatraman test improperly documented as comparing AUC HOT 4
- /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found HOT 2
- Value of AUC and ROC curve HOT 1
- Error in CI for sensitivity HOT 4
- Summary ROC curves HOT 1
- Error in delongPlacementsCpp(roc) : function 'Rcpp_precious_remove' not provided by package 'Rcpp' HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from proc.