GithubHelp home page GithubHelp logo

Mean ROC curve in ggroc() about proc HOT 5 CLOSED

AngCamp avatar AngCamp commented on June 22, 2024
Mean ROC curve in ggroc()

from proc.

Comments (5)

xrobin avatar xrobin commented on June 22, 2024

This is the same as the request in #41, except specifically requesting support in the ggroc function.

from proc.

AngCamp avatar AngCamp commented on June 22, 2024

Sort of, except the visualization they show should be smoother. I have provided the sklearn version above which you should take a look at. Another issue I've noticed and this prehaps should be a seperate bug issue, is that the ROC curve is sometimes plotting diagonal lines. This absolutely should not happen.

This plot is made using your ggroc() function from a list of roc objects:
image

It should not be displaying diagonal interpolations between cuts. There should be steps occurring.

from proc.

xrobin avatar xrobin commented on June 22, 2024

I don't know what you mean by they should be smoother. Of course if you average many curve the result will be smoother, but this is given by the averaging itself.

By the way diagonal lines in ROC curves are 100% normal.

from proc.

AngCamp avatar AngCamp commented on June 22, 2024

Smoother CI: The blocks of CI are not a very nice solution IMO, if you look at the sklearn method they show the CI as a blurred or shaded area displaying the error bars across the whole plot. Here is an example of this in ggplot2: https://stackoverflow.com/questions/26396149/custom-ggplot2-shaded-error-areas-on-categorical-line-plot

Diagonal Interpolation: I am only passing on a complaint from my supervisor, he's not wrong either. I think he is being nitpicky but it isn't invalid criticism. If you think about what the ROC is it should only grow in steps, as the threshold is increased it does not result in smooth FPR or FNR changes, it jumps as the classification of the datapoints changes. With lots of data it will look smoother since the steps will be smaller but in reality small discrete jumps are occurring. I appreciate that the diagonals are just a result of interpolating between two datapoints but in this case it is technically incorrect. If you look at any other ROC plotting package in R they do not plot the ROC like that because it's technically incorrect.
https://cran.r-project.org/web/packages/plotROC/vignettes/examples.html

The ROC curve should be a sawtoothed curve, no graded continuous changes should be present because it jumps in discrete steps.

from proc.

xrobin avatar xrobin commented on June 22, 2024

This is getting out of hand. You're asking about mean which should be smooth, but now suddenly confidence intervals are coming into the picture. You're linking to stack overflow posts which do things but I don't know which part you're referring to. Ultimately I have no clear idea what you're after exactly. Please be very clear and precise with what you'd like, what algorithm should be used, how it should be displayed, etc. See the Feature request template and follow it as much as possible. Please include things rather than link to pages that may change or disappear. Stick to one feature request at a time if possible. Otherwise I'll have to close this one as unclear.

Diagonals are the correct way to handle ties as per ROC definition, see Fawcett's An introduction to ROC analysis. I don't care what your supervisor is thinking, breaking ties other than with the expected segment is wrong. The plotROC package does that too, as expected and as appropriate.

from proc.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.