nhs-r-community / funnelplotr

Funnel plots for comparing institutional performance, with overdispersion adjustment

Source code for NHS-R Community website

Following the values and commitment of the NHS-R Community to open source this website has been built in Quarto to be open in its code and further contribution.

Many of the blogs, which go back to the start of NHS-R Community in 2018, were first posted through a WordPress site and so we don't have a record in GitHub for their contributions, however, the author and the date they wrote the blog are in the source code.

New contributions and corrections can be made directly to this repository through a pull request. Details on how to contribute to an NHS-R Community GitHub repository can be found in the Statement on Tools technical appendix.

This website is not yet live!

Whilst this site is being built the current url www.nhsrcommunity.com is directed to the WordPress site. All the blog posts are being saved under the original titles so that when this site is redirected to the www.nhsrcommunity.com url they will be same as the WordPress.

funnelplotr's Issues

Allow choice of tau2 estimation method

An interesting additional feature could be the ability to use REML estimation to obtain the tau2 term, which could also allow for additional levels of variance terms for inflating the control limits.

A possible workflow could be:

1. User calls tau2_method="REML" in the funnel_plot call (where the tau2_method option would default to DL)
2. Construct unadjusted z-scores
3. Truncate data based on unadjusted z-scores
4. Pass truncated data to metafor::rma.mv, using the transformed outcome and standard errors
5. Extract estimated variance

Alternatively, metafor's escalc function could be used directly for constructing the standard errors.

Let me know what you think, happy to work this up and open a PR

outlier selection

Would it be possible to be able to select just the upper outliers to label, please?

outlier highlighting not working with multiplier or Poisson limits at 95%

Suspect this is to do with the logic of the outlier function, and order of logic.

Update ggplot size argument

geom_line size argument has been deprecated in ggplot2, needs updating to linewidth

RC SE calculation

With the calculation of the standard errors for ratio of counts data, Spiegelhalter gives the formula:

But I see that you're using:

I know that the 0.5 is to handle 0 counts, but would you have a brief explainer for the rest? Is this a more robust standard error? Thanks!

Labels not working for 99% limits

As title.

General tidy and hide internal functions

It's all a bit messy with the internal functions not being hidden

SR CQC limits bug?

Hopefully this is just me misunderstanding the adjustment, but it looks like the CQC limits aren't being back-transformed to the original scale. Given that it uses a square-root transform, shouldn't the returned limits be squared?

od_limits.R

mod_plot_agg$OD95LCL <- multiplier * (mod_plot_agg$target_transformed - (1.959964 * sqrt( (mod_plot_agg$s^2 + tau2))))

build_limits_lookup.R

dfCI$odll95 <- multiplier * (1 - (1.959964 * sqrt((dfCI$s^2 + sqrt(tau2)^2))))

Midline doesn't move with multiplier

Need to look if multiplier is moving the centre line too

Resolve the scaling rules

Scaling rules current work for large data sets, but a bit hit and miss for smaller ones. Revise the rules.

Posterior predictive intervals with overdispersion models

Hi Chris, my organisation uses your package for funnel plot control limits but have been having some issues with overdispersed data (even with overdispersion adjustments). I've put together a new approach which uses different distributions for overdispersion (i.e., the beta-binomial and negative-binomial) to construct the control limits, which seems to be working very well for our data.

If the approach seems reasonable to you, I'd like to open some PRs to integrate it into the package so our workflow stays consistent. I've put together a summary and some examples comparing the results in this document

If you've got the time would you be able to have a look? I'm extremely new to this area, so feel free to let me know if I've made any basic mistakes.

Modularise funnel function

Split the function into chunks calling sub functions for aggregation, overdispersion, limit generation, plot, and plot labels

Zero numerator error

Need to add explicit handling of cases with a numerator of zero. Needs to manually set the indicator to zero prior to z-score. Might need handling differently for different data types.
Probably better to not remove them, as they are likely outliers and probably handled by winsorisation during OD adjustment.

Limits calculation not matching SHMI

Tau and Phi seem to be a bit out on the most recent version of SHMI, making limits different. Reported by @richardi7 originally.

Vignette not working on github.io

The vignette does not work on github.io. I know I can find it on CRAN, but it would be kind of nice to access through the io pages too.

https://chrismainey.github.io/FunnelPlotR/articles/funnel_plot.html

Add methods for proportions

It is currently limited to standardised ratios, need to add the methods for proportions

Error handling issue around highlighting

Current error handling is not working for highlighting where more than one value is passed as a vector. Current mechanics are designed for just a single highlight and some users have been passing several as vector. Needs a rewrite to handle multiple and should probably print out any missing in error text.

funnel formatting

At the moment it seems that the colour of the funnel limits can only be changed as matching pairs of upper and lower limits. Would it be possible to amend so that the upper and lower limits can be different colours?

Add custom scaling option

Sometimes plots are skewed by extreme outliers or poor application of scaling rules. Add xlim and ylim type arguments to set it manually

Prepare new release.

Prepare 0.4.0 release.

• Hide unneeded functions from UI.
• Document changes in functions in NEWS and CRAN-COMMENTS.

• label_outliers now label
• Poisson_limits now draw_unadjusted
• OD_limits now draw_adjusted
• Highlight
• Updated method for limit calculations: fix non-monotonic transformation for PR. Add truncation and adjustment at lower values.
• Change to guide with new ggplot2 syntax

• Add soft deprecation and redirect catches for old arguments in CRAN version
• Add Andrew J as contributor
• Increment version number
• Increase test coverage
• devtools::check
• winbuilder
• rhub
• github actions
• Update pkgdown site
• Shout about it on social media

winsorising seems to be incorrect...

I might be wrong, but I think your implementation of winsorising is incorrect... or at least different to the methods described in the Spiegelhalter papers.

The supplementary file to Spiegelhalters BMJ paper (https://qualitysafety.bmj.com/content/qhc/suppl/2005/09/28/14.5.347.DC1/145347appendix.pdf) and the other paper you mention in the help file, both state "set the lowest 100q% of z-scores to Zq, and the highest 100q% of z-scores to Z1-q" and (the supplement at least) follows with "this retains the same number of z-scores but discounts the influence of outliers". You seem to use trimming instead... https://github.com/chrismainey/FunnelPlotR/blob/c9029f5906594f5310ca21f59a29a596e5857369/R/OD_adjust_func.R#L123-L124

Outliers not calcualting correctly

Seems like the outlier function is not picking up the multiplier argument.
Though this was fixed in previous version. Needs adding to outliers function, main calling plot function, and tests.

ggplot2 changes cause error text

ggplot2 has updated the way it handles restricting the 'guide' setting. This is causing deprecation messages to come through. Need to update to handle.

Add option to set control limits

At the moment, the control limits are fixed at 95 and 99.8%. This may not be standard for all applications of funnel plots. It would be nice to have options to set these in funnel_plot.

Add options to set colours of points.

I think at the moment it is not possible to set the colours of the points. This would be a helpful addition

Add S3

Move to S3 class for funnel plot output with appropriate methods

Colour theming points

Would it possible to adjust so that groups of points on the funnel plot can be identified according to a grouping variable which could be chosen. Group membership to be shown either by colour , or shape of point as per standard GGplot?