GithubHelp home page GithubHelp logo

dscolby / causalelm Goto Github PK

View Code? Open in Web Editor NEW
19.0 1.0 0.0 2.54 MB

Taking causal inference to the extreme!

Home Page: https://dscolby.github.io/CausalELM.jl/

License: MIT License

Julia 100.00%
causal-inference causal-machine-learning extreme-learning-machine g-computation interrupted-time-series metalearning observational-studies quasi-experimental assumption-check double-machine-learning nonparametric-model sensitivity-analysis metalearner debiased-machine-learning r-learning s-learner t-learner x-learner doubly-robust

causalelm's Introduction

Build Status Code Coverage License Documentation Develmopmental Documentation Aqua QA Code Style: Blue

CausalELM enables estimation of causal effects in settings where a randomized control trial or traditional statistical models would be infeasible or unacceptable. It enables estimation of the average treatment effect (ATE)/intent to treat effect (ITE) with interrupted time series analysis, G-computation, and double machine learning; average treatment effect on the treated (ATT) with G-computation; cumulative treatment effect with interrupted time series analysis; and the conditional average treatment effect (CATE) via S-learning, T-learning, X-learning, R-learning, and doubly robust estimation. Underlying all of these estimators are extreme learning machines, a simple neural network that uses randomized weights instead of using gradient descent. Once a model has been estimated, CausalELM can summarize the model, including computing p-values via randomization inference, and conduct sensitivity analysis to calidate the plausibility of modeling assumptions. Furthermore, all of this can be done in four lines of code.

Extreme Learning Machines and Causal Inference

In some cases we would like to know the causal effect of some intervention but we do not have the counterfactual, making conventional methods of statistical analysis infeasible. However, it may still be possible to get an unbiased estimate of the causal effect (ATE, ATE, or ITT) by predicting the counterfactual and comparing it to the observed outcomes. This is the approach CausalELM takes to conduct interrupted time series analysis, G-Computation, double machine learning, and metalearning via S-Learners, T-Learners, X-Learners, R-learners, and doubly robust estimation. In interrupted time series analysis, we want to estimate the effect of some intervention on the outcome of a single unit that we observe during multiple time periods. For example, we might want to know how the announcement of a merger affected the price of Stock A. To do this, we need to know what the price of stock A would have been if the merger had not been announced, which we can predict with machine learning methods. Then, we can compare this predicted counterfactual to the observed price data to estimate the effect of the merger announcement. In another case, we might want to know the effect of medicine X on disease Y but the administration of X was not random and it might have also been administered at mulitiple time periods, which would produce biased estimates. To overcome this, G-computation models the observed data, uses the model to predict the outcomes if all patients recieved the treatment, and compares it to the predictions of the outcomes if none of the patients recieved the treatment. Double machine learning (DML) takes a similar approach but also models the treatment mechanism and uses it to adjust the initial estimates. This approach has three advantages. First, it is more efficient with high dimensional data than conventional methods. Metalearners take a similar approach to estimate the CATE. While all of these models are different, they have one thing in common: how well they perform depends on the underlying model they fit to the data. To that end, CausalELMs use extreme learning machines because they are simple yet flexible enough to be universal function approximators.

CausalELM Features

  • Estimate a causal effect, get a summary, and validate assumptions in just four lines of code
  • All models automatically select the best number of neurons and L2 penalty
  • Enables using the same structs for regression and classification
  • Includes 13 activation functions and allows user-defined activation functions
  • Most inference and validation tests do not assume functional or distributional forms
  • Implements the latest techniques form statistics, econometrics, and biostatistics
  • Works out of the box with DataFrames or arrays
  • Codebase is high-quality, well tested, and regularly updated

What's New?

  • Now includes doubly robust estimator for CATE estimation
  • Uses generalized cross validation with successive halving to find the best ridge penalty
  • Double machine learning, R-learning, and doubly robust estimators suppot specifying confounders and covariates of interest separately
  • Counterfactual consistency validation simulates outcomes that violate the assumption rather than the previous binning approach
  • Standardized and improved docstrings and added doctests
  • CausalELM talk has been accepted to JuliaCon 2024!

What's Next?

Newer versions of CausalELM will hopefully support using GPUs and provide textual interpretations of the results of calling validate on a model that has been estimated. However, these priorities could also change depending on feedback recieved at JuliaCon.

Disclaimer

CausalELM is extensively tested and almost every function or method has multiple tests. That being said, CausalELM is still in the early/ish stages of development and may have some bugs. Also, expect breaking releases for now.

Contributing

All contributions are welcome. Before submitting a pull request please read the contribution guidlines.

causalelm's People

Contributors

dscolby avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

causalelm's Issues

Make smaller functions and methods

Specifically, for the estimators and metalearners we should make a separate risk ratio function; we should find a way to make estimate_causal_effect! for g-computation smaller; estimate_causal_effect! for double machine learning should definitely be broken up into smaller functions; and crossfitting_sets should also be smaller. These smaller functions will then need to be testes.

Add W argument

Add W parameter for covariates for the treatment function in double machine learning, X learning, doubly robust estimation, and R learning.

Change contributor guide

Add that for functions with multiple methods there should be a function definition with no parameters in the main module that includes the docstring for all the methods.

Calculate p-values for ITS using alternative type of permutation

The current permutation test for p-values using ITS estimates the null distribution by using alternative cutoff points for pre and post-intervention periods. Instead, we should permute the order of the post-treatment observations to estimate null effects and use those estimated effects under permuted post-treatment observations to estimate the null distribution.

As a reminder, we can easily create permutations using a[shuffle(1:end), :].

Mild refactoring

Change multi-line function definitions and calls to

 foo(arg1, arg2, 
        arg3, arg4)

remove docstrings from every struct field and only put docstrings at the top of each struct.

Add support for GPUs

Priority should probably be integration with CUDA, then Intel, then ATI, then Mac. In theory this should be easy but we should watch for type instability.

Add doctests

Only where the output can be kept short and there is no randomization.

Implement E-values for all estimators

ITS conducts sensitivity analysis by generating random variables and re-estimating the causal effect. Instead, we should implement E-values, as proposed by VanderWeele, Tyler J., and Peng Ding. "Sensitivity analysis in observational research: introducing the E-value." Annals of internal medicine 167, no. 4 (2017): 268-274. Besides ITS, we should implement this as a test of confounding/exchangeability for all estimators.

The steps to implement are:

Start with the observed effect estimate (e.g., mean difference or average treatment effect) from your study, denoted as MEAN_OBS.

Calculate the lower and upper confidence limits (LL and UL) for your observed effect estimate (MEAN_OBS) based on the confidence interval from your statistical analysis.

To calculate MD_U (Minimum Strength for Upper Limit):

Start with the UL (Upper Confidence Limit) of the mean difference.

Mathematically, you can set up the following equation:

MD_U * MEAN_OBS = UL

Solve for MD_U:

MD_U = UL / MEAN_OBS

To calculate MD_L (Minimum Strength for Lower Limit):

Start with the LL (Lower Confidence Limit) of the mean difference.

Set up the following equation:

MD_L * MEAN_OBS = LL

Solve for MD_L:

MD_L = LL / MEAN_OBS

It's common for researchers to consider an E-value greater than 2 as indicative of a relatively strong association, meaning that unmeasured confounding would need to be at least twice as strong as the observed effect to explain it away. However, this is a heuristic, and the specific threshold for what is considered "high" may vary based on the field and the judgment of the researchers involved in a given study.

Refactor GComputation, SLearner, DoubleMachineLearner, and RLearner

S-learning does almost the same things as G-computation and R-learning is almost the same as double machine learning. Instead of having GComputation and DoubmleMachineLearner encapsulated within SLearner and RLearner and reusing the estimate_causal_effect! method for GComputation and DoubleMachineLearner, we should create AbstractSingleModelLearner and AbstractDoubleMachineLearner types. Then we can get rid of the encapsulation and having separate methods for them in inference.jl and overloading Base.getproperty and Base.setproperty! to get the Y vectors and just have estimate_causal_effect! methods for the abstract classes. The methods would then just do slightly different things for CATE vs ATE estimation.

Make models compatible with other data types

Right now the models work on arrays but they should be able to work with other input like dataframes, tables, etc. This will also require figuring out what people usually use to read and manipulate their data in Julia. Getting this done will probably be a v0.4 task.

Make methods and functions more general

Currently, most of the functions and methods only work with floats but they should be able to accept any real numbers or subtypes of real numbers. This should be a very easy fix.

Make sigmoid layer for binary classifiers

Currently, classification is done by just using the sigmoid activation function, which is basically just regression. This could potentially lead to predicted probabilities being outside of [0, 1]. Instead, for classification we should use a normal ELM with ReLU or another activation to get raw predictions and apply the sigmoid to those outputs similar to the way we use a softmax layer for multiclass classification.

Add doctests

While all of the examples in the documentation are from the unit tests and have all passed, it would be better to make as many doc tests as possible.

However, a lot of the tests give the expected output and fail even though they are correct, so we need to figure out how to get doctests working.

Make better looking documentation

This is not a priority and it will take a while. It is going to require working with CSS. Right now the theme is clean but the color palette is very bland.

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Update Docstrings

They should have separate sections for notes and references and not have "..." to sandwich the arguments section. The updated format should also be in the contributor guidelines.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.