wdempsey / sense2stop-lvm

Sense2Stop: Smoking lapse detection

Sense2Stop: Smoking detection via hierarchical latent marked point processes

This project is statistical models for detection of smoking episodes using self-report and sensors to measure smoking episodes. Time-varying covariates such as urge and stress are measured via self-report. Interventions are provided over the study in order to reduce proximal stress. The proposed model attempts to account for the uncertainty in event times when jointly modeling risk and time-varying health processes (e.g., urge/stress) as well as in assessing impact of intervention on proximal risk of smoking.

Project Description

This project includes the code needed to reproduce results. This includes (A) exploratory data analysis, (B) algorithmic development, and (C) application of models to the cleaned datasets. If using this code please cite the paper using the following bibtex:

@article{dempsey:2020,
author = {Dempsey, Walter},
title = {Hierarchical point process and multi-scale measurements: data integration for latent recurrent event analysis under uncertainty},
booktitle = {arXiv},
year = {2020}}

The goal of this project is to do.

Code Description

If there are steps to run the code list them as follows:

0. Dependencies: all code is developed in Python using Anaconda.

• The Anaconda environment can be installed using bayesian.yml. See here for instructions on creating the environment. Simply open Anaconda shell, open to github repo and run:

conda env create -f bayesian.yml

• Symbolic links: Please setup symbolic links for linking to the Box directory since the data location can change across OS and systems.
  • Requires 4 symbolic links
    • cleaned_data maps to location ...\Box\MD2K Northwestern\Processed Data\smoking-lvm-cleaned-data
    • data_streams maps to location ...\Box\MD2K Northwestern\Processed Data\Data streams
    • data_streams_backup maps to location ...\Box\MD2K Northwestern\Processed Data\Data streams - phone backup files
    • final_data maps to location ...\Box\MD2K Northwestern\Processed Data\smoking-lvm-cleaned-data\final
  • For example: On Windows the following run from home directory will generate the correct symbolic link for final_data

  mklink /d final_data ...\Box\MD2K Northwestern\Processed Data\smoking-lvm-cleaned-data\final
  

  • On Mac or Linux, replace mklink /d with ln -s

1. Data access, preprocessing, and exploratory data analysis

• Data is stored on Box and is owned by PI Bonnie Spring. Access is limited to the study team; however,
• Data preprocessing converts the raw data into a set of data files
• Exploratory data analysis is presented as a set of ipython notebooks. Descriptive statistics are used to inform the prior on the measurement-error models using in the analysis phase

2. The methods directory contains all algorithms for MCMC estimation. Algorithms are developed within the pymc3. The algorithm, at a high-level, performs the following

• Sample event times given observations and parameters (using reversible-MCMC adjustment)
• Sample parameters given latent event times (using pyMC3 software)

3. All evaluation functions can be found in the the evaluation directory. In particular, we perform posterior predictive checks to confirm model fit to the data.
4. Final project report can be found in the write-up directory

sense2stop-lvm's Issues

pp-models-sr duplicates rows

I noticed that the current file duplicates rows
Happens in code block starting at line 162.

Pipeline for Self-report

build sr_gen.py file that builds the self-report for analysis/inference phase

Mem: EOD

1. Finish sr_eod.ipynb for EOD given SR
2. Finish random_eod.ipynb for EOD given random EMA
3. Finish pM_eod.ipynb for EOD given puffMarker
4. Update README with results

Recurrent event EDA fix

• Convert eod-puffmarker to event-eda
• Combine self-report and random EMA
• Build first-smoking day survival curve
• Build # of smoking events per day since first lapse

Sigma lower and upper bound

Project the scale parameter sigma into a proper window (low of 2.38^2/k * 10^(-6) and high 10)

Mem: survival

Add a survival comparison at day level under random, sr, and eod versus combined

See how it would impact the curve

Local directory

• Update README so that symbolic link instructions are included
• These are used for data pre-processing and to tell the system where "final" is as well

Mem: HTMGS

• Fix sr-htmg.ipy to run with self-report and get final estimates
• Fix random-htmg.ipy to run with final random EMA file and get estimates
• Update readme with results

Mem: SR

1. Create a file 'sr_confirmation.ipynb'
2. We can use random EMA to get at the
   • If random EMA is "Yes", then the person didn't self-report
   • This gives us P( SR = 0 | event) = # of random Yes / (# of random Yes + # of SR).
   • Check this is a good estimate
3. Report in the README

Mem: puffmarker

1. Construct pM_confirmation.ipynb
2. For every SR or random EMA 'Yes', what percent are NOT confirmed by puffMarker
3. For every random EMA 'No', what percent had a puffMarker detection go off in that window
4. Report in README what the implied false positive, false negative rates could be