TimeVaryGFormula in Cython about zepid

[pzivich]

SAS is ungodly fast at the g-formula...

Using Keil et al. as a comparison (Nmc = 10000), SAS runs in 10.65 seconds. After some better code writing, It takes the v0.2.1 implementation (no Cython yet) 77.54 seconds!

The time difference is due to the MC loop. I think I have gotten near the limit of speed improvements due to better code writing alone. Cython is probably my next best best for closing the gap

[pzivich]

After some additional changes, I have sped up TimeVaryGFormula up quite a bit compared to this first implementation. It is still slower than SAS. However, it seems to be any further improvements regarding speed would inhibit user-functionality (rewrite to only handle NumPy rather than pandas, wrap with cython after NumPy conversion). NumPy would probably lead to a similar time as SAS, but handling to arrays and dynamically recoding them becomes less straightforward for the user. Specifically errors on where a column is located become easier to mess up. Since the g-formula implementation is not the most straightforward, I think at this time it is better to take the hit on speed

Maybe at a future date I will try to implement only NumPy in the background but I am happy with the current speed. The big time crunch issue is the bootstrapped confidence intervals. My recommendation would be to use a cluster at this point (or rewrite my code but with NumPy for your implementation if speed is that big of an issue)

In case anyone views this thread in the future, rewriting TimeVaryGFormula to handle NumPy arrays would be my advice for quicker run time.