thumbtack / abba Goto Github PK

Hello,

Firstly, thank you for creating the Abba project. I am using it for a project I am working on for my internship where I am displaying auto-generated A/B tests between different features of a product. This project has saved me a lot of time and effort in getting this implemented.

However, I believe I have found an issue which causes the Improvement column to display "(NaN%)". This occurs when the Baseline has 0 success out of n trials with n >= 0 and any number of Variations with 0 success out of m trials with m >= 0.

To reproduce:
1.) Create Baseline with following features:

• Number of Success: 0
• Number of Trials: 30

2.) Create one or more Variations with following features:

• Number of Success: 0
• Number of Trials: 40

I already have a fix to display "0%" under Improvement. Would it be okay to fork the project, make my changes and open a pull request for them? In addition, I have found a few small syntax mistakes such as missing semi-colons that I can fix as well.

Thank you.

Currently, if you paste a value with a comma in it, the number gets truncated to be everything prior to the comma. The easy work-around is to remove the comma yourself. It's just tedious when done often.

What's the license?

Quick suggestion:
In my opinion, num_trials is not descriptive enough and does not match the wording used in the interface. I would rather use num_variations.
Changes would affect stats.py and stats.js.

Thanks for building abba and for listening to my suggestion! Keep it up!

Greetings. First of all, thank you for your work creating this tool. It's a go-to in my A/B testing toolkit. I have been using your web-based live demo tool for sometime now and I noticed this past week that the calculator was down. I'm unable to compute results or add another group. When you have a moment please investigate. Thank you.

I think something is timing out. There is something at the bottom of the page that seems to be hanging.

Here's a screenshot of some js loading:

Sometimes one is working with numbers which are formatted with commas and decimal places, e.g. 1,157.00. If you paste a number like that into trials or successes, you get NaN. It would be a nice usability improvement to strip commas and ".00" decimal strings from the input, rather than making the user do it. This becomes very tedious when done repeatedly.

We generate links to Abba containing many treatments groups. Sometimes we want to prune down the set of treatments so that we can focus the debrief on the control vs. some of the key experimental groups. If one of the "irrelevant" treatments is the first group in Abba, we can't delete it since there's no remove button.

To work around this, we cut and paste all of the values from one of the interesting treatments into the first row, and then delete the now-blank row.

The documentation on the demo page was one of the best succinct summaries of AB testing I've come across, and it explained what the tool was doing. The documentation is now gone.

Also, now this portion of the readme is incorrect:

You can find a usable app and detailed documentation on the live demo.

I'd suggest either restoring the documentation or moving it to the github wiki and linking to that wiki page from the demo page.