Comments (5)
Hey @kyleschmolze, different code paths are run after the field_test_events
table is created. Here's a good place to start debugging:
field_test/lib/field_test/experiment.rb
Line 169 in 05ea051
from field_test.
@kyleschmolze , did you have any progress on this? I am having the same issue.
from field_test.
Yeah, I haven't had time to finish, yet, but I have some code along the right path. The problem is with the way the "participated" number is calculated here. The memberships-events join prevents any memberships from being returned that don't also have an event. And since events are only recorded for converted users, this means "participated" will always be equal to "converted" in this calculation, and so it will say 100% conversion rate, and ignore all un-converted users (returning 0 for that).
Instead, I just split it into 2 queries. One to calculate the number of total participants, and one to get the number of converted participants (using the same join). My code looks like this, starting on line 99:
relation = memberships
relation = relation.where("created_at >= ?", started_at) if started_at
relation = relation.where("created_at <= ?", ended_at) if ended_at
if use_events?
participants = relation.group(:variant).count
data = {}
data = relation.joins(:events).where(field_test_events: {name: goal}).uniq.group_by(&:variant)
data.keys.each do |variant|
data[[variant, true]] = data[variant].length
data[[variant, false]] = participants[variant].to_i - data[variant].length
data.delete variant
end
# if no conversions, make sure to still put the participants
participants.keys.each do |variant|
if data[[variant, true]].blank?
data[[variant, true]] = 0
data[[variant, false]] = participants[variant].to_i
end
end
else
data = relation.group(:variation, :converted).count
end
from field_test.
I also had this behaviour on my project.
I am trying to implement a recommendation system but will come back to take a look at this.
from field_test.
PS: @ankane Thanks for your awesome gems
from field_test.
Related Issues (20)
- Arbitrary Variants Via Query Parameters in Field Test 0.3.0 HOT 1
- Dynamic Variants? HOT 3
- Does it make sense to exclude internal devices / IP addresses? HOT 2
- Exclude users from experiments HOT 4
- Rails 6 Support HOT 1
- Use FieldTest in Sidekiq Workers/Models HOT 1
- Filter by date on primary dashboard HOT 1
- Allow newer `browser` gem versions HOT 1
- CSRF Vulnerability with Non-Session Based Authentication
- Probability of Winning With More Than 3 Variants HOT 1
- Old experiment data preventing conversions from being tracked in new experiments HOT 3
- Performance improvement for 0.5 HOT 1
- Unable to install gem due to "Could not find rice/rice.hpp header" HOT 15
- Question: how is "converted" defined? HOT 2
- Feature toggle, rollout library HOT 2
- Failed to build gem native extension with v0.5.2 on Fedora 36 HOT 2
- BinaryTest supports up to 4 variants HOT 1
- Experiment showing 0 conversions HOT 2
- segfault when used with fasttext-ruby? HOT 3
- YAML.load permitted_classes HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from field_test.