User-tied A/B testing about turtledove

[appascoe]

I believe there are multiple ways of accomplishing this.

1. Interest groups themselves could be split into A/B groups, from whatever set of criteria (say, hash of a first-party cookie identifier). For example, on pets.com/cats, you could throw half the users into cats-GroupA and the other into cats-GroupB. Then, during aggregate reporting, you can supply which interest group the browser was added to. Then you would receive some aggregate results for both groups A and B.

2. The bidding.js function could also do the split at auction time and then perform the same through aggregate reporting. I think the unclear thing here is to make sure the browser itself stays in one group of you're looking for a user-level split. Perhaps the browser itself could have something like IDFA that is user-resettable and gets passed into the bidding.js function (which, of course, can't make network calls or anything like that for external tracking). I haven't completely thought through the implications here; I'm just spitballin'.

[michaelkleber]

In addition to Andrew's approach 1 above, Facebook's @benjaminsavage wrote up a proposal "Private Lift Measurement" which would allow this kind of A/B testing for contextually-targeted ads as well.

There is a lot of overlap between the machinery needed for Ben's proposal and for TURTLEDOVE — in some sense Andrew's point 1 above is about just how similar the two ideas are! So once we have an appropriate way to do private rendering and aggregate reporting for TURTLEDOVE-style interest groups, supporting A/B test diversion will be straightforward.

[JensenPaul]

Closing this issue as it represents past design discussion that predates more recent proposals. I believe some of this feedback was incorporated into the Protected Audience (formerly known as FLEDGE) proposal. If you feel further discussion is needed, please feel free to reopen this issue or file a new issue.