How Taplytics Assigns Users Into Experiments

Bucketing is the process used by Taplytics to assign users and devices to different variations of an experiment. Taplytics offers two types of bucketing:

Device-Based Bucketing (DBB)
User-Based Bucketing (UBB)

Device-Based Bucketing

Device-based bucketing ensures consistent experiment and feature flag behaviour across pre and post log-in user states where userID’s are not required or necessary.

Benefits:

Users are targeted based on their device:
- For mobile devices, Taplytics sets a device id
- For web applications, Taplytics sets a device id associated with the browser cookie
  This requires a standard, out-of-box implementation
Users always see the same experience on the same device

Considerations

Depending on how your users use and interact with your product, this implementation method does not guarantee a consistent experience across multiple devices.

For example, if the expected user behaviour is to access the platform on their mobile device as well as through their desktop, a consistent experience is not guaranteed because the assignment and distribution is on a device level.

Implementation

Device-Based Bucketing is the default setting and no additional setup is required.

User-Based Bucketing

User-based bucketing ensures consistent experiment and feature flag behaviour across devices with pre and post logged in user states where users are identified with a userID.

Benefits:

Users are targeted based on a unique userID
Users always see the same experience across devices in a logged-in state
User-based bucketing is the best option for true cross-platform optimization and attribution

Considerations

Applications must pass a unique userID for each user to Taplytics to ensure a consistent user experience. Also, there are additional Taplytics implementation steps.

The main consideration affects anonymous users who never log in and use multiple devices. It is possible these users will receive different pre-logged in experiences. In this use case, an anonymous user will have two unique IDs on different devices until they log in at least once on both devices. The pre-logged-in anonymous data will not be included in the logged-in user model.

Implementation

Users looking to leverage UBB will need to contact Taplytics support or your CSM to configure a project to the user-based bucketing setting.

Server-Side SDK (including Universal API) implementations must have user-based bucketing enabled on their project to provide consistent bucketing.

For Client-Side implementations, add a Start Option to your initialization code. You can find this info in the Start Options section of the SDK docs.

Client-Side SDKs (start options):

Bucketing: How it Works

Taplytics uses a deterministic MurmurHash algorithm that takes either the user ID or device ID (depending on the bucketing type) and combines it with the experiment ID to generate a unique random index for each user. This index is used to determine the variation that a user receives in the experiment. In each experiment, we use a random distribution start index to ensure randomness between experiments. From there, each Variation is slotted between a range of values determined by the distribution percentage of the Variation.

Example

If variation 1 has a distribution start index of 0.3 and a distribution percent of 20%, it means that variation 1 will be assigned to users whose random index falls within the range of 0.3 to 0.5 (i.e. 0.3 + (1 * 0.2)), wrapping back to 0 if the value exceeds 1.

In summary, the bucketing process assigns a random index to each user based on their user ID or device ID. This index falls within a range of indexes allocated to each variation in an experiment which is the resulting variation that a user will receive on their App.

Stickiness

Taplytics uses a deterministic algorithm to create indexes for each user per experiment. This means that users will always be bucketed into the same variations if they use the same user ID or device ID (depending on the bucketing type).

On client-side SDKs without any specific configurations, we retain the last set of experiments and variations that a user has received and use it to ensure that users retain the same variations from experiments that they were previously enrolled in. This ensures that even if the distribution percentages of an experiment’s variations change, the user will continue to receive the same experience. It also ensures that users that are force-bucketed into experiments, will continue to receive those experiments.

Example (Client-Side SDKs)

Variation 1 is set to 50% and a user receives Variation 1 on their mobile device. The experiment is then modified and Variation 1 is set to 0%. The user, because they’ve already received Variation 1 previously, will receive the same Variation on their next session.