4. How to Segment Users for Experiment Distribution

Get advanced filtering control over your experiments

This article shows you how to use Taplytics' powerful user-segmentation controls and allows you to target an experiment to a very specific segment of your users.

If you haven't done so already, take a look at how to set up the different types of Experiments through the Taplytics dashboard.

To learn more about A/B testing for any platform, check out our video tutorial here.

---

There are three steps to completing the Distribution section of your experiment.

1. Experiment User Segment
2. Experiment Variation Distribution Percentage
3. Experiment Rollout

Each of these help you target and control the audience who will receive the experiment. Please see below for an overview of how to use and better understand each item.

1. Experiment User Segment

This is where you select the specific audience who will see your experiment.

Taplytics has several out of the box filters you can leverage. For a full description of all of the filters please find more info here: Taplytics Distribution Filters.

You can leverage several filters at once to narrow down your experiment audience and Taplytics will give you an estimate of the number of users who qualify for the experiment. For example, see the image below:

2. Segmenting Users from CSV Files

🚧

M1 Mac Users on Google Chrome (v105.0.5195.125) unable to upload CSVs

As of October 11th, 2022, M1 Mac Users on Google Chrome v105.0.5195.125 (Official Build) (arm64) may experience a bug that prevents the file attachment window from opening when attempting to upload files.

If this issue affects you, please update your Google Chrome to the latest version, which includes the patch for this bug.

Taplytics supports targeting users from a CSV file exported from a CRM or similar system. Users can be segmented and filtered, within experiments and feature flags, by uploading a single-columned CSV file containing one of the following identifiers as a header:

• user_id
• email

📘

CSV Structure Validation

Before uploading a CSV file, please ensure that the identifiers in the header area are in the correct form. Users may be identified by either their “user_id” OR “email”. The CSV file may only have a single column.

1. When the CSV file is ready, navigate to a feature flag or open an experiment and navigate to the Experiment Distribution tab.

2. Scroll down to the Audience Segmentation section and click on the User Attribute icon.

3. From the dropdown menu select Use / Upload CSV.

4. Upload your CSV file by selecting Upload a CSV file from the dropdown menu.

5. Select Upload a Local File within the Upload CSV window.

6. Once the CSV is finished uploading, you can select it from the dropdown menu.

📘

Warning Message

A warning message will appear when the CSV is finished uploading. To get rid of the warning, simply select your CSV file from the dropdown menu.

3. Experiment Variation Distribution Percentage

The distribution percentage controls the percentage of users who will be placed into variations.

The ideal set up for experiments is an even distribution percentage between all variations. This ensures that baseline and the variations have similar numbers of users and as a result, your experiment goals will reach statistical significance (chance of beating baseline) in a reasonable amount of time.

4. Experiment Rollout

The Experiment Rollout controls how many users within the variation will actually be shown the change.

For example, click on the image below and see how this experiment distribution is setup:

In the example above;

• A total of 60 users will be eligible for the experiment
• Based on the distribution percentage, approx. 30 users will be bucketed into baseline and 30 users will be bucketed into the variation because the Variation Distribution percentage is set to 50/50

Assuming 30 users are in Variation 1 and with an Experiment Rollout set to 50%, approximately 15 users will SEE the actual variation and 15 will see the baseline. Only after you move the Rollout to 100% will all users who are in the variation actually be seeing the changes.

Rollouts are randomly distributed per user but are deterministic in nature. Using the example above, on User 1's first session after the Experiment is started, they will be randomly distributed into the Experiment. They have a 50% chance of being bucketed into the Experiment based on the rollout.

Now, let's say User 1 receives the Experiment at a rollout of 50%. The rollout is then changed to 30% and User 1 stops receiving the Experiment. If we then change the rollout back to 50%, User 1 will once again, receive the Experiment. They will not get randomly distributed again.

Most experiments start with a 100% rollout since the objective of experimentation is to determine which variation is performing better. In that case, a 10% or 50% rollout isn't necessary.

But in some cases, you might be testing sensitive updates and would like to gauge your audience's reaction to the changes therefore, a rollout is a handy way to control access to a variation.

--

Keep moving through the experiment workflow!

What's next? You're at the last step! Review your experiment setup and launch it to users!