5. Running an A/A test

The best way to QA a Taplytics integration with a 3rd-party analytics tool is to setup an experiment to gather results from users and compare the data in Taplytics and the analytics tool afterwards. This can be done by running an experiment without any changes to your app but includes all the correct events that you would like to test.

Experiment Details

Visual Editor / Code Blocks & Variables
Ignore this section. We do not need to include any code variables because we’re not making any changes to the app. By default the Experiment will contain a baseline and variation so you won’t need to add any.

Goals

Please include all the Taplytics / Amplitude Events that you wanted to test here
There is no limit to the number of goals allowed in an experiment

If you decide in the middle of the test that more Goals should be added, you are welcome to edit an Active experiment and add more Goals; these will be filled retroactively (given that the Events being tracked have existed since the start of the experiment)

Distribution

Segment the experiment to All Users
Keep the Distribution at 50-50% for baseline and variation 1
Keep Rollout at 100%

Gathering Results

Generally, the recommendation for experiment data to reach a steady state is to run the experiment for 2 weeks, however, you may stop the experiment at any time or until you’ve gotten a large enough sample size of users and events. Afterwards, you can compare the results from the Experiment Results page to the data you’ve collected in Amplitude to see if the numbers on both platforms are similar.

Some considerations for data analysis when comparing both sides are:

Ensure that the users/events you’re comparing are the same on both ends. This includes applying the same filter on your Analytics tool to reflect the ones that are being used by the A/A test in Taplytics.
Not all Analytics tools will yield the same results 1 to 1; there will likely not be a perfect match and some variance is to be expected due to any sampling, session lengths, and other platform-specific attributes for gathering data.