The Reporting page displays the results for a trigger-based campaign.
To access the Reporting page for a specific campaign, click on its name under the campaign dashboard:
This article contains the following topics:
On the left hand side, you will see the number of people that triggered the campaign event, as well as how many of those users were sent a message:
Each of the different branches represents the variants you have setup for your report. The percentage bubbles to the left of each variant represents their allocation. As you continue to get results, you can fine-tune your allocation to push users toward the winning campaign.
Each variant displays the following:
- Number of users who received the message
- Percentage of users who reach the specified goals
- Percentage of users who opened the message
- Percentage of users who clicked through links (email only)
- Percentage of users who uninstall (mobile SDK only)
- Percentage of users who opt out (mobile SDK only)
Understanding the control group
Control groups are an excellent way to see whether or not your campaigns are driving users to your goals. A control group will not be sent a message, but is still measured for goal conversion. You can then take the control's goal percentage and subtract it from your variants' goal conversions. The difference indicates the lift - i.e. the net effectiveness of each variant.
If the lift for a variant is small, or in some cases smaller than the control, it is recommended that you turn off the allocation for said variant. This way you can reduce the amount of messages you send while maximizing your impact.
The bottom of the reporting page gives you specific values that were used to calculate the main results:
The following is a breakdown of what each row means:
- Users who did trigger - The number of users who did the trigger event and were considered in the campaign
- Users who did not match filters - These users did not meet 1 or more of the criteria specified in the filters of this campaign.
- Users who were frequency Capped - These users have already received a message and cannot receive more based on the frequency cap specific to this campaign.
- Users who were missing contact info - These users did not have the necessary contact info to receive a message. This attribute will change based on the channel being used to message the user. For Push notifications, it means the user does not have a value for APNS or GCM.
- Users not sent due to unsubscribed - These users have unsubscribed in the past and were not sent a message
- Failed to send message - This generally means that the contact info for this user was incorrect. In the case of mobile push, it usually means invalid APNS or GCM values. If all messages are failing, it's possible your certifications have expired. These can be updated on the settings > providers page.
- Users who were sent a message - These are the users who passed all the previous criteria and received a message for this campaign.
The following is a breakdown of each row in the goals section:
- Conversion rate for <goal> - This displays the overall conversation rate for a specific goal, as well as the individual percentages for each variant/control.
- Users who did <goal> after message - This displays the number of people who reached a specific goal, as well as the number attributed to each variant/control.
The following is a breakdown of each row in the unsubscribes section:
- Users who unsubscribed from campaign - The number of users who received a message and unsubscribed from this specific campaign.
- Users who unsubscribed from all messages - The number of users who received a message and unsubscribed from all future campaigns.
Declaring a winning campaign
Eventually we will declare a winner based on conversions to your primary goal, which is indicated by the ribbon icon:
This campaign is likely to bring you the best conversion to your primary goal. At this point you may choose to allocate 100% of your traffic to this variant. You may also want to generate new variants to continue testing against the winning variant. All of this can be achieved by editing the original campaign.