How to Conduct Effective Creative Testing for Mobile Apps

What is creative testing?

Creative testing in marketing is a process that involves trying out different creative elements to determine which ones are most effective in achieving goals. It includes experimenting with various content formats, designs, messaging, and other aspects. 

Even more insights from ASO & UA pros in ASO Busters on Discord!

The main goal of testing is to find the optimal combinations to increase your audience engagement, improve conversions, and overall effectiveness of advertising campaigns. 

‍

Here is an example for better understanding:

‍

Let's imagine that a company has launched a new product – eco-friendly water bottles. 

Then it needs to launch an advertising campaign on social media. The main goal is to attract as many leads as possible and increase sales. To do this, the team creates several variants of ads that differ in elements, i.e:

1. Images:

• option A – product photo on a white background;
• option B – visual in a natural environment (for example, in the steppe, forest, or beach).

2. Title text:

• option A – “ Contribute to the preservation of the planet with our eco-friendly bottles!”;
• option B – “Your new favorite water bottle - stylish and eco-friendly!”.

3. Call to action (CTA):

• option A – “Order now”;
• option B – “Learn more”.

Then, the company launches several ad groups, where each group is a combination of different images, headlines, and CTAs:

• group 1 – image A, headline A, CTA A;
• group 2 – Image B, headline B, CTA B;
• group 3 – Image A, headline B, CTA A;
• group 4 – Image B, headline A, CTA B.

The next step is to analyze the results of each ad group by key metrics. Examples of metrics:

• click-through rate (CTR);
• cost per click (CPC);
• conversions (purchases)
• cost per conversion (CPA);
• engagement (likes, comments, shares).

The final step is to determine the most effective variant, based on the collected data. For example, the team may find that the combination of image B, headline A, and CTA B has the highest CTR and conversions.

This example shows how creative advertising testing helps to identify the most effective elements of an advertising campaign.

Why do you need to conduct testing?

Creative performance can improve the effectiveness of marketing campaigns and help you achieve your business goals. 

‍5 reasons to start creative testing:

1. ‍Expense optimization

By understanding which elements of advertising bring the best results for your brand, you can optimize your marketing spend. This will allow you to reduce the expenses on less effective ads and invest more in those that bring the highest ROI.

2. Increase campaign efficiency

If you know which creatives (images, text, CTAs, etc.) work best, you can focus resources on the most effective options and increase the overall effectiveness of your campaigns.

3. Measuring efficiency

Specific metrics make it possible to measure the effectiveness of advertising campaign elements.

4. Better understanding of the audience

Insights into target audience behavior are important for business. They can be used to create new ideas and personalized campaigns that will resonate with the audience.

5. Конкурентна перевага

Being the first to identify and implement effective strategies through testing is one of the competitive edges in the market.

‍

Thus, testing helps to better understand audience preferences. Due to effective combinations, campaigns can be optimized, which in turn helps to achieve results and increase return on investment (ROI).

Types of creative testing

A/B testing (split testing)

One of the most popular types of testing for your brand is a b testing, where two different variants (A and B) of the same element (for example, a headline, image, CTA) are tested on a part of the audience. This helps to determine which of the variants works better. 

A/B testing involves creating two variants (A and B) of the same element. Variant A is called the control variant (usually the current or old version of the creative), and variant B is called the test variant. Then these variants are shown to different groups of users simultaneously and their effectiveness is compared by certain metrics. 

Tools for A/B-testing. In Google, this is Google Ads Experiments. In ASO in App Store Product Page Optimization is used for this purpose, in Google Play - Store listing experiments.

‍Proper use of A/B testing will help you make an informed decision to achieve better results.

A/A/B- or A/B/B-testing

A/A/B testing is a method that compares two identical control (current) variants (A and A) with one variable variant (B). 

A/B/B testing involves comparing one control (current) variant (A) with two identical modified variants (B1 and B2).

‍This type of testing helps to obtain more accurate confidence, as a larger sample reduces the possibility of random errors or mistakes in the results.

Multivariate testing (MVT)

MVT is a simultaneous testing of several variants of different elements (for example, headlines, images, CTAs) to determine the most effective combination. It differs from A/B testing, which compares only two variants of the same element.

‍Multivariate testing involves creating several test variants for each element under test and their combinations. For example, if you are testing headlines, images, and CTAs, and there are three variants of each, the number of combinations will be 3x3x3=27.

It can include the following components:

• headlines;
• images/videos;
• texts;
• CTA buttons.

Due to this type of testing, we can speed up the optimization best practices and find effective solutions to achieve business goals, as we have the ability to test creative elements and their combinations at the same time.

Conversion Lift tool for Meta

This tool allows you to measure the increase in advertising performance on Meta platforms. When you run a test, you create two similar audiences - a test audience and a control audience. People from the test audience see ads, while those from the control audience (also called the “control sample”) do not.

‍The test results provide answers to the following questions:

1. How effective is advertising on Meta platforms?
2. Do I need to increase my campaign budget?
3. How to increase the effectiveness of advertising on Meta platforms?
4. How many conversions have you managed to get thanks to advertising on Meta platforms?
5. Do the results obtained with the current attribution model match the results of a controlled experiment?

Requirements for testing by Meta:

1. The ad account has a campaign with a start date within the last year and a budget of at least $5000. Also, the campaign must have at least 500 optimized conversions with attribution settings of “1 day after click”, “7 days after click”, or “1 day after view”. 
2. For campaigns longer than 90 days, the requirements change proportionally. For example, a 180-day campaign requires a budget of $10000 and 1000 optimized conversions. Optimized conversions include only the ones you specify and only for the events for which you performed optimization.

You can create a test in the Experiments tool or using the Meta account representative (if you have one).

Note. To perform any testing, you need to have a sufficient amount of traffic. Then the test will be considered valid. The less traffic there is, the fewer test variants should participate in the test.

Where to find ideas for tests

• competitor analysis – observe what your competitors are doing and how they are doing it;
• communication with customers – collect feedback, it will become a valuable source for your ideas;
• internal brainstorming sessions – invite employees from different departments to discuss ideas (marketing, sales, customer support);
• trend research – keep track of current trends and adapt them to your topic;
• social media – Instagram, Facebook, LinkedIn, Pinterest, and TikTok are sources of inspiration;
• creative agencies – cooperate with creative agencies;
• training and courses – participate in training, webinars, and online courses;
• collaborate with influencers, they often have fresh ideas and insights into the audience;
• explore successful case studies of other companies in your industry and beyond.

A step-by-step checklist for testing

☑️Analyze current creatives and identify gaps

1. Ask yourself – what creatives are you using at the moment?
   • images;
   • videos;
   • texts;
   • headlines;
   • CTAs (calls to action).
2. Determine how well your current creatives resonate with your target audience. Collect data on the demographics, interests, and your audience behavior.
3. Based on the data, determine which elements of your current creative are not working effectively. For example, if a certain headline has a low CTR, it should be replaced.
4. Find areas where there is potential for improvement. New content formats, advertising channels, or approaches to audience segmentation are all points of growth.
5. Based on the analysis of the current situation and identified gaps, create hypotheses that you plan to test. For example:

• if the images don't grab enough attention, the hypothesis might be: “Brighter images with people will improve engagement.”;
• if CTAs don't lead to conversions, the hypothesis might be: “Changing the CTA text into a more specific one will increase conversions.”.

☑️Define your testing goals

Clearly define the goals of testing. It can be an increase in conversions, a decrease in cost per click, a boost in engagement, etc. But it's worth remembering that even test creatives should be in line with the business concept.

☑️Select items for testing

Write down what creative elements you plan to use in the test: headlines, images, CTAs, text messages, page layouts, etc.

☑️Create multiple variants of creatives

Develop several versions of each element. For example, if you plan to test headlines, create a few variations of them.

☑️Define success metrics

Set the success metrics you want to measure for each option: CTR, conversions, cost per conversion, engagement rate, etc.

☑️Select a testing methodology

A/B testing (split testing), A/A/B or A/B/B testing, multivariate testing (MVT), Conversion Rate Increase tool.

☑️Conduct a test launch

Divide your audience into groups and start testing. Check and make sure that the test is conducted for a reasonable period and with the necessary amount of traffic to obtain the correct data.

☑️Analyze the data you receive

Gather and compare the results of the different options across the defined metrics. Analyze which options performed best.Based on the data, decide on the next stage of the action plan. For example, it can be either choosing the best option for use in campaigns or conducting additional testing of other hypotheses.

☑️Implement and optimize the system

Implement the best creatives. Constantly monitor their effectiveness.

☑️Keep on testing

Creative testing is an ongoing process. Don't stop and keep testing new ideas and approaches to continuously improve your campaigns and reach new peaks.

Five mistakes to avoid when creative testing

To avoid these mistakes, you can increase the effectiveness of creative testing and get more relevant results that will help you make a decision:

1. Insufficient sample – running a test on a few users can lead to incorrect results. To determine the size, use the sample size calculator.
2. Short testing period – conducting a test over a short period can lead to inaccurate results. It is necessary to set aside time for testing, taking into account the seasonality and behavioral changes of users. For example, the minimum duration of small tests for applications with a large volume of traffic can last from one to two weeks. For applications with less traffic, the testing time can be three to six weeks.
3. Ignoring external factors – seasonality, competitors' marketing campaigns, and political, social, and economic changes can affect test results.
4. Using low-quality creatives – testing low-quality creatives negatively affects the results and can lead to wrong outcomes.
5. ‍Lack of a post-test action plan – before you start testing, determine how you will use the results and what actions you plan to take based on them.