Boost Your App
Please fill in the form below and we'll call you back in no time.
Your request has been received! We'll be in touch shortly
Oops! Something went wrong while submitting the form.

How to Conduct Effective Creative Testing for Mobile Apps

App Store Optimization
CRO
Aug 6, 2024

“I know that half of my advertising budget is wasted; the problem is, I don't know which half” – retail magnate John Wanamaker said this about his marketing budget more than 100 years ago. However, there are now more opportunities to measure the impact of advertising on the audience and make it more effective. The RadASO team explains how to do creative testing.

What is creative testing?

Creative testing in marketing is a process that involves trying out different creative elements to determine which ones are most effective in achieving goals. It includes experimenting with various content formats, designs, messaging, and other aspects. 

Even more insights from ASO & UA pros in ASO Busters on Discord!

The main goal of testing is to find the optimal combinations to increase your audience engagement, improve conversions, and overall effectiveness of advertising campaigns. 

Here is an example for better understanding:

Let's imagine that a company has launched a new product – eco-friendly water bottles. 

Then it needs to launch an advertising campaign on social media. The main goal is to attract as many leads as possible and increase sales. To do this, the team creates several variants of ads that differ in elements, i.e:

1. Images:

  • option A – product photo on a white background;
  • option B – visual in a natural environment (for example, in the steppe, forest, or beach).

2. Title text:

  • option A – “ Contribute to the preservation of the planet with our eco-friendly bottles!”;
  • option B – “Your new favorite water bottle - stylish and eco-friendly!”.

3. Call to action (CTA):

  • option A – “Order now”;
  • option B – “Learn more”.

Then, the company launches several ad groups, where each group is a combination of different images, headlines, and CTAs:

  • group 1 – image A, headline A, CTA A;
  • group 2 – Image B, headline B, CTA B;
  • group 3 – Image A, headline B, CTA A;
  • group 4 – Image B, headline A, CTA B.

The next step is to analyze the results of each ad group by key metrics. Examples of metrics:

  • click-through rate (CTR);
  • cost per click (CPC);
  • conversions (purchases)
  • cost per conversion (CPA);
  • engagement (likes, comments, shares).

The final step is to determine the most effective variant, based on the collected data. For example, the team may find that the combination of image B, headline A, and CTA B has the highest CTR and conversions.

This example shows how creative advertising testing helps to identify the most effective elements of an advertising campaign.

Why do you need to conduct testing?

Creative performance can improve the effectiveness of marketing campaigns and help you achieve your business goals. 

5 reasons to start creative testing:

  1. Expense optimization

By understanding which elements of advertising bring the best results for your brand, you can optimize your marketing spend. This will allow you to reduce the expenses on less effective ads and invest more in those that bring the highest ROI.

  1. Increase campaign efficiency

If you know which creatives (images, text, CTAs, etc.) work best, you can focus resources on the most effective options and increase the overall effectiveness of your campaigns.

  1. Measuring efficiency

Specific metrics make it possible to measure the effectiveness of advertising campaign elements.

  1. Better understanding of the audience

Insights into target audience behavior are important for business. They can be used to create new ideas and personalized campaigns that will resonate with the audience.

  1. Competitive advantage

Being the first to identify and implement effective strategies through testing is one of the competitive edges in the market.

Thus, testing helps to better understand audience preferences. Due to effective combinations, campaigns can be optimized, which in turn helps to achieve results and increase return on investment (ROI).

Types of creative testing

A/B testing (split testing)

One of the most popular types of testing for your brand is a b testing, where two different variants (A and B) of the same element (for example, a headline, image, CTA) are tested on a part of the audience. This helps to determine which of the variants works better. 

A/B testing involves creating two variants (A and B) of the same element. Variant A is called the control variant (usually the current or old version of the creative), and variant B is called the test variant. Then these variants are shown to different groups of users simultaneously and their effectiveness is compared by certain metrics. 

Tools for A/B-testing. In Google, this is Google Ads Experiments. In ASO in App Store Product Page Optimization is used for this purpose, in Google Play - Store listing experiments.

Proper use of A/B testing will help you make an informed decision to achieve better results.

A/A/B- or A/B/B-testing

A/A/B testing is a method that compares two identical control (current) variants (A and A) with one variable variant (B). 

A/B/B testing involves comparing one control (current) variant (A) with two identical modified variants (B1 and B2).

This type of testing helps to obtain more accurate confidence, as a larger sample reduces the possibility of random errors or mistakes in the results.

Multivariate testing (MVT)

MVT is a simultaneous testing of several variants of different elements (for example, headlines, images, CTAs) to determine the most effective combination. It differs from A/B testing, which compares only two variants of the same element.

Multivariate testing involves creating several test variants for each element under test and their combinations. For example, if you are testing headlines, images, and CTAs, and there are three variants of each, the number of combinations will be 3x3x3=27.

It can include the following components:

  • headlines;
  • images/videos;
  • texts;
  • CTA buttons.

Due to this type of testing, we can speed up the optimization best practices and find effective solutions to achieve business goals, as we have the ability to test creative elements and their combinations at the same time.

Conversion Lift tool for Meta

This tool allows you to measure the increase in advertising performance on Meta platforms. When you run a test, you create two similar audiences - a test audience and a control audience. People from the test audience see ads, while those from the control audience (also called the “control sample”) do not.

The test results provide answers to the following questions:

  1. How effective is advertising on Meta platforms?
  2. Do I need to increase my campaign budget?
  3. How to increase the effectiveness of advertising on Meta platforms?
  4. How many conversions have you managed to get thanks to advertising on Meta platforms?
  5. Do the results obtained with the current attribution model match the results of a controlled experiment?

Requirements for testing by Meta:

  1. The ad account has a campaign with a start date within the last year and a budget of at least $5000. Also, the campaign must have at least 500 optimized conversions with attribution settings of “1 day after click”, “7 days after click”, or “1 day after view”. 
  2. For campaigns longer than 90 days, the requirements change proportionally. For example, a 180-day campaign requires a budget of $10000 and 1000 optimized conversions. Optimized conversions include only the ones you specify and only for the events for which you performed optimization.

You can create a test in the Experiments tool or using the Meta account representative (if you have one).

Note. To perform any testing, you need to have a sufficient amount of traffic. Then the test will be considered valid. The less traffic there is, the fewer test variants should participate in the test.

Where to find ideas for tests

Where to find ideas for tests

  • competitor analysis – observe what your competitors are doing and how they are doing it;
  • communication with customers – collect feedback, it will become a valuable source for your ideas;
  • internal brainstorming sessions – invite employees from different departments to discuss ideas (marketing, sales, customer support);
  • trend research – keep track of current trends and adapt them to your topic;
  • social media – Instagram, Facebook, LinkedIn, Pinterest, and TikTok are sources of inspiration;
  • creative agencies – cooperate with creative agencies;
  • training and courses – participate in training, webinars, and online courses;
  • collaborate with influencers, they often have fresh ideas and insights into the audience;
  • explore successful case studies of other companies in your industry and beyond.

Five mistakes to avoid when creative testing

To avoid these mistakes, you can increase the effectiveness of creative testing and get more relevant results that will help you make a decision:

  1. Insufficient sample – running a test on a few users can lead to incorrect results. To determine the size, use the sample size calculator.
  2. Short testing period – conducting a test over a short period can lead to inaccurate results. It is necessary to set aside time for testing, taking into account the seasonality and behavioral changes of users. For example, the minimum duration of small tests for applications with a large volume of traffic can last from one to two weeks. For applications with less traffic, the testing time can be three to six weeks.
  3. Ignoring external factors – seasonality, competitors' marketing campaigns, and political, social, and economic changes can affect test results.
  4. Using low-quality creatives – testing low-quality creatives negatively affects the results and can lead to wrong outcomes.
  5. Lack of a post-test action plan – before you start testing, determine how you will use the results and what actions you plan to take based on them.

A step-by-step checklist for testing

☑️Analyze current creatives and identify gaps

  1. Ask yourself – what creatives are you using at the moment?
    • images;
    • videos;
    • texts;
    • headlines;
    • CTAs (calls to action).
  2. Determine how well your current creatives resonate with your target audience. Collect data on the demographics, interests, and your audience behavior.
  3. Based on the data, determine which elements of your current creative are not working effectively. For example, if a certain headline has a low CTR, it should be replaced.
  4. Find areas where there is potential for improvement. New content formats, advertising channels, or approaches to audience segmentation are all points of growth.
  5. Based on the analysis of the current situation and identified gaps, create hypotheses that you plan to test. For example:
  • if the images don't grab enough attention, the hypothesis might be: “Brighter images with people will improve engagement.”;
  • if CTAs don't lead to conversions, the hypothesis might be: “Changing the CTA text into a more specific one will increase conversions.”.

☑️Define your testing goals

Clearly define the goals of testing. It can be an increase in conversions, a decrease in cost per click, a boost in engagement, etc. But it's worth remembering that even test creatives should be in line with the business concept.

☑️Select items for testing

Write down what creative elements you plan to use in the test: headlines, images, CTAs, text messages, page layouts, etc.

☑️Create multiple variants of creatives

Develop several versions of each element. For example, if you plan to test headlines, create a few variations of them.

☑️Define success metrics

Set the success metrics you want to measure for each option: CTR, conversions, cost per conversion, engagement rate, etc.

☑️Select a testing methodology

A/B testing (split testing), A/A/B or A/B/B testing, multivariate testing (MVT), Conversion Rate Increase tool.

☑️Conduct a test launch

Divide your audience into groups and start testing. Check and make sure that the test is conducted for a reasonable period and with the necessary amount of traffic to obtain the correct data.

☑️Analyze the data you receive

Gather and compare the results of the different options across the defined metrics. Analyze which options performed best.Based on the data, decide on the next stage of the action plan. For example, it can be either choosing the best option for use in campaigns or conducting additional testing of other hypotheses.

☑️Implement and optimize the system

Implement the best creatives. Constantly monitor their effectiveness.

☑️Keep on testing

Creative testing is an ongoing process. Don't stop and keep testing new ideas and approaches to continuously improve your campaigns and reach new peaks.

Traffic
Popularity
Keyword
1
385 585
99
instagram
2
325 480
98
whatsapp
3
201 485
95
telegram
4
177 737
94
facebook
5
119 067
91
tik tok
6
92 427
90
ifood
7
80 473
87
tinder
8
69 379
87
capcut
9
65 120
87
nubank
10
64 602
86
uber

To conclude, creative testing is a powerful tool for increasing the effectiveness of marketing campaigns. A structured and clear approach to testing, setting goals and metrics, proper audience segmentation, and analyzing the results allow you to make informed decisions.

As a result, you can optimize strategies to achieve better results. Avoid common mistakes, improve conversions and marketing campaign performance, reduce costs, and deliver a better user experience.

Home
Up