How to A/B Test Responsive Search Ads: Step by Step and Tips

Advertisers who want to optimize performance have long used A/B testing to improve their ads. It’s well known how expanded text ads (ETA) do this.

But recently, Google has Announce It will cancel the ETA and make responsive search advertising (RSA) a Primary text ad format From July 1, 2022.

It’s time to revisit our best practices for ad testing.

What is an A/B advertising test?

Even novice PPC advertisers who do not have a clear advertising test goal may have used the function of adding multiple ads to an ad group to conduct quite a lot of A/B advertising testing.

This is done to allow Google to show products with better performance more frequently.

After all, optimized rotation is the default setting for ad rotation.

Google screenshot, December 2021

More advanced advertisers usually choose the option of rotating ads indefinitely and use their own methods to pick winners and losers. Instead of focusing on click-through rates, advertisers are more concerned with conversion rates.

But focusing only on conversion rates ignores the possibility that ads with high conversion rates may receive fewer clicks because of low click-through rates, thereby reducing the advertiser’s total conversions.

By combining the click-through rate and the conversion rate into one called Conversions per impression (CTR * conv rate = conv/imp), advertisers can better measure which advertisement can best achieve one of their goals (i.e. maximize conversion rate).

In the following example, you can see the traditional A/B test between RSA and ETA.

Based on performance data Optmyzr’s recent RSA research (Disclosure: My company) In the same ad group, the average click-through rate of RSA is 10% higher than that of ETA, but the conversion rate is 20% lower.

Combining these two metrics, RSA is 12% worse than ETA in converting impressions into conversions.

Author image, December 2021

This finding helps explain why some advertisers are still hesitant to fully adopt RSA: According to the traditional A/B metric, their performance seems to be worse.

Why traditional A/B testing methods are not suitable for RSA

But just as the conversion rate alone portrays an incomplete picture of which advertisement performs best, when RSA is added to the mix, conv/imp provides an incomplete picture.

This is where the problem lies. RSA is eligible for more impressions than ETA. According to our research, RSAs are much better at getting good ad ratings, and they can show about 4 times as many queries.

Under the same other conditions, the impression obtained by RSA is 4 times that of ETA.

RSA ads get far more impressions than ETA because they can get better ad rankings for more queries.

Author image, December 2021

“Impressions per ad” is calculated by dividing the number of impressions of a particular ad type by the number of ads of that type.

In this study, we separately analyzed ad groups with only RSA, ad groups with only ETA, and ad groups with two types of ads.

The numbers in the image above are based on more than 157,000 ad groups containing these two ad formats. Even if we look at an ad group with only one ad format, the number of impressions per ad group for the ad group with RSA is more than twice that for the ad group with only ETA.

Therefore, our old assumption that RSA and ETA only compete for the same impression pool based on the target keywords in the ad group is wrong.

Because RSAs get far more impressions than ETAs, even if they perform slightly worse on the conversion per impression metric, they are still more likely to drive more conversions.

RSA A/B testing must include impressions in the analysis

When RSA is part of the portfolio, a more accurate way to determine the winning ad is to include the number of impressions in the analysis, as shown in the image below.

Author image, December 2021

Any A/B advertising test, including RSA, should include impression data to enable incremental measurement.

As mentioned earlier, compared with ETA, RSA’s conversion rate per impression is 12% lower. We conservatively increased RSA’s double impressions, resulting in a 76% increase in conversions.

Therefore, if you care about getting a lot of conversions, RSA is the real winner.

When transferring your A/B advertising test from ETA to RSA, it is critical to include impressions in order to be able to get a real increase in RSA.

Side note: Conversions are usually not the real goal of advertisers, because profit is also important. As more and more advertisers now use automated bidding, this article will not explain how to ensure that these additional conversions are profitable.

This should be done by setting the correct goals and entering the correct conversion data into Advertising platform.

Even if you only conduct A/B testing on ETA, consider the number of impressions. Historically, this is more because we want to ensure that any advertisements we compare have an equal opportunity to prove ourselves.

If one ad is much newer than the other, it will show much less, thus invalidating the test. Or, if an ad only runs on weekends and other weekdays, there will be a similar discrepancy in impressions, rendering the test invalid.

Now, even if two RSAs run on the same day and have the same activity time, they may still produce a difference in the number of impressions, because one is better at qualifying for more auctions than the other.

Be sure to consider impressions in all evaluations.

Now that we have established a better way to measure results, let’s look at some ways to find out what to test when creating new ad variants to challenge existing ads.

Ad strength indicates whether you are following best practices

When you write a new RSA, Google will immediately provide feedback in the form of advertising strength indicators.

Ad strength is a best practice score that measures the relevance, volume, and diversity of responsive search advertising content, even before the RSA launch.

According to Google, each increase in advertising intensity will increase the number of clicks by approximately 3%.

Therefore, going from “poor” to “average” should increase clicks by about 3%, and going from “average” to “good” will increase clicks by another 3%.

Google screenshot, December 2021

Advertising intensity is a best practice indicator and should only be relied upon when creating a new RSA. But the point is that this is a “best practice” score and has nothing to do with actual performance.

Just as there may be a low-quality scoring keyword that brings valuable conversions to your account, you may also have an ad with weaker ad strength that can beat other ads with higher scores.

Try to follow Google’s guidelines so that you can start your next test with the greatest chance of winning.

But don’t get into trouble by trying to get full marks. Actual performance will be reflected in your asset performance grouping tags, let’s take a look at these tags next.

Asset performance labels reflect your performance

Once your RSA has accumulated approximately 5,000 impressions at the top of the search page in the past 30 days, the asset tag will provide you with guidance on which texts are performing well and which texts you should consider optimizing.

Google publishing Useful table of meaning of different performance grouping labels:

Google screenshot, December 2021

The most important thing is to replace all assets with a “low” label, and replace assets that have not received any impressions for more than two weeks.

Remember, what you see in this asset report is based on actual performance. Optimizing based on this rather than ad strength score is more important.

Test ideas

Many ads will not accumulate enough top impressions in a 30-day window to show performance data, but if this is your case, there are still ways to optimize.

Keep in mind that Google allows up to three RSAs per ad group, and they allow specific text to be pinned to different locations in the ad.

With this, you can construct some useful experiments.

Google screenshot, December 2021

Advertisers can now pin multiple assets to one location in RSA advertisements.

The first experiment tried was based on message passing.

Create multiple RSAs, each with different core information. The key information element of any advertisement, whether it is RSA or ETA, includes the company name, unique value proposition, and call-to-action.

Value proposition and call-to-action are the best attempts, because your brand is your brand, and for most advertisers, there is not much room for creativity.

For example, the first RSA may be more focused on the large number of choices your website offers, while the second may focus on another value proposition, such as how highly rated your website is by past customers.

Then, when you use the A/B testing process, you will discover which information resonates more and which information leads to more conversions.

The second experiment took messages that worked well and tested them in different locations.

You can do this by pinning a set of assets to different locations, such as a specific title or description.

Google now allows multiple assets to be pinned to one location, so even if your value proposition has 4 assets, you will pin all 4 assets to the desired location. Now, when you run an A/B test, you may find that it performs best when you fix the value proposition in the title 2 position.

Author image, December 2021

fixed

Whether it is fixed or not is one of the biggest debates in RSA. After the ETA ends, you can only create RSA from July 1, 2022, and advertisers can fix a piece of text to each position to effectively convert RSA to ETA.

But this may defeat the entire purpose of RSA, which is to allow the machine to help dynamically build the best advertisements for each user, so that the advertisements are more relevant and eligible to appear in more searches.

In some industries, fixation is a legal requirement. But beyond that, here is an analysis of fixation and whether it helps (or hurts) performance.

Author image, December 2021

In the final analysis, fixation is about advertiser control. What the data shows us is that when advertisers let the algorithm do their thing without interference, the machine will bring the best click-through rate. In this case, the cost-per-click and cost-per-conversion are also lower.

However, when advertisers want control, they’d better provide the machine with at least 2 text variants for each fixed location.

Pinning a piece of text to a certain location has the greatest negative impact on clickthrough rate, cost-per-click, and cost-per-conversion.

Fixing multiple assets seems to provide a good balance of control and performance.

Title change

Ad strength metrics take into account the number of variations you provide for headlines and descriptions. Since it can be daunting to come up with 15 titles, we analyzed whether it is really worth adding multiple titles to each RSA.

Author image, December 2021

We found that ads with more headline variations resulted in higher conversions per ad. So when A/B tests RSA, more titles are indeed better.

in conclusion

If you only remember one thing in this article, it is that the basic assumptions about advertising testing must change. The number of impressions is now also largely determined by the ad, not just the keywords of the ad group.

When conducting A/B tests on RSA ads, it is important to consider this potential imprint difference and try to find the ads that drive the largest increase.

Over time, we expect that Google will introduce more metrics for RSA assets, which will help all advertisers further build the optimization techniques described in this article.

More resources:

Featured image: Jane0606/Shutterstock