YouTube A/B Testing: The Complete Guide (2024)

Article last updated on:
July 06, 2024

This post covers everything about A/B Testing thumbnails + titles on YouTube. We’ll go through YouTube’s Test & Compare feature and a few other options.

Join me 🙌

Key Takeaways

FactorA/B Testing
Available nowYes
ToolsThumbnail Test, YouTube Test & Compare
Works forBoth thumbnails and titles
Cost$0 → $29/month
OutcomeIncrease CTR by +65%

Let’s go deeper and break down everything.


The Basics

1. What does A/B testing mean on YouTube?

Answer: A/B testing on YouTube means uploading thumbnails/titles for a video and comparing them to see which performs better.

It’s used to optimize videos for higher engagement and viewer retention.

Illustration of thumbnail testing on YouTube
Visualisation of Thumbnail A/B Testing

Disambiguation: “YouTube A/B Testing” can refer to creators testing multiple thumbnails per video, or to YouTube experimenting with new features.

When YouTube is experimenting with features, it means that:

  1. They may release a new feature, but…
  2. Only some creators get (this happened with Test & Compare too)
  3. While others don’t until they release it completely, if at all

YouTube experiments with multiple features to see how they’re perceived and how they perform — if they’re doing good, they’re likely to release it to the public

If not, they’re likely to drop the feature.

2. Does YouTube offer A/B testing?

Answer: Yes, YouTube does offer A/B Testing as of June 11th, 2024. It’s a feature for content creators in YouTube Studio, called Test & Compare — right now, it only works for thumbnails.

Here’s what it looks like:

Screenshot from YouTube's Test & Compare feature
Screenshot from Test & Compare

External tools like Thumbnail Test can A/B test both thumbnails and titles at once, while Test & Compare only works for thumbnails (at least right now).

3. How to get YouTube A/B thumbnail testing?

Answer: To get thumbnail A/B testing on YouTube, go to the “thumbnail” section in YouTube Studio (or when uploading a video). Click the 3-dot icon and select “Test & Compare”.

Not every channel has access to Thumbnail A/B Testing.

That’s why we’ve made Thumbnail Test → a tool that allows everybody to A/B test their videos, regardless of their subscriber count.


How to Run an A/B Test on YouTube

Answer: You’ve got two options. Either use Thumbnail Test for thumbnails and titles, or YouTube’s Test & Compare feature just for thumbnails.

Since not every user has access to YouTube’s official tool, you’re better off with Thumbnail Test. Speaking of which, here’s how the two features compare.

1. With Test & Compare

Answer: Go to YouTube Studio, and select one of your videos. In the ‘thumbnail’ section, you want to click ‘Test & Compare’ and upload up to 3 thumbnails.

Watch this 2-min video to see the process:

VIDEO: How to A/B Test on YouTube

Here’s a step-by-step list:

  1. Go to YouTube Studio
  2. Choose the video you want to A/B test
  3. Click the 3-dot icon on its thumbnail
  4. Select “Test & Compare” and upload your images
  5. Click “DONE” and “SAVE”

That’s right about it. Now, let’s go to the second option.

2. With Thumbnail Test

Answer: To run an A/B Test on YouTube, go to ThumbnailTest.com and connect your channel. Select a video and choose which test to run (thumbnail, title, or both).

Beginning a thumbnail A/B test in Thumbnail Test
  1. Open Thumbnail Test
  2. Connet your channel
  3. Select a video and start a new test
  4. Run & Wait for the test’s analytics

You’ll see how many clicks each thumbnail/title brought to your video as soon as we start collecting data (which is usually in under a day).


Top 3 Apps for YouTube A/B Testing

Currently, these are the top 3 A/B testing apps for YouTube: Thumbnail Test, Test & Compare, and TubeBuddy.

We’ll cover the basics for each, as we’ve explored them thoroughly here.

1. Thumbnail Test

Thumbnail Test Homepage
Homepage of Thumbnail Test

Pricing: Starts at $29/month on the Basic Plan.

ProsCons
Works for thumbnails + titlesNot YouTube’s official tool
Clear analyticsPaid option
Test +10 thumbnails per video
Used by the most popular channels
Discount for small creators

One of the greatest parts about Thumbnail Test is that it offers a 40% discount for channels under 10k subscribers (visible on the pricing page).

2. Test & Compare

Release Announcement

Test & Compare is actually YouTube’s official tool, though it’s still not available to every content creator… To this date!

It has been postponed for +5 years now, and even after its release, people aren’t entirely happy with it.

ProsCons
YouTube’s official toolJust ‘watch time’ in analytics
FreeTest just 3 thumbnails/video
Works for old videos tooCan’t A/B test titles
Not available for everybody
Bad customer support

Overall, YouTube’s feature is great just because it’s free — in the end, it’s still a pretty limited A/B Tester when you put it in contrast with Thumbnail Test.

3. TubeBuddy

TubeBuddy’s homepage
ProsCons
Test thumbnails + titlesA/B Testing is paid, not free
Helps with SEO by suggesting relevant keywords and tags.Requires a separate license for each YouTube channel.
Provides time-saving features like bulk editing, card templates, and end screen automation.Some users report occasional slowness and connection problems.

TubeBuddy remains a pretty basic but working A/B testing tool, however their #1 goal isn’t A/B Testing.

This app is rather focused on YouTube SEO.

Your best bet: Thumbnail Test remains the #1 A/B Tester for all YouTubers, regardless of their channel size.


FAQs

1. Does YouTube let you A/B test thumbnails?

The answer is yes. YouTubes lets you A/B test up to 3 thumbnails per video, while Thumbnail Test (external app) lets you test over 10 per video.

2. Can you A/B test titles on YouTube?

The answer is yes, but not with YouTube. A third-party tool called Thumbnail Test lets you A/B Test as many titles as you need per video.

YouTube’s Test & Compare feature doesn’t work for titles, it only lets you A/B test thumbnails — that’s why Thumbnail Test is more powerful, because it works for both at a time.

Conclusion

Today you’ve learned everything about A/B testing on YouTube. We went through all you have to know — from what it is, to which app to use.

Hope you found just what you needed!


Thank you for reading this,
Thumbnail Test

About the author

David is the head of the editing team at ThumbnailTest. With his help, the editorial team is able to provide you with the best free guides related to YouTube thumbnails and A/B testing.