The Beginner’s Guide to YouTube A/B Testing

Article last updated on:
July 06, 2024

Want to get into A/B Testing on YouTube? This starter’s guide has everything you need to begin testing your videos.

Pro Tip: Click “Show Sections” to easily navigate this guide.

Highlighting where to click to toggle the Table of Contents on a Thumbnail Test.com guide
Click there to open the Table of Contents

Chapter I: The Basics

We’re not going to bore you with stuff you don’t know…. But, to understand YouTube A/B Testing, you need to know the roots of this principle.

1. What is the A/B Testing concept?

Answer: A/B testing (or split testing) is a method used to compare two versions of something (e.g. a YouTube video) to determine which one performs better.

“Comparing the outcomes of two different choices (A and B) by running a controlled mini-experiment.”

Harvard Business School

Got it? Let’s move on to our context.

2. What is A/B Testing on YouTube?

Answer: A/B testing on YouTube is when creators or advertisers experiment with different versions of their videos to see which one performs better with viewers.

Illustration of thumbnail testing on YouTube
This is what Thumbnail Testing looks like

YouTubers might test different thumbnails, titles, descriptions, or even content itself to figure out what resonates most.

It’s like a trial-and-error method to optimize engagement and reach on the platform.

3. Why do YouTubers A/B Test?

Answer: YouTubers A/B test to improve their content’s performance and audience engagement.

Here is Nolan from Think Media to explain this (in ~20 sec):

By experimenting with different elements like thumbnails, titles, or content, creators can learn what resonates best with viewers and optimize their videos accordingly.

Ultimately, it helps them attract more views, subscribers, and revenue.

Fun Fact: A/B testing works for old YouTube videos too, not just for new uploads.


Chapter II: How to A/B Test

To start A/B testing on YouTube, make multiple versions of a video’s thumbnail and title — don’t hesitate making them very different from each other.

This help you understand elements bring more views.
  • So you got the main idea…
  • Now you wanna start testing yourself.

Join us, and let’s see your current options to A/B testing.

1. How to Test a Thumbnail

Answer: To A/B test a YouTube video’s thumbnail, go to ThumbnailTest.com and connect your channel. Select a video, upload thumbnails, and begin the test.

Upload thumbnails to A/B Test them for YouTube
  1. Open Thumbnail Test
  2. Upload (at least) 2 thumbnails
  3. Set the thumbnail changing frequency: Daily or Hourly
  4. Run the test

Few hours/days later, you can come back to this test and check the video’s analytics to see how many clicks you get for each thumbnail.

This’ll help you figure out which thumbnail drives a better CTR for the video. More importantly, Thumbnail Test will help you understand why that happens.

2. How to Test a Title

Running a Title Test in Thumbnail Test
  1. Go to ThumbnailTest.com
  2. Select which video you want to test
  3. Type in a few titles (or choose AI-written options)
  4. Run the test

Just like on the thumbnail A/B test, you’ll see analytics for this test as well.

3. Which Software to Use

You’re probably wondering: “Is Thumbnail Test really is the only A/B Tester?” — without biasing this, the real answer is Yes.

Reason being that YouTube has cancelled this feature for over 5 years now, which doesn’t give people another option.

Redditors being excited for YouTube A/B Testing in 2019
Screenshot was taken in February 2024 via Reddit

And that’s just one Reddit comment. There’s more like that.

Besides, here’s how YouTube compares to Thumbnail Test:

FactorYouTubeThumbnail Test
PlatformYouTube StudioConnects with YouTube
TestingThumbnails, TitlesThumbnails, Titles
Quantity3 per video10 per video
CostFree, part of YouTubePaid, discount for small channels
AvailabilityNot releasedAvailable for everyone
DurationLimited test durationFlexible test duration + frequency
SetupEasy, directly in YouTube1-click connection to YouTube
SupportNone, just articlesLive Support, Email, Community
YouTube vs Thumbnail Test

This comparison is based on what we know YouTube’s feature will be like and comments from fellow beta testers who already have access to the feature.

Winner: Thumbnail Test.

As new A/B testers come out, we’ll update this guide to let you know the 100% honest answer.


Chapter III: Real Examples

By now, you may ask: “What is an example of AB testing?” → This section covers an honest example of a successful A/B testing case.

This story can also be seen as a testimonial. Fear not, we’ll break it down to the points of interest for a beginner in A/B Testing.

1. Famous YouTuber

Here’s Ali Abdaal to explain how his video got 800k views just by changing the thumbnail:

No need to read the whole post (tweet) if you don’t have the time for it.

2. How he did it

Ali Abdaal’s video views went from 300,000 to 1.1 million after changing the thumbnail.

  • A/B Testing: The success wasn’t random; it came from A/B testing different thumbnails.
  • Viewer Engagement on YouTube: To watch a video, viewers must first click on it. So, good thumbnails are crucial for attracting clicks.
Ali Abdaal's stats on a Thumbnail Test

So then… Titles + Thumbnails serve as packaging, enticing viewers to watch the video.

3. The Outcome

Screenshot of Ali Abdaal's outcome of using ThumbnailTest.com

Making the perfect thumbnail requires creativity and an understanding of audience preferences.

Ali’s experience shows the importance of optimizing thumbnails for increasing engagement metrics and channel growth.


FAQs

1. Does YouTube allow A/B Testing?

The answer is yes, but not for everybody. YouTube rolled out Test & Compare in June 2024, but only for what they call “eligible creators”.

  • YouTube allows A/B Testing only for beta testers and *some* channels with lots of subscribers.
  • That’s why people use Thumbnail Test instead — it is available for everybody.

2. What is the YouTube Thumbnail Tester feature?

Answer: YouTube Thumbnail Tester (Test & Compare) is a feature designed to help you upload and cycle 3 different thumbnails for one video.

You’d do this to find out which thumbnail works better for your audience → which image brings you more clicks, and why.

3. Can you A and B test a YouTube Thumbnail?

Yes, you can A and B test a YouTube Thumbnail, but YouTube’s feature is not available to the public.

Feel free to use Thumbnail Test instead.


The Bottom Line

  • Today we learned what is meant by YouTube A/B testing…
  • And how to start testing ourselves

Got any questions? Get in touch with us by clicking the purple bubble in the bottom-right corner of this page.

You’ll get connected to one of our support team’s members. We’re always happy to help!

Thank you for reading this,
David Ch
Chief Editor of Thumbnail Test

About the author

David is the head of the editing team at ThumbnailTest. With his help, the editorial team is able to provide you with the best free guides related to YouTube thumbnails and A/B testing.