How to A/B Test Adverts on a Small Budget

Most small-business advert tests don’t tell you anything. Someone runs two versions for three days, sees one has 12 clicks and the other has 9, and declares a winner. A week later the “winner” performs worse than the loser did.

The problem isn’t the idea. Testing is the best way to stop guessing, and AI adverts make it cheaper than ever to produce a second version. The problem is that a test needs a certain amount of data before the result means anything, and a small budget can only buy so much. This guide shows how to work out what your budget can honestly test, with a calculator, and how to run a clean test either way.

Key takeaways

• Small differences need big budgets. Detecting a 20% improvement can need several times more traffic than detecting a 50% improvement.
• Test bold changes. Two clearly different hooks or offers beat two near-identical adverts.
• Change one thing. Otherwise you won’t know what made the difference.
• Run whole weeks and decide the end date up front. Don’t stop when one version gets ahead.
• A tie is a result. If the difference is too small to detect, pick the cheaper or simpler version and move on.

Why Most Small Tests Fail

Three mistakes account for nearly every useless test:

1. Too little data. Results from a few dozen clicks swing wildly. A version can look 50% better purely by luck.
2. Too many changes. If version B has a new image, a new headline and a new offer, you can’t tell which one moved the result.
3. Peeking and stopping early. If you check every day and stop the moment one version leads, you’re very likely to stop on a lucky streak.

All three are about patience and planning, not clever tools. A test with a defined question, a defined end date and enough traffic is worth far more than five rushed ones.

What to Test, in Order

Work from the biggest lever to the smallest:

Order	What to test	Why it comes first
1	The first three seconds of the video, or the main image	If people don’t stop scrolling, nothing else matters.
2	The offer or main message	A stronger reason to act beats any design tweak.
3	The call to action	”WhatsApp us” or “Get a quote” can change who responds.
4	Format and length	A 15-second video versus a 30-second one, or a video versus an image.
5	Audience	Compare after the advert itself is working.

Our video ad sizes guide covers the formats, and our guide on how to brief an AI advert explains how to ask for two clearly different versions.

How Much Traffic Does a Fair Test Need?

The number depends on three things:

• Your baseline rate. If 5 in 100 people who click go on to enquire, your baseline is 5%.
• The improvement you want to detect. A 50% improvement means 5% becoming 7.5%.
• How sure you want to be. The standard is 95% confidence, meaning that if there were no real difference, you’d see a gap this big by chance only about 1 time in 20. The calculator also assumes 80% power, meaning that if the real improvement is at least the size you chose, you’d detect it about 4 times in 5.

As an illustration, if 5% of clicks become enquiries, spotting a 50% improvement takes roughly 1,500 clicks per version, while spotting a 20% improvement takes roughly 8,000. At an assumed R3 per click, that’s about R9,000 against about R49,000 in total. The same logic applies to click-through rate, where baselines are much lower and the traffic needed is higher still.

Try it with your own numbers below. All figures are yours to enter, so the defaults are examples only.

How to Run a Clean Test

1. Write the question first. “Does a price-led hook get more WhatsApp enquiries than a product-led hook?” is a test. “Let’s see what works” is not.
2. Change one thing. Same audience, same budget, same placement, same start time. Only the hook or the offer differs.
3. Pick one result to judge by. Enquiries, clicks or cost per enquiry, chosen before you start.
4. Split the budget evenly between the two versions.
5. Set the end date in advance and run whole weeks.
6. Don’t touch it. Edits can reset how the platform optimises that version.
7. Judge at the end. If one version won clearly, use it. If it’s a tie, choose the cheaper or simpler one.

Making Small Budgets Go Further

• Test bigger differences. Two completely different hooks need far less traffic than two slightly different headlines.
• Use a higher-funnel result. Click-through rate needs fewer events than enquiries, which need fewer than sales. It’s a weaker signal of what you ultimately want, but it can show which hook stops people scrolling.
• Test in sequence. Run the hook test first, then the offer test on the winner. It takes longer but each test is clean.
• Accept less certainty. A directional result at 90% confidence is better than guessing, as long as you treat it as a lean rather than a proof.
• Use platform tools with care. Built-in A/B tools are convenient, but the same rules about sample size apply.

Test Plan for Black Friday

If you want to go into Black Friday (Friday 27 November 2026) with a proven advert, work backwards from the date. Our Black Friday and festive season planner and marketing calendar cover the dates in detail. A simple schedule:

• This week: brief two clearly different versions of your main advert.
• Monday 26 October to Sunday 8 November: run a two-week test, with the budget split evenly.
• Week of 9 November: pick the winner and build the full set of Black Friday adverts around it.
• From 20 November: launch the teaser, then the main offer on the day.

Because AI adverts are produced in 3 to 5 business days rather than weeks, producing two versions to test fits easily into that window.

Test Ideas by Business Type

• Restaurants and cafés: a dish close-up versus a lifestyle scene. See restaurants and coffee shops.
• Salons and beauty: a price menu versus a simple “book this week” message. See hair salons and barbers.
• Online stores: product on white versus product in use. See AI product images for e-commerce.
• Services and trades: a problem-first hook versus a trust-first hook. See professional services.
• Bakeries: a custom-cake message versus an everyday-treats message. See bakeries and cake businesses.

For regulated industries, any variant has to comply with the same rules as the original. See our guides for medical and dental practices and financial services.

How to Measure Whether Adverts Are Working

Testing tells you which version is better. Measuring tells you whether the advert is worth running at all. Our complete guide to AI advertising covers the numbers to watch, and our guide to what AI video ads cost covers what to budget for production.

Getting Started

A Starter package from R3,000 gives you one video advert, three images and two revisions, which is enough to create two different versions of an advert to test. Growth, from R8,500 a month, adds weekly posting, which gives you a steady supply of fresh versions. See the full pricing breakdown.

As a South African agency for AI adverts, we’d rather help you run one clean test than five noisy ones. Get a free demo advert and tell us which two ideas you want to compare, with no cost and no commitment.

About the Calculator

The calculator uses the standard two-proportion sample-size formula for A/B tests (two-sided, 80% power), the same method most A/B testing calculators use. The R3 per click in the examples is an assumption for illustration, so replace it with the real cost from your own ads account.