Most video testing produces no usable information. Not because the tools are bad but because the tests are constructed in ways that cannot answer a question. Here is how to construct ones that can.
Test ideas, not variations of one idea
This is the mistake that wastes the most budget. Two versions of the same video with different music, or the same video cropped two ways, are not two ideas. Whichever wins, you have learned nothing about your customers, because both were making the same argument.
A real test compares different arguments:
- The problem it solves against the moment it is used.
- A demonstration against a comparison.
- Leading with the price against leading with the feature.
Each generation on Artvizon plans its own sequence rather than recolouring the previous one, so asking for a different angle gives you a genuinely different ad. That is the input a test needs.
Test one thing at a time
If the two ads differ in hook, pacing, length and call to action, a winner tells you nothing about why. Change the variable you care about and hold the rest.
The exception is the very first test, where you are looking for direction rather than explanation. There, two completely different ads are fine, because the question is "does anything work" rather than "which element mattered".
Run it long enough, and know why
The temptation is to call a winner on day two. Early differences are almost always noise: small numbers, uneven delivery, one audience segment happening to see one version first.
Let each variant accumulate enough conversions that the difference is larger than the day-to-day fluctuation you would see between two identical ads. If you cannot state the difference in a sentence without hedging, it is not a result yet.
What to measure
Cost per acquisition first, because it is the number the business runs on.
Watch-through as a diagnostic, not a goal. High completion with poor conversion means an entertaining ad that does not sell; low completion with good conversion often means an ad that qualifies hard and early, which is fine.
Frequency alongside cost per acquisition. Rising frequency with rising cost is creative fatigue, not a losing ad, and the correct response is to rotate rather than to conclude.
Ignore views and engagement. They go up when a video is merely seen.
The loop that compounds
- Two or three genuinely different ads.
- Modest, equal budgets, run long enough.
- Keep the winner as the benchmark.
- Generate new challengers against it.
- Repeat.
The loop only survives if producing a challenger is cheap. When a variant needs a shoot, testing stops after the first round, which is why most small advertisers run one ad until it dies.
Length is worth testing separately
The €5 entry offer produces a short single clip of about 12 seconds; the paid plans produce multi-chapter videos of 24, 32 or 36 seconds. Longer is not better by default. The useful test is whether the extra seconds are adding an argument or restating the first one, and the audience will tell you faster than your judgement will.
Related reading: Facebook and Instagram video ads and Social media ads.