Last updated: October 2026
Landing page A/B testing is the practice of showing two versions of the same page to different visitors at the same time, then comparing which one gets more of the action you want, such as a form submission, a call or a purchase. Done well, it replaces opinions with evidence. Done badly, it produces confident-looking numbers that mean nothing, and in the worst case it quietly damages the page's search visibility.
This guide covers the whole job: what is worth testing, how to write a hypothesis, why one change at a time usually beats five, what sample size and statistical significance actually mean, the mistakes that ruin most tests, and the part most guides skip, which is how to run a test without hurting your rankings. Every example here is a hypothetical, built to explain the method. None of them are real results.
What Landing Page A/B Testing Actually Is
In a split test, visitors are divided at random into groups. One group sees the original page, called the control. Another sees a variation that differs in some deliberate way. You measure one outcome for both groups and compare. Because the groups run at the same time, outside factors such as the day of the week, a holiday or a news story affect both equally, which is the main reason this beats the old method of changing a page and comparing this month against last month.
Google describes the idea in plain terms in its own documentation, where A/B testing is "where you test two (or more) variations of a change." It separates this from multivariate testing, where several kinds of change are tested at once. For most sites, and certainly for most landing pages, the simpler version is the right one.
A landing page is also a good place to test because it has a single job. Unlike a homepage, which serves many visitors with many goals, a landing page usually exists to get one action. That makes the success measure obvious and the result easier to read.
Decide Whether the Page Is Ready to Be Tested
Not every page should be tested yet. A test needs enough visitors and enough conversions for a difference to show up above the ordinary noise. A page that gets a few dozen visits a week will take so long to produce a readable result that the market, the offer or the season will have changed before it finishes.
If traffic is low, there are better uses of the time. Fix the obvious problems first: a vague headline, a slow page, a form that asks for too much, a button nobody can find. Talk to a few customers, watch how people use the page, and make the change you are already fairly sure about. Reserve formal testing for pages with steady traffic, where a wrong guess is expensive and the right answer is genuinely unclear.
Before any test, check the basics are working. Confirm that conversion tracking records every real conversion, that the form works on phones, and that the page loads properly in the browsers your visitors use. A test on top of broken tracking only measures the breakage.
Start With a Hypothesis, Not a Hunch
A good hypothesis has three parts: what you observed, what you will change, and what you expect to happen and why. Writing it down before the test starts protects you from the habit of finding a story in the data afterwards.
Here is an illustrative example. Suppose visitors to a local service page scroll past the introduction but rarely reach the contact form. A workable hypothesis would read: "Because visitors do not see a way to get in touch without scrolling, moving a short enquiry form above the fold will increase the share of visitors who submit it." Notice that it names a reason. If the test loses, you learn that the reason was probably wrong, which is more useful than learning only that a variation lost.
Pick one primary measure for each test, decided in advance. If you look at ten measures afterwards, one of them will almost always look like a winner by chance alone.
What to Test on a Landing Page
Start with the elements that shape the decision to stay or act. In rough order of how often they matter:
- Headline: does it say what the visitor gets, in their words, within a couple of seconds? Test clarity before cleverness.
- Offer: a free quote versus a free consultation, a guarantee, or a clearer statement of what happens next. Changing the offer usually matters more than changing the wording around it.
- Call to action: the text, the position and how prominent it is. Google itself notes that changing the text of a "call to action" can have a surprising impact on how users interact with a page.
- Form length: fewer fields reduce effort but can bring in lower-quality enquiries, so judge by qualified leads, not just submissions.
- Social proof: where reviews, ratings or testimonials appear, and how specific they are. Only use real ones.
- Page speed: a slow page loses visitors before the headline is read, and it is the one change that helps both conversions and search.
- Layout and imagery: a long page versus a short one, a photo of the team versus a generic image.
The best test ideas come from evidence rather than a brainstorm: support questions, session recordings, survey answers and the drop-off points in your funnel. Our on-page SEO checklist is a useful companion here, since many of the same elements, such as the headline and the heading structure, serve both the visitor and the search engine.
One Change or Many?
The cleanest test changes one thing. If you rewrite the headline, swap the image, shorten the form and recolor the button all at once, and the variation wins, you do not know which change did the work. One of them might have helped a lot while another quietly hurt.
Testing several changes together makes sense in two situations. The first is a full redesign, where you are really testing a new page against the old one and accept that you will not learn which element mattered. The second is true multivariate testing, which needs far more traffic because every combination needs its own audience. For a typical small or mid-sized site, a series of single-change tests, each building on the last, is slower to plan but far easier to learn from.
Sample Size and Statistical Significance in Plain Words
Two versions of a page will almost never convert at exactly the same rate, even if they are identical. Chance alone makes one look slightly ahead. Statistical significance is a way of asking: if the two versions were really the same, how surprising would a gap this big be? If it would be very surprising, you can be fairly confident the difference is real. If it would be quite ordinary, the gap is probably noise.
Sample size matters because small samples are noisy. Flip a coin four times and getting three heads tells you nothing. Flip it a thousand times and a lopsided result starts to mean something. The same applies to visitors and conversions. What counts is not only the number of visitors but the number who actually converted in each version, because a page that rarely converts needs many more visitors before a gap can be trusted.
Two other ideas help. A bigger real difference is easier to detect than a small one, so a test aiming to catch tiny improvements needs much more traffic than one looking for a large jump. And the confidence level you choose is a trade-off between how sure you want to be and how long you are willing to wait. Rather than quote a magic number, use a sample size calculator before you start, enter your current conversion rate and the smallest improvement that would matter to your business, and let it tell you roughly how many visitors you need.
Decide the Duration in Advance
In landing page A/B testing, this is the single habit that prevents most bad conclusions.Before the test goes live, decide how long it will run, or how many visitors it needs, and write it down. Then do not end it early because the numbers look good.
A sensible plan also covers whole weeks. Visitor behavior often differs between weekdays and weekends, so a test that runs for a few days and stops on a Tuesday can overweight one pattern. Running in full-week blocks smooths that out. If your business has monthly or seasonal swings, allow for them or avoid testing during unusual periods such as a major sale.
The aim is not to run tests for as long as possible. Longer is not safer, as the search section below explains. The aim is to run for exactly as long as your plan says, then read the result once.
Common Mistakes That Ruin a Test
Stopping early. A variation often looks like a runaway winner in the first days, then drifts back toward the original as more visitors arrive. Ending the test at the high point locks in a false result.
Peeking. Checking the dashboard daily and stopping the moment it says "significant" is a form of the same mistake. Each look gives chance another opportunity to produce a convincing gap. If you want to check for technical faults, look at whether the test is working, not at who is winning.
The novelty effect. A new design can get extra attention simply because it is different, and returning visitors may click on a changed button out of curiosity. That boost fades. Running the test long enough, and looking separately at new and returning visitors, helps you spot it.
Testing low-traffic pages. As covered earlier, a page without enough conversions cannot produce a trustworthy answer in a reasonable time.
Changing things mid-test. Editing the offer, the price, the traffic sources or the page itself while the test runs mixes new conditions into old data. Freeze everything else until the test ends.
Ignoring the quality of the result. A shorter form may raise submissions and lower the number of customers who actually buy. Follow the outcome as far down the funnel as you can.
Treating a loss as a failure. A variation that loses still teaches you something about your visitors, particularly if the hypothesis was written down.
Illustrative Test Examples
These are hypothetical, written to show how a test is framed. They contain no results, because the point is the method.
- Headline clarity. Control: a headline about the company's mission. Variation: a headline that states the service and the area served. Hypothesis: visitors decide faster when the first line says what the page is for.
- Form length. Control: a form with six fields. Variation: name, email and a short message only. Hypothesis: removing optional fields increases submissions. Measure qualified leads as well as raw submissions.
- Call to action. Control: a button reading "Submit". Variation: a button describing the outcome, such as "Get my quote". Hypothesis: a label that names the benefit reduces hesitation.
- Social proof placement. Control: reviews at the bottom of the page. Variation: one specific review beside the form. Hypothesis: reassurance helps most at the moment of decision.
- Offer. Control: "Contact us". Variation: "Free 15-minute review of your page". Hypothesis: a specific, low-commitment first step feels easier to accept.
The SEO Side: Running Tests Without Hurting Rankings
Google has a dedicated help page for this, and it is short enough to follow closely. It explains that the page is about ensuring test variations have minimal impact on Google Search performance, and it is reassuring on one point: small changes, such as the size, color or placement of a button or the text of a call to action, often have little or no impact on a page's search snippet or ranking. Its guidance on A/B testing and Google Search then lists three best practices.
1. Do not cloak. Google says not to show one set of URLs to Googlebot and a different set to humans, calling this cloaking and saying it is against its spam policies. Its spam policies define cloaking as presenting different content to users and search engines with the intent to manipulate rankings and mislead users. A test that splits real visitors at random is fine. A setup that sends crawlers to one version and every person to another is not. Do not identify Googlebot and treat it differently.
2. Use rel="canonical" on variant URLs. If your test serves the variation from a separate URL, Google says to use the rel="canonical" link attribute on all alternate URLs to indicate the original URL is the preferred version. That keeps the test pages from competing with, or replacing, the page you want in search results. If you test by changing content dynamically on a single URL with JavaScript, there is no second URL to canonicalize.
3. Use 302 redirects, not 301. If your test redirects some visitors from the original URL to a variation, Google says to use a 302 (temporary) redirect, not a 301 (permanent) one, so search engines understand the redirect is temporary. Google's redirects documentation explains the difference: temporary redirects show the source page in search results, while permanent ones show the new target.
There is a fourth point that connects back to duration. Google tells site owners to run the experiment only as long as necessary, and, once the test is concluded, to update the site with the winning content and remove all elements of the test as soon as possible. It also warns that serving one variant to a large share of users for an unnecessarily long time could be seen as an attempt at deception. In practice, that means ending the test on schedule, redirecting everyone to the winner, deleting the losing variant and its test code, and not leaving old variant URLs live.
One more practical note from the same page: Googlebot generally does not support cookies, so a test controlled by cookies will show only the non-cookie version to Google. That is another reason to keep the page you want indexed as your control, and to check that your main page is the one Google sees.
Speed, Headings and Links Are Test Inputs Too
Some of the most reliable improvements are not variations at all. They are fixes. A heading structure that matches the page's purpose helps a visitor skim and helps search engines understand the page, and you can audit it quickly with our heading checker. A page that loads slowly will lose visitors whichever headline you test, so check speed before you test copy. If you are just starting out and deciding where limited effort should go, our guide to SEO for startups helps you sort the basics from the experiments.
A Checklist for Your Next Test
- Confirm the page gets enough visitors and conversions to read a result in a reasonable time.
- Check that conversion tracking, forms and page speed work properly on mobile and desktop.
- Write a hypothesis that names the observation, the change and the reason you expect it to work.
- Choose one primary measure and one change.
- Estimate the sample size needed, then decide the duration in advance, in full weeks.
- For separate URLs, add rel="canonical" pointing to the original, use 302 redirects, and show crawlers the same thing as users.
- Do not edit the page, the offer or the traffic sources during the test.
- Read the result once, at the planned end, and look at lead quality as well as volume.
- Apply the winner, remove the losing variant and all test code, and record what you learned.
Quick Recap
- Landing page A/B testing compares two versions of a page shown at the same time to randomly split visitors, measuring one outcome.
- Test pages with enough traffic and conversions; fix obvious problems on low-traffic pages instead.
- Write a hypothesis with an observation, a change and a reason. Change one thing at a time unless you are comparing a full redesign.
- Statistical significance asks whether a gap is bigger than chance would normally produce. Use a sample size calculator before you start.
- Decide the duration in advance, run full weeks, and do not stop early or peek.
- For SEO, do not cloak, use rel="canonical" on variant URLs, use 302 redirects, and run the test only as long as necessary.
- When the test ends, ship the winner and remove every trace of the test.
Frequently Asked Questions
What is landing page A/B testing?
It is a method of showing two versions of a landing page to different visitors at the same time and comparing which produces more of a chosen action, such as form submissions or purchases.
Does A/B testing hurt SEO?
Not if it is set up properly. Google's guidance says to avoid cloaking, use rel="canonical" on variant URLs, use 302 redirects rather than 301 for redirect-based tests, and run the test only as long as necessary.
Should I use a 301 or 302 redirect for a test?
A 302. Google says a temporary redirect tells search engines the redirect is temporary, whereas a 301 signals a permanent move and may lead Google to show the variation instead of your original.
How long should a landing page test run?
Long enough to reach the sample size you planned, in full weeks, and no longer. Decide the duration before launch and stick to it. Google also advises running experiments only as long as necessary.
How much traffic do I need?
It depends on your current conversion rate and the smallest improvement you care about. Use a sample size calculator before the test. Pages with very few conversions are usually better improved through direct fixes than through testing.
Can I test more than one change at once?
You can, but you will not know which change caused the result. Testing one change at a time gives clearer learning. Multivariate tests need much more traffic because every combination needs its own audience.
What does statistical significance mean?
It describes how unlikely a gap this large would be if the two versions were actually the same. A significant result suggests the difference is probably real rather than random noise, though it is not a guarantee.
Why is peeking at results a problem?
Each time you check and are willing to stop, you give chance another opportunity to produce a misleading lead. Stopping the moment a result looks significant makes false winners much more likely.
What is the novelty effect?
It is a short-term boost a new design can receive simply because it looks different. It fades over time, so a test that is too short can overstate how well a variation really performs.
What should I do when the test ends?
Apply the winning version to the original URL, remove the losing variant and any test code, delete or redirect leftover variant URLs, and record what the result taught you about your visitors.
If you would rather have someone check that your tests, redirects and canonical tags are set up safely, Contomatix can review it as part of an SEO audit.
