Define Your Goals and KPIs
The very first thing you need to agree on is what the goals are for your CRO test, and how they’ll be measured.
Setting specific goals will clarify what you should focus on and remind you of what you’re trying to achieve. These can be as big or niche as you want.
These goals often look like:
- Increasing the number of leads by getting more form completions, demo requests or contact enquiries.
- Better user engagement by encouraging more content downloads, longer duration times or reduced bounce rates.
- More conversions from specific landing pages by influencing people to click buttons or share their details.
The important thing is that your goals support broader business objectives and are relevant to your audience.
You also need to make sure you know how to measure your B2B conversion rate, and that your website analytics tool is set up to do this. With GA4, for example, the out-of-the-box predefined events are very basic, which means you’ll need to define your own enhanced goals to track conversions more acutely.
Try The SMART Goals Framework
Whatever you choose, make sure your goals are SMART. That means they need to be:
- Specific
- Measurable
- Achievable
- Relevant
- Time-bound
For example:
The new CTA banner will increase clicks to the form page by 20% in two weeks.
The new product copy will increase subscription purchases by 5% in one month.
Define Your Hypothesis
Just like the science experiments you ran at school, your CRO tests also need to have a hypothesis before you can do anything.
It’s important because it helps you consider what the impact of your changes will be and check you’re tracking this properly.
Your hypothesis needs to be clear, specific and testable. There’s an easy formula you can use to help structure yours:
Because we observed [x]
we predict that by changing [x]
we will improve [x].
We will measure the impact by [x].
For example:
Because we observed the click-through rate on our book a demo button was below average on the landing page, we predict that by changing how we communicate the features of our service, we will improve the number of demo bookings. We will measure this by looking at the number of button clicks and the number of visitors who get to the demo booking page.
When you do this, make sure your reporting allows you to measure the metric you’ve focused on.
Define Your Test Type
There are two main types of website testing you can use for your CRO work:
- A/B testing will show half your traffic the original, control version and the other half will see the new experiment.
- Multivariate testing allows you to compare multiple versions of pages across a bigger split of traffic. For example, you could keep 40% of your traffic on the control page but show 30% one version and 30% another version.
For beginners, A/B testing is the easiest to understand and work with.
If you have a niche B2B website that doesn’t get a high volume of traffic, you may also want to consider other ways to test your website changes. Alternative methods like beta testing to a small group of users, qualitative assessment of heatmaps or session recordings, and usability testing can give powerful insights.
Define How Your Test Will Run
You also need to think about the mechanics of your CRO test, including how long it will run for and what tools you’ll use to try.
You need to decide:
- What CRO testing tool will you use? While it is possible to find free A/B testing with VWO, or free trials with other A/B testing tools, you’ll most likely need to find some budget for the right testing tool.
- How long will the experiment run for? You might decide to let the test run for a month, or you might have a high-traffic site, and you’ll get enough data over two weeks.
- When do the changes need to be live on your site? Don’t forget to consider key events, like B2B conferences or upcoming webinars, which could influence your timeline.
Set Up Your Experiment
Thanks to all your thorough planning, the testing part should be relatively straightforward—as long as you follow the right steps when setting up your experiment.
No matter which testing tool you use, you’ll most likely follow the same steps when setting up your experiment:
- Create a new experiment
- Define the experiment by specifying which URL(s) the test will run on
- Set up your variation(s) to test
- Configure your targeting and audience and decide how much traffic will go to each version of the page
- Select the goal(s) you want to measure
- Preview the test to make sure the versions display correctly
- Launch the experiment
- Achieve statistical significance of 95% or higher (the point at which you’re confident the same results will be replicated on the website). If you have low traffic, you may need to consider how to send more visitors to the test or review your goal measurement.
- Stop the experiment and analyze the results.
Keep An Eye on Your CRO Test
Your CRO testing tool should allow you to log in and observe the latest results. Occasionally checking in on your experiment can be a clever way to see how the data changes as your sample grows and may give you an indication of what the result could be.
Make sure you monitor the statistical significance of the experiment, too. This tells you how close you are to getting robust results that you can trust. Some tools will call it the p-value, in which case it needs to be under 0.05, while others will call it the confidence interval and it needs to be 95% or higher.
Other things to check during your experiment include:
- Traffic allocation. Is the test version being shown to people?
- Traffic volume. Are enough people seeing the page? How long will it take to reach statistical significance at your current traffic levels?
- Data collection. Does the tool show different results for each version?
- User feedback. Has anyone reported problems with the pages you’re testing?
- Error reports. Has there been an increase in the number of website errors?
Check For Unforeseen Consequences
The nature of CRO experimentation is that you don’t have control over what happens, and you may not have visibility of how the test is impacting behavior on pages outside of the experiment.
That means there’s always a chance you could negatively impact conversions on different pages, or further down the buying journey.
To counter this risk, it’s advisable to review your analytics data for the period of the test to check that there haven’t been any negative impacts that you didn’t anticipate.
Refer to your sales funnel and user journey research and think of all the pages that the test visitors may have seen after viewing the new page. If these weren’t included in your experiment, you should check that conversions performed as expected.
If you see an unexplained drop in conversions elsewhere, the test may have had an impact on behavior beyond that page. In this instance, it’s advisable to run the experiment again and include all the pages within the test.
Analyze Your Results
Your testing tool will tell you which version was the winner – but what do the results really tell you about your changes?
Depending which tool you use, you should be able to see standard data such as:
- How many visitors saw the test page?
- How many of those converted? What was the conversion rate?
- The improvement in conversion rate
- The confidence level of the results
However, it’s also helpful to dig a little deeper and understand things like:
- How did traffic from different sources behave?
- Was there a difference between mobile and desktop traffic?
- Do you notice anything interesting about how people from different countries behaved, or did old users convert differently from new users?
You may need to overlay the testing data with user behavior insights from your analytics tool to get a clearer understanding of this.
Plan How to Deploy the Winning Version
If the test had a marked improvement on conversion rates, it’s time to think about how you can apply those updates to the rest of your website traffic – and potentially across your entire site.
If you haven’t already involved your colleagues in the product/development team/agency, now’s the time to get them on board and ask for their support to implement your winning ideas.
It’s wise to stagger the rollout, if resources allow. By breaking up the work and only deploying changes to pages in specific areas of your website, for example, you’re able to continuously analyze performance and check the updates are favorable for those users.
Tips For Getting Technical Buy-In
Tech teams are always inundated with requests, so how can you get yours to the top of the list?
These tips could help:
- Communicate the value of the change, and the potential impact to the business. If the change was deployed to the whole website, what could that do to your total conversions? How will this help you meet the business’ goals?
- Involve the tech team as early as possible. Not only does this help foster a sense of collaboration, but they may have some fantastic ideas of things you could try.
- Celebrate wins together. When you share the success of the changes with the wider business, make sure you give your colleagues plenty of credit for helping to get the changes on the live website.
