In better economic circumstances, we marketers are put on a long leash to get results. Hilariously irreverent ad campaigns. Wacky publicity stunts. Even ultra-creative emails—all in the name of seeing what sticks and scales.
But right now? Inflation is high. Interest rates continue to rise. And, as such, consumer behaviors are changing. And marketers across the board are being asked to do more with less. In some cases, a lot less. Whether or not we’re in the throes of a recession, it’s still our job to deliver results.
The headwinds are real.
The question is: so how well will you adapt your approach?
Enter: incremental experimentation. A risk-mitigated, resource-efficient, methodical way to eke out additional value from the tactics you’re already using today. By conducting small, controlled experiments on ad campaigns, emails, websites, and landing pages, we can gain valuable insights into what works and improve our results over time.
Instead of swinging (and…yeah, often missing) for the fences, we focus on “base hits”' and we use rigorous, results-oriented experimentation to get us to our goals.
Google Optimize will go offline on September 30, 2023
That means if you’re running any experiments beyond that date, you won’t be able to access any of that data. We’re partnering with Optimizely—the world’s leading optimization platform—to help you make that transition. Optimizely is easy to implement and use, so you’ll be able to continue running experiments without missing a beat.
What is incremental experimentation?
Incremental experimentation is a method of conversion rate optimization that involves making small and gradual changes to campaigns based on data and testing. The goal of incremental experimentation is to improve the user experience and increase conversions over time, without risking large losses or negative impacts.
By testing elements such as subject lines, headlines, images, forms, copy, and CTAs, and measuring their effects on key metrics like bounce rate, click-through rate, and conversion rate, you can improve the performance of your campaigns throughout the funnel over time, with very little risk.
Incremental experimentation can also help you learn more about your audience’s preferences and behavior, and use those insights to inform future decisions and strategies.
Incremental experimentation vs. radical experimentation
If incremental experimentation is about “base hits”, then radical experimentation is like trying to hit a home run on every swing. Making small changes might be less exciting than the odd homer. But you’re more likely to win the game.
More practically, where incremental experimentation is about making small changes that you can quickly test, radical experimentation is about making big, dramatic changes based on assumptions or intuition. Radical experimentation can sometimes lead to breakthrough results, but it can also be risky and costly if the changes backfire. Even in the best case scenario, you never quite know what it is that worked. It’s not the best strategy to work from when times are tight.
"Organizations often aim too large with their experimentation practice, forgetting to factor in that your user engages in a series of events on your website, and can be measured in these increments. This allows you to build toward Optimization through incremental improvements. Too-large of changes when testing your UX create attribution haziness and ineffective results"
Jeff Cheal, VP of Product for Optimizely
If you’re looking for a way to minimize risk and improve ROI on your campaigns, incremental experimentation is the best way forward.
Where do you start incremental experimentation?
You have the gist of how incremental experimentation works. Test something small, measure the results, optimize, repeat—it’s pretty straightforward. But there’s some pre-work that needs to be done before you start testing. You first need to decide what you should test, establish your baseline, define your goals, and create a hypothesis.
What to test first
It might be tempting to start testing Google Ads copy right off the bat. But we recommend starting a bit closer to home (or point of conversion). Your own audience (the people that have interacted with your content before) are a much better gauge of what’s effective and what isn’t. Test high-visibility email campaigns and landing pages to see more actionable results quickly.
We typically use the ICE framework (Impact, Certainty, Effort) to prioritize what to test.
- Impact: What is the potential for impact? Look for areas most likely to net significant results.
- Certainty: How certain are you that there will be an impact?
- Effort: Is this easy to test? Or will it require the input of people across the org with a long implementation timeline?
Define goals
Once you’ve established what you should be testing, it’s time to define how they should be performing. This can vary quite a bit between business types and verticals, what your current KPIs are, your company size, and what your industry benchmark is.
Ensure that you can clearly define the metrics you’re trying to improve, and which one is your top priority.
Establish a baseline
This is simply looking at how an email, landing page, ad, or website is currently performing. What’s the bounce rate? How long do they stay on the page? What’s the open rate? Most importantly, what’s the conversion rate? Making a record of these numbers first is absolutely crucial for establishing your goal, hypothesis, and measuring success.
Look at the data
Your current page performance and data can provide key insights on where to focus your testing. If you’re seeing a high email open rate but very few click-throughs, it could be that the email content is not aligning with the expectations of the recipient. Maybe your landing page is getting a lot of traffic and time spent on the page, but they bounce when they see the form. In that case, it could be that your form fields. Your data will give you key insights on what you need to focus on.
Create a hypothesis
If your baseline is where you’re at, and your goals are where you want to be, then your hypothesis is how you get there. Your data is going to point you in the right direction. Using the above example of a high email open rate but low CTR, your hypothesis could be that the hero section of your email needs to communicate the value of your offer more clearly. So your hypothesis would be “Recipients will convert if the value of the offer is clearer when they open the email. We will test a new headline in the hero section that is more explicit.”.
Be as specific as possible with your hypothesis. If your test improves results, you’ll have a clear understanding of why. And if it doesn’t, you’ll still have a much better idea of what the path forward is. You’ll also have insights and data that you can apply to future tests to improve your starting position.
Test again
Hopefully your first test resulted in a lift. Maybe you even reached your goal. One of the great things about incremental testing is that there’s almost always room for improvement. Look at the new data from your test to see what else you might be able to optimize, establish your hypothesis, and test it again.
Testing ideas
There are so many different page or email elements that can be tested. And while your data should help inform what to test, it may not always be explicit. So we’ve listed a number of different things that you can test incrementally and move that needle.
"Testing should start at your first point of contact with your customer. Think of testing these "wayfinders" in your UX to capture and better understand how users navigate your experience."
Jeff Cheal, VP of Product for Optimizely
Web marketing tests
Headline/hero section
Any given web or landing page’s headline and hero section is the first thing a visitor sees, so it needs to grab their attention. There’s a lot of room to experiment with different headlines, images, and subheads, so there’s a not-insignificant opportunity for a huge lift here. (Just remember: test one at a time.)
Calls to action
CTA buttons are easy to change and easy to test. For such a small page element, they can have a massive impact on your conversion rate.
Copy
What’s better: less copy or more copy? Do your prospects want to see benefits? Or do they want storytelling? What’s the best tone? There’s ample room for testing your copy on any landing page or web page.
Images
Images can do a lot of heavy lifting in helping you better express your brand or product benefits. They can also get in the way. Consider testing different types of images, where you place them, or even using no images at all.
Forms
If your landing page has a form to collect lead information, is it asking too much or too little information from visitors?
Layout/page length
Simple changes to a page’s layout and/or length can drastically change how a visitor interacts with it.
Be sure to check out Optimizely’s Big Book of Experimentation for a deep dive into the world of testing.
Email marketing tests
Send time and date
Is your audience more likely to engage with your email first thing in the morning on a Monday or at some other time of day or week?
Subject line
Your subject line is the most important element in ensuring your email gets opened, so testing it can have a massive cascading impact on your open rate, click through rate, and conversion rate.
Preheader
The text you see right after the subject line (in most email clients) can also have a dramatic impact on open rates (and thus, everything else). Your preheader can provide context, increase mystique, or hint that there’s something of value to be gained from opening the email.
Sender
Depending on your business, email recipients may be more likely to open an email from the company or a specific person at your company. For example: An email from “John Smith” might seem more personable than an email from “Sales”. The same applies to the sender email address.
Hero section
If your email has a hero section, the content it contains is important because it should immediately communicate what the email is about, and why they should read it.
Tone/style
Does your audience prefer a buttoned-down enterprise tone, or would they prefer a personal tone, like they were hearing from a friend or colleague?
Length
In marketing, we often say less is more. But sometimes, less isn’t enough. Testing the length of your content can help you determine how much information your audience needs to convert.
Content structure
Where your content lives is as important as what it says. Sometimes, putting the offer up front results in higher conversions. Other times, presenting pain points before solutions is more effective. There are many different ways you can structure your content to get different results.
Call to action
What does your call to action say? Is it styled as a button or hyperlink? Where does it exist in the email? Do you use it just once, or is it repeated throughout?
Personalization
Using a recipient’s name or company name in the subject and/or body of the email can be an effective way to grab their attention and increase their engagement with it.
Images
Would including images of products, the sender, or icons have an impact on engagement and conversions?
Plain text vs HTML or templates
Does your audience prefer straightforward text emails or are they more likely to read an email that was thoughtfully designed and branded?
A last word on incremental experimentation
Testing your web pages and email campaigns can take time and planning. But, the payoff has the potential to compound faster than current interest rate hikes. When the economy isn’t certain, incremental experimentation is a sound way to continue optimizing your marketing efforts and growing your business.
Over time, these smaller tests won’t just help you boost sales, subscriptions, and leads, they’re also going to help you better understand your customers, improve the customer journey, and perhaps most importantly, position you to succeed when economic circumstances improve.
Google Optimize will go offline on September 30, 2023
That means if you’re running any experiments beyond that date, you won’t be able to access any of that data. We’re partnering with Optimizely—the world’s leading optimization platform—to help you make that transition. Optimizely is easy to implement and use, so you’ll be able to continue running experiments without missing a beat.
