If you need a simple, inexpensive way to figure out what's going to make your website's users click on that big, red button, you've come to the right place.
A/B testing is one of the easiest ways to figure out whether one specific variable of your website is working. It could be a button color, a bit of web copy, an image -- something extremely finite that may (or may not) have a measurable impact on desired actions, be they conversions or a simple click-through.
In an A/B test, the variable changes as users visit the page; user actions are recorded and analyzed; and the developer (or startup) is able to take action based on the results.
A/B testing is generally faster and simpler than other types of testing, allowing a fast-moving startup or solo developer to quickly iterate and improve without wasting too much time on guesswork or esoteric design decisions.
What is A/B Testing, Exactly?
A/B testing differs from multivariate testing in that it only tests one variable at a time; in other words, you change only the button color (not the button copy or any images) and see how many users click the red button versus the green button. If the red button gets more clicks, the page gets a red button.
It's a pure-and-simple way of making design decisions that rely on real data rather than divergent and emotional opinions from co-founders and designers; after all, you can't argue with facts. To quote Admiral Grace Hopper, "One accurate measurement is worth more than a thousand expert opinions."
In the more complex process of multivariate testing, you'd introduce more than one variable at a time and measure the effects of all variables on your desired outcomes. Simply put, you could change the button color, surrounding copy and images on the page to come up with the ideal combination of variables to achieve the favorable actions from the users.
If you'd like to learn more about the process of testing website variables and the concept of getting user feedback in lieu of acting solely on the HiPPO (Highest Paid Person's Opinion), check out this paper [PDF] on controlled experiments on the web.
What's the Best Way to A/B Test?
If you plan to conduct A/B testing to optimize your website, there are some best practices to observe and some expectations to lay out at the beginning.
For example, you (and your team, if you have one) will want to declare criteria, variables and expectations before you begin your testing. Make sure you're not doing anything that will invalidate the results of your test.
Also, make sure you've got enough traffic to make an A/B test worthwhile. If you're working with a relative handful of site visitors each day, your A/B results may not be accurate.
Likewise, make sure you conduct your tests over a long enough period of time to allow for variance caused by traffic fluctuations, days of the week, holidays, time of day, etc.
Finally, make sure that your website is functional and optimized for excellent, fast, cross-browser performance before you commit to testing. After all, no one will care whether the button is green or red if the page takes a minute and a half to load.
Some Code-Free A/B Testing Resources
1. Optimizely
In a quick survey of our startup-minded Twitter followers, Optimizely was far and away the most highly recommended resource for startup and developer A/B testing. A Y-Combinator startup itself, Optimizely was used by the groundbreaking online Obama campaign in 2008.
You can use Optimizely even if the thought of coding makes you break out in a cold sweat. Just create an experiment in the visual interface, and you're on your way to optimization.
Check out this quick demo video to get an idea of Optimizely's services, which range from $19 to $399 per month:
2. Performable
Performable gives you a simple and code-free way to conduct A/B testing. It allows you to create pages and variations from within the site, then publish the pages and collect the data, all without touching any code.
The folks at Performable also recently added a new analytics backend that focuses on conversion goals rather than just testing alone.
Pricing plans run from $50 per month to $1,200 per month, with available e-mail, marketing and CRM integration.
3. Unbounce
Unbounce is a similar resource, providing its clients with a browser-based WYSIWYG editor, page templates and simple stats for A/B testing. The service comes with a strong anti-code, pro-marketing stance, putting the focus on optimizing for conversion rather than tinkering around in CSS.
Prices run from $50 to $500 per month.
Light-Coding-Required A/B Testing Resources
Google's Website Optimizer was another highly recommended resource from our quick Twitter poll.
With Website Optimizer, all you have to do is create two pages with two URLs, each one showing a different variable. Then, you tell Google your desired outcome, i.e., you provide a conversion page URL. With just three URLs, your A/B test is ready to run. Install some JavaScript on your site, and let the games begin.
Website Optimizer also offers multivariate testing, if you ever feel the need for more complex analysis. And all the Website Optimizer tools come completely free of charge.
This tool comes complete with a WYSIWYG editor to customize buttons, headlines and more for A/B and multivariate testing. This service ranges from $49 to $249 per month, and you can try it free for 30 days.
Once you've selected and modified your variables, all you need to do to begin testing is to insert a small amount of JavaScript into your site. Check out the demo video below to see the interface in action.
Heavy-Coding-Required A/B Testing Resources
1. Genetify
Think of Genetify as A/B-plus. It's been called "a simple concept that should sit well with most mathematically inclined types."
Genetify gives you a JavaScript library for conducting A/B tests on your site's CSS, HTML and JavaScript variables. Then, the program "trains" the tests as time goes by, adjusting based on users' actions and predetermined desired actions.
You can also choose to set up Genetify on your own server. Check out Genetify's live demo to see how it works in practice.
2. Vanity
Vanity is described by its creators as "an experiment-driven development framework for Rails." An open source bit of software, Vanity allows you to conduct A/B testing on your site and returns a simple report back to you. You can also integrate Google Analytics data in your Vanity dashboard.
You can get the source directly from GitHub, and you can also contribute to the project, particularly its more experimental side.
Other Resources?
If you've found an A/B testing resource that's great for small companies, startups or solo developers, please share it with us and other readers in the comments!
More Web Development Resources from Mashable:
- 10 Intermediate and Advanced Tips from PHP Masters
- 5 Website Designs That Blew Us Away
- HOW TO: Make Your WordPress Blog More Like Tumblr