Site   Web

February 5, 2016

My Must-Try List of A/B Testing Tools — Compared

Photo Credit: Éole Wind

A/B testing has become so pervasive that data nerds (that’s us) are using it to optimize their dating profiles. Let’s see if the profile picture with me wearing my favorite blue shirt while petting my dog gets more clicks and e-mails. You know, in direct comparison with the beach picture where it looks like I took a bath in tanning oil. It was only half a bottle! There were cervezas involved. No kidding. It happens. Don’t judge.

Long before people started doing conversion optimization to get more dates, bloggers got really good at using A/B testing tools too. More sign-ups, sales, and CPC ad clicks.

The results are too good not to try. And if you didn’t, then you’ve been losing out on conversions and earnings.

As A/B testing exploded in popularity (like the picture of me and my kitten wearing matching New Year’s hats), more tools started to emerge. So let’s explore some of the top tools out there.

Here’s a complete list of the best ones on our must-watch list. If you haven’t tried at least one of them already, you may need an intervention. It’s time to get started.

A quick warning. There are a couple examples we included that require some coding. If you’re not a coder, just hold your breath, peek through your fingers, and scroll to the next A/B testing tool on the list… really fast. All will be ok. We promise.

What features should every A/B testing tool have?

• Baked in A/B testing algorithm

• Rapid set up

• Easy upload of HTML/CSS/ JavaScript

• Segmentation

• Third-party integration

• Cross device testing

• Analytics and reporting

• WYSIWYG editor

The below tools have all of the above. Our 12 favorite A/B testing tools are:

• Unbounce

• PageWiz


• Google Analytics Experiments

• Convert

• Truconversion

• A/Bingo

• Five Second Test

• Optimizely

• HotJar

• AB Testy

• AdNgin

1. Unbounce

For starters, what a great name. Unbounce is a complete landing page creation, management, and optimization platform. It provides us a variety of professionally designed templates to create a landing page. They are responsive designs that work for mobile too. And the A/B testing functionality is built in.

So that means you’ll be creating testable variations and then seeing which ones bring the most conversions. They are more geared towards landing pages. If you’re want to test different landing page variations, are building an e-mail list or selling a product or service, Unbounce is a great option.

Pricing: Starts at $49 per month.

Best for: Campaign managers, online marketers, entrepreneurs, bloggers.

2. PageWiz

If you like Unbounce you’ll love PageWiz. PageWiz is not as well known but it’s an up and comer that\’s worth checking out. Their templates are some of the nicest and they also offer a built-in and automated A/B testing solution.

Pagewiz integrates with third parties like SalesForce and Mailchimp and also provides real-time analytics so you can put those leads to quick use. If you run a marketing agency you can also use their interface to manage various clients through one account.

Pricing: Starts at $29 per month

Best for: Campaign managers, online marketers, entrepreneurs, bloggers.

3. Visual Website Optimizer by Wingify

VWO calls itself the “A/B Testing Software for Marketers.” Of course, if you’re using any A/B testing software, whether you like it or not, you’re acting like a marketer. Don’t worry. That’s a good thing. Marketers get results.

Wingify’s team built a full gamut product with features that go far beyond the basics of A/B testing. If you’re new to conversion optimization and testing, start out with the basic A/B testing components first. You can take advantage of the rest as you get more familiar with VWO.

Unlike Unbounce, there is more versatility with VWO. You’re not just creating custom landing pages. You can actually adapt new versions of your website and test the variations to see what works best. There are also additional features like heatmaps, targeting options, and third party integrations available.

Pricing: Starts at $49 per month.

Best for: CROs (conversion rate optimizers), webmasters, content marketers, user experience experts.

4. Google Analytics Experiments

There are some undeniable advantages of being a company like Google. One of them is the complete freedom to hire as many geniuses as needed. And it results in Google software at or near the top of lists like these. Maybe a little bit like the USA and the Olympic medal count, or Canadians who dominate slapshots and beaver tattoos.

The caveat here with Google Analytics Experiments is that it’s a part of the overall Analytics package. It’s not the core focus of Analytics. Likewise, Google’s known to be hit and miss with their user interface. It’s probably slightly superior to the USS Enterprise knobby controls that Captain Kirk used back in 1969, but only slightly. If you use Google Analytics and are still hesitant to get started with the other options, give Experiments a try.

The setup isn’t as complex as you might expect. But it’ll probably just whet your appetite for a better A/B testing tool set, like the ones showcased here. If you’re comfortable with Google Analytics already, we think Experiments is a good place to get started.

Pricing: Comes with Google Analytics so if you don’t pay for GA you won’t pay for GA experiments.

Best for: A/B testing noobs.

5. Convert

Another great name. No beating around the bush here. There’s no mistaking the objective with Convert. However, they are targeted at the Enterprise market and won’t be suitable for some lower trafficked blogs. Convert boasts flicker free tests. This is actually an important feature.

Flickering sometimes happens during A/B testing. When the test modifies your page content in real time, it could show flickers or blips on the screen as the test renders. That’s not a deal breaker, but it isn’t a good thing either. Technically, it could introduce a contaminant into an otherwise controlled test. That’s why the Convert team works so hard to eliminate it.

There are also some good code integrations. However, we think this one is best for more advanced teams. A higher learning curve is expected to take full advantage of all that it has to offer.

Pricing: Starts at $125 per month

Best for: Large eCommerce

6. TruConversion

Opportunity! We like to keep a watchful eye out for new products. TruConversion is one of them. It’s (very) similar to one of our other products on the list (HotJar) and bids itself as an “all-in-one analytics and feedback application”. The opportunity here is if you can get in on their beta you’ll receive the full enterprise package free for a whole year. Not bad.

Their marketing doesn’t suck either. Even though they’re only in beta, they’ve managed to gain a following of 20K e-mail subscribers and publish a lot on their blog.

Though their strength is in understanding how your users interact with you site (including conversions), they combine many of the best features into a single streamlined package. And then tie directly into some other major applications and platforms. They include heat maps, recorded visitor sessions, conversion funnels, micro surveys, and form analytics. All of which, you can A/B test through the system.

Looks like there’s some serious backing behind this tool so we predict it will gain popularity quickly once out of beta.

Pricing: Free while in beta

Best for: CROs, Online marketers.

7. A/Bingo

If you’re not a coder, you might want to skip over A/Bingo. However, it’s growing in popularity as it’s a plugin written for the highly popular Ruby on Rails development framework. If your site is built on top of Rails, there may be some potential here for you.

One of the key advantages of integrating A/B testing directly within code is speed. The plugin is very cool and open source. So we’ll forgive them for the 15-year-old webpage design. It gets the job done and shares a cool tool — and that’s what matters. None of your designer snobbery. If you’re interested in A/Bingo, you should also compare with Vanity.

FYI, the creator of A/Bingo recently sold it so I’d grab the code quickly before it becomes unavailable.

Pricing: Free

Best for: Coders who like no frills software.

8. Five Second Test

Five seconds may seem like a short time, but in fact it is more than enough time for a website visitor to determine if there is enough quality in your website to stay, or to leave potentially never to return.

It’s a novel way to collect user feedback. You prepare a mockup of your site or page, ask some questions, and then get feedback from real users. They have five seconds to see it. Then they are presented with questions. And Five Second Test probes them for answers to understand what they recall and what left the biggest impression.

What can I learn in five seconds?

Who you are

What product or service your provide

Why should I care

So a five-second test can be a great starting point to test your entire conversion funnel. First, do I pass the five-second test? Second, everything else.

Pricing: Basic is free and a pro package costs $99 per month.

Best for: Technically anyone but online marketers will probably gain the most from it.

9. Optimizely

Just in case we startled you above by talking about code, we’ll bring you back into the A/B testing comfort zone. Optimizely is probably the best known A/B testing solution of the bunch. Best of all, for most users, it’s free for life with a pretty decent set of features that will do nicely for most A/B tests.

Optimizely has been, and still is, a great starting point for most A/B testers while also providing the necessary features for more advanced users.

Pricing: Free with most features. More robust packages are most likely priced according to page views.

Best for: CROs, online marketers, UX experts, webmasters.

10. HotJar

Not branded as an A/B testing tool but the functionality is there among some other really great features like heatmaps, visitor recording, conversion funnels, and more.

One thing you need to keep in mind with HotJar is that the data you will be provided is sampled (just like Google Analytics started doing recently). This allows HotJar to offer very competitive pricing but you don’t get all of your data and need to rely on HotJar’s algorithms to provide you with accurate sampling.

By the way, if you’re running an agency, HotJar offer whitelabeling, AKA custom branding, options for your reporting.

Pricing: Basic package is free for life. Paid packages start at $29 per month for users with over 10,000 daily page views.

Best for: CROs, Webmasters, Online Marketers, UX experts.

11. AB Tasty

A good user experience is one born through continuous testing and AB Tasty try to provide their users with all the tools necessary not just to convert but also to create a great user experience that really connects with end users.

So obviously AB Tatsy is more than your nuts and bolts A/B testing, although there’s that too. You can create communication messages through modules that pop up or slide in in real time to segmented audiences. As a marketer, you gotta love the potential.

They also offer a DMP (data management platform) integration to really put your data to good use.

Pricing: Starts at $34 per month.

Best for: Online marketing and UX professionals.

12. AdNgin (shameless plug)

If you’re a digital publisher, we’ve saved the best for last. We’re going to have to suspend a little modesty just this one time. While the other A/B testing tools we’ve showcased are more focused on content optimization, AdNgin is unique. We’re fully focused on maximizing your ad earnings through continuous testing.

That’s right, AdNgin does not modify content. Instead it allows you to experiment with Google AdSense layout like placement, size, device, and much more. It isn’t just about changing the layout of your ads though. We also help you test across different ad networks. We want to find the absolute best combination of ad layouts, sizes, types and networks to increase your earnings.

Pricing: Free for life up to 10,000 daily page views. Paid packages start at $250 per month.

Best for: Individuals and companies who use Google AdSense.

Always be testing

Between you, me, and the Internet, we don’t really care which A/B testing tool you choose from above (as long as you use AdNgin:)). The most important thing to remember is that the best time to start A/B testing was yesterday. Don’t drown yourself in endless research on finding the perfect tool. Start testing now cause a day without testing is a day you’re losing money.

Did we leave out your favorite tool? Tell us how many of these A/B testing tools you’ve tried. How many are on your list of to-dos.


Content creator, expert on everything you can Google.