TESTING FOR MOBILE VS. WEB
One of the biggest challenges is testing across multiple platforms. After all, what works on a desktop browser doesn’t necessarily work on a mobile device. Consider that the same user may prefer a green button on a laptop but a blue button on a smartphone screen.
“Digital publishers should learn what they can from A/B web testing and apply it to A/B mobile testing,” notes Slade. “Mobile testing is more complicated, so you probably won’t want to test as many factors on mobile as you do with web. But it’s worthwhile running separate tests on both because web and mobile experiences are so different.”
Remember that mobile apps “represent a more captive audience, which significantly changes behavior and how testing is accomplished,” says Greg Hinkle, CTO and co-founder of Evergage. “[Mobile] users may require time to get used to a new pattern or feature, so tests may need to run longer.”
Web-based apps can typically undergo the same type of testing you’d conduct for a traditional webpage. But if you’re testing a native app, the procedure is much different and implementing changes is more complicated. “The store providing the app will need to approve app changes, and your customers will need to download the app update, whereas web changes occur instantly. Also, you’ll need to find an A/B test provider for apps as opposed to websites,” Slade says.
HOW A/B TESTING WILL EVOLVE
Targeting and personalization should also improve. “The leading optimization tools, today, can automatically adjust which segments of customers receive each test variant,” says Nelson. “While this technology is available today, we can expect its adoption and performance to continue to expend in the coming years.”
Abisambra predicts a forthcoming era of A/B/C or even A/B/C/D testing. “I can see artificial intelligence supplementing how and when experiments are executed to not waste a single minute and to improve the percentage of experiments that yield a lift or improvement,” he says.