Also called split/multivariate testing is a method for comparing two versions of something; for instance 2 different versions of a call to action button to determine which one is more successful. To identify which version of a design approach is better, we create 2 versions at the same time, and show each to half of the same target audience simultaneously. The test measures which one performed better with the target audience. The version that prompts the most users to take the desired action or the better conversion rate, is the winner. The test must be statistically significant to be trustable.
Though A/B testing is an ideal means to refine approaches, it’s not good to compare variations with a lot of differences as you won’t be able to measure which variation is performing better for which reason.