Use Cases & Best Practices
Learn how to apply A/B testing to common e-commerce scenarios to discover what drives more conversions and revenue for your store.
Common Testing Scenarios
Test Different Products
Scenario: You're running a promotion and want to know which product generates more sales.
Example Test:
- Link A: Direct to your premium water bottle ($45)
- Link B: Direct to your starter water bottle ($25)
What to measure: Which product drives more revenue and conversions
Insight: Even if the cheaper option gets more orders, the premium option might generate higher total revenue per visitor.
Test Discount Strategies
Scenario: You want to know if percentage discounts or fixed amount discounts perform better.
Example Test:
- Link A: 20% off entire order
- Link B: $10 off entire order
What to measure: Conversion rate and average order value (AOV)
Insight: Percentage discounts often drive higher AOV, while fixed discounts can boost conversion rates on lower-value carts.
Test Free Shipping Thresholds
Scenario: Test whether free shipping or a discount drives more conversions.
Example Test:
- Link A: Free shipping (no minimum)
- Link B: 15% off + standard shipping
What to measure: Conversion rate, revenue per visitor, and total revenue
Insight: Free shipping often increases conversion rates, but percentage discounts can drive higher cart values.
Test Cart Bundles
Scenario: Compare different product bundles to find the highest-performing combination.
Example Test:
- Link A: Skincare routine bundle (cleanser + moisturizer + serum)
- Link B: Skincare basics bundle (cleanser + moisturizer)
What to measure: Conversion rate and revenue per visitor
Insight: Sometimes simpler bundles convert better despite lower AOV. Run the test to see what your customers prefer.
Test Upsell Strategies
Scenario: Test different upsell approaches to maximize revenue.
Example Test:
- Link A: Main product only
- Link B: Main product + recommended add-on (with small discount)
What to measure: Conversion rate, AOV, and revenue per visitor
Insight: Adding a small, relevant upsell can increase AOV without hurting conversion rates if positioned correctly.
Test Seasonal vs. Evergreen Offers
Scenario: Determine if seasonal messaging or evergreen messaging performs better.
Example Test:
- Link A: "Spring Sale - 25% off all fitness gear"
- Link B: "Limited Time - 25% off all fitness gear"
What to measure: Click-through rate and conversion rate
Insight: Test whether urgency-based evergreen messaging outperforms seasonal campaigns for your audience.
Best Practices for Successful Tests
Start With High-Impact Changes
Test major differences first:
- Different product categories
- Significant discount variations (10% vs 25%)
- Bundle vs single product
Avoid testing minor variations like:
- Button color changes (that's for landing page A/B testing)
- Small discount differences (15% vs 17%)
Focus on One Variable
Good test:
- Link A: Product X with 20% off
- Link B: Product Y with 20% off
- Testing product preference
Bad test:
- Link A: Product X with 20% off
- Link B: Product Y with free shipping
- Testing too many variables at once
Consider Your Traffic Volume
High traffic (500+ sessions/week):
- Test subtle optimizations
- Run shorter tests (1-2 weeks)
- Aim for 95% confidence
Medium traffic (100-500 sessions/week):
- Test more dramatic differences
- Run longer tests (2-4 weeks)
- Accept 75-85% confidence for business decisions
Low traffic (< 100 sessions/week):
- Only test major differences
- Be patient (4-8 weeks)
- Look for practical significance over statistical significance
Set Clear Success Criteria
Before launching your test, decide:
- Primary metric: Usually conversion rate or revenue per visitor
- Minimum improvement: What % increase makes it worth implementing?
- Test duration: When will you make a decision?
- Confidence threshold: What confidence level do you need?
Common Mistakes to Avoid
Problem: Running 3+ A/B tests simultaneously with overlapping audiences.
Solution: Focus on one test at a time, or ensure tests target completely different traffic sources (e.g., Instagram vs Email).
Why it matters: Overlapping tests dilute your traffic and make it take much longer to reach significance.
Problem: Seeing "Link A is winning!" after 2 days and ending the test.
Solution: Wait for at least 1 week and 100+ sessions before making decisions.
Why it matters: Early results are often misleading due to day-of-week effects, time-of-day patterns, and random variation.
Problem: Modifying Link A or Link B during the test period.
Solution: Set up your test completely before launching and don't touch it until you have results.
Why it matters: Changes invalidate all previous data and force you to start over.
Problem: Implementing a "winner" that has 0.3% higher conversion rate.
Solution: Set a minimum improvement threshold (e.g., 5% or 10%) that makes the change worthwhile.
Why it matters: Tiny improvements often aren't worth the effort to implement, and may not hold up over time.
Real-World Example: Discount Test
Store: Athletic apparel brand
Goal: Maximize revenue from email campaign
Traffic: ~1,000 email opens expected
The Test:
- Link A: 25% off sitewide
- Link B: Buy 2 get 1 free on all items
Setup:
- 50/50 traffic split
- 2-week test duration
- Primary metric: Revenue per visitor
- Secondary metric: AOV
Results (after 2 weeks):
- Link A: 1,247 sessions, $8,453 revenue, $6.78 revenue/visitor
- Link B: 1,198 sessions, $11,290 revenue, $9.42 revenue/visitor
- Winner: Link B (buy 2 get 1 free) with 39% higher revenue per visitor
- Confidence: 91%
Key Learning: Customers were more motivated by getting a "free" product than by a percentage discount, even when the math was similar.
Related Resources
- Getting Started with A/B Testing - Setup and configuration
- Analytics Dashboard - Track performance across all your links and tests
- UTM Tracking - Add campaign tracking to measure marketing channel performance