Articles on: What's new

Pre-Purchase EDD A/B Testing

Plans: Premium and Enterprise Advanced Platforms: Shopify


Overview


Pre-purchase EDD A/B Testing helps you quantitatively measure the impact of displaying Estimated Delivery Dates (EDD) on your storefront. By running controlled experiments directly within Shopify, compare shopper behavior when EDD is shown versus hidden. This feature will help you understand whether EDD improves conversion rates and shipping revenue, plus enables data-driven decisions for optimizing the pre-purchase experience. 


Available to Premium and Enterprise Advance plan users who have the Pre-purchase EDD Shopify widget installed and enabled.


What you’ll learn


In this guide, you’ll get to know about:


  1. How Pre-purchase EDD A/B testing works
  2. Step-by-step pre-purchase A/B testing process


Step 1: Create an A/B test

Step 2: View and manage the test list

Step 3: View test results



  1. Key takeaways


How Pre-purchase EDD A/B testing works


You can run A/B tests on the two surfaces:


  • Product Detail Page (PDP)
  • Checkout Page


At a high-level, each pre-purchase A/B test compares two shopper experiences:


  • Hide EDD group: Shoppers do not see Estimated Delivery Dates
  • Show EDD group: Shoppers see Estimated Delivery Dates


Traffic is split between the two groups for a defined period. During the test, the system tracks key shopper behaviors, such as page interactions, completed orders, and shipping revenue, and compares performance between the two.


Once the test ends, the system analyzes the outcomes and provides performance metrics and optimization suggestions when applicable.


Step-by-step pre-purchase A/B testing process


The A/B testing process consists of three main steps, each followed by multiple sub-steps.


Step 1: Create an A/B test


Required setup


Before creating a test, ensure that the selected surface (PDP or Checkout) has pre-purchase EDD installed and enabled. 


If EDD is not enabled, the A/B testing entry point may not be visible.


Create a test:


  1. Go to Tracking > Pre-purchase EDD > Getting Started.
  2. Choose the entry point for Product Page EDD or Checkout Page EDD.


You’ll find two separate entry points for Product Page EDD and Checkout Page EDD A/B tests:



Each A/B test can cover only one surface. To test both PDP and Checkout, you must create two separate tests.


Test configuration:


When creating a new test, provide the following details:


1. Test name 


A custom name to help you identify the test later. Each test name must be unique.


2. Traffic split


Defines how traffic is divided between the Show EDD and Hide EDD groups. Each group must receive between 20% and 80% of traffic, ensuring sufficient data volume for reliable analysis.


3. Test duration


Defines when the test starts and ends:


  • Both dates (start date and end date) are based on your organization’s time zone
  • The test duration must be between 7 and 30 days
  • Test periods cannot overlap with other tests on the same scope.
  • If a time conflict exists, the system will prevent test creation


4. Save test


Click Create to save the test. After saving, the test is created in a Scheduled state, and it will start automatically at the configured time.



Step 2: View and manage the test list


All created tests appear on the Test List page to monitor progress and access results. 



Test statuses


Each test appears with one of the following statuses:


1. Scheduled: The test has been created but has not started yet.


  • You can review or edit test details
  • Delete the test if it is no longer needed
  • The test will start automatically at the scheduled time



2. Running: The test is currently active and collecting data.


  • You can view test details and interim (D-1) metrics
  • Results are still accumulating and are not final
  • Click End Test to stop the running test early



Tests ended early or with insufficient data may not generate final conclusions.


3. Ended: The test has ended, and final metrics and insights are available.


  • You can view test details and final metrics
  • Click Delete to remove the tests permanently, and they cannot be recovered.



Step 3: View test results


Click View Details to review the test’s configuration and performance. The result page includes:


  • Test configuration summary
  • Optimization suggestions (if applicable)
  • Key performance metrics
  • Trend charts and detailed data


Result metrics


Metrics are updated daily using D-1 data (previous day’s data). 


1. PDP EDD tests


PDP EDD tests mainly focus on shopper engagement on product pages.


Metric

Description

How it’s calculated

Clickthrough Rate (CTR)

Measures how often shoppers take the next key action

CTR = (Number of shoppers who proceed from PDP Ă· PDP page views)



2. Checkout EDD tests


Checkout EDD tests focus on order completion and shipping revenue.


Metric

Description

How it's calculated

Conversion Rate

Measures how often shoppers take the next key action

CVR = (Number of completed orders Ă· Checkout page views)

Average Shipping Price

Measures the average shipping price generated per order to evaluate whether EDD influences shipping option selection.

Average shipping price = (Total Shipping Revenue Ă· Order Volume)



Qualitative outcome


In addition to numeric metrics, the system applies statistical analysis to determine a qualitative outcome for each experiment. It displays the outcome as a visual label next to the core metrics. This helps users to interpret the results quickly. 



Possible outcomes


  • Winner (Show EDD / Hide EDD)


This shows that one group statistically outperforms the other with sufficient confidence.


  • Equivalent


The performance difference between the two groups is statistically insignificant. In this case, it's hard to conclude that either group performs better. The performance results of both groups are statistically comparable.


When results are not available


In some cases, the system cannot generate reliable conclusions. When this happens, detailed metrics are hidden, and an optimization suggestion is shown instead. The possible suggestions could be:


Suggestion 1: Insufficient test durations


  • Triggered when the test runs for less than 7 days
  • Recommendation: Click the link “Create new test” to restart a new test with a longer duration



Suggestions 2: Not enough data


  • Triggered when total traffic is too low to reach statistical significance
  • Recommendation: Click the link “Create new test” to increase test duration or adjust traffic split



Suggestions 3: Discontinuous sample


  • Triggered when the system detects that EDD may not be functioning properly on certain days or when daily traffic is abnormally low.
  • Recommendation: Click the link “EDD settings”, review EDD configuration, and ensure it is running correctly.



View detailed chart data


Scroll down on the results page to view detailed performance charts, showing metrics at daily or weekly levels. Metrics are shown with clear trends for deeper analysis. 



Export test data


Click Export to save experiment data in CSV format. This allows further analysis and reporting outside of AfterShip. 



Key takeaways


  • Pre-purchase EDD A/B testing measures the real impact of showing Pre-purchase EDD on shopper behavior using controlled experiments.
  • Run tests on Product Detail Pages or Checkout Pages, with each test covering only one surface at a time.
  • Shoppers are split into Show EDD and Hide EDD groups, and performance is compared using engagement, conversion, and shipping revenue metrics.
  • A/B tests must run between 7 and 30 days with a 20%–80% traffic split to ensure reliable results.
  • Results include both quantitative metrics and a qualitative outcome (Winner or Equivalent) to simplify decision-making.
  • If data is insufficient or inconsistent, the system provides clear optimization suggestions instead of unreliable conclusions.
  • Test data can be reviewed through trend charts and exported as CSV for deeper analysis and reporting.



Updated on: 12/01/2026