Shopify Rollouts: Native A/B Testing for Themes and Checkout (2026 Guide)

Shopify Rollouts is a free, server-side A/B testing tool built into the Shopify admin under Markets > Rollouts. It splits traffic between two whole themes, checkout configurations, or customer account setups with no third-party script and no monthly fee for stores on the Grow plan or higher. Shopify shipped the feature on June 5, 2026 as part of the Summer '26 Editions, after introducing Rollouts as a scheduling tool in the Winter '26 Edition, per Shopify's official changelog. This guide covers what Rollouts can and cannot test, how to run a valid experiment, and which tests UAE and GCC store owners should prioritize. It is part of the complete Shopify CRO guide, which lays out the full conversion framework these experiments fit into.

What Is Shopify Rollouts?

Shopify Rollouts is a native experimentation tool that schedules, gradually publishes, and split-tests theme variants, checkout configurations, and customer account configurations directly from the admin. The split runs server-side: the visitor is assigned a variant before the page renders, so there is no flash of the original content (FOUC) and no injected JavaScript to slow the page down. That detail matters because every third-party testing script added to a storefront costs Core Web Vitals points, and page speed directly influences checkout completion. In Shopify's own data, every 0.1-second improvement in mobile checkout load time correlates with a 1.5% lift in checkout completion rate. Rollouts adds zero page-speed cost to the equation.

When Was Rollouts Released, and Which Plan Do You Need?

Rollouts first appeared in Shopify's Winter '26 Edition as a scheduling and publishing tool, then expanded on June 5, 2026 to cover themes, checkout, and customer accounts with full A/B testing. Plan access breaks into two tiers: scheduling and gradual launches are available from the Basic plan up, while measured split experiments require the Grow plan or higher. On Shopify Plus, Rollouts pairs with Checkout Components the composable checkout architecture that replaced the deprecated checkout.liquid template which gives Plus merchants the full checkout customization surface to test against. For a store already on Grow or Plus, that means native experimentation costs $0 per month versus $149 to $499 per month for price testing tools like Intelligems, or Shoplift's £74 to £699 tiers.

What Can You Test With Rollouts?

Rollouts tests three surfaces: themes, checkout configurations, and customer account configurations. On themes, you can compare layouts, section arrangement, and content changes made through the theme editor. On checkout, you can put two complete checkout setups head-to-head field order, trust elements, payment method sequencing, and messaging. When you start a rollout, Shopify automatically copies your published configuration, so you can keep editing the live store while the experiment runs. The reporting splits by surface: theme experiments show conversion rate, bounce rate, reached-checkout rate, and added-to-cart rate, while checkout experiments show checkout conversion rate completed purchases relative to sessions that reached checkout.

What Can't Rollouts Test Yet?

Rollouts does not test prices, discounts, audience segments, or Liquid code. You cannot split-test a product price point, a free-shipping threshold, or a discount strategy natively, and you cannot target a variant to new versus returning visitors, mobile versus desktop, or a specific geography. Multi-page funnel tests, post-purchase survey splits, and deep segment-level analytics are also outside its scope. The feature tests only customizations made through the theme editor and checkout settings not edits to Liquid template files, and not older vintage themes that predate the current theme architecture. These boundaries are the honest reason to keep a third-party tool around, and they are narrower than most Shopify marketing coverage admits.

Should You Cancel Your CRO App for Rollouts?

Cancel the app if your last completed A/B test was more than 90 days ago, because you do not have a CRO program you have a CRO subscription. That is the contrarian take that holds up in practice. Blackbelt Commerce reported on a $4M apparel brand paying $1,100 per month for a testing platform that produced two inconclusive tests in 14 months, neither of which was acted on. The brand cut the tool, replaced it with a structured quarterly testing calendar, and lifted PDP conversion 18% in the first cycle. The tool was never the constraint; the process was. Keep a third-party app only if you are actively testing price, shipping thresholds, segment-specific offers, or post-purchase sequences the exact gaps Rollouts leaves open. The clean rule of thumb: if a change can be built in the theme editor or checkout settings, test it natively. If it needs price, segments, or code, keep the app. Never run two overlapping traffic splits on the same surface, because concurrent tools contaminate each other's results.

How Do You Run a Valid A/B Test on Rollouts?

A valid Rollouts experiment needs one variable, a written hypothesis, a pre-defined sample size, and a run of at least two full business cycles. Peep Laja of CXL Institute has documented the math repeatedly: to detect a 10% relative lift at 95% confidence on a 1.4% baseline conversion rate the median Shopify conversion rate in Littledata's benchmark data you need roughly 1,000 conversions per variant, which works out to about 70,000 sessions per variant. Even a low-stakes checkout test in Shopify's own examples calls for around 120,000 sessions per variant to detect a 3% change. The two most common failures are stopping a test early because one variant "looks like it's winning," and launching a change at 100% traffic, which collects no analytics and cannot be reverted. Write the hypothesis down before you look at any data, run the experiment to its planned end date, and log the winner so you are not re-testing the same idea next quarter.

How Do Gradual Rollouts and Scheduling Work?

Gradual rollouts expose a new theme or checkout to a defined percentage of visitors and scale it up in steps typically 5% for the first 48 to 72 hours, then 20%, then 50%, then 100% once metrics hold. This pattern de-risks major changes: if a broken checkout ships, only 5% of traffic hits it instead of the whole store. Scheduling covers calendar events like Black Friday or Ramadan launches: you can schedule a themed checkout to go live on a specific date and revert automatically when it ends. Rollouts also supports market-localized experiments through Shopify Markets, which lets you test different CTA text, payment messaging, or layout per region. A 5% safety canary is not the same as a controlled split test use gradual rollout to de-risk, then run a proper A/B split for measurement.

What Checkout Tests Should UAE Stores Run First?

For UAE stores, the highest-impact Rollouts experiments are payment presentation tests, because local payment behavior is where GCC conversion differs most from Western benchmarks. Test three things in sequence. First, Tabby and Tamara installment messaging placement: in ConvFetti's audit set of 50+ Shopify stores, stores with BNPL properly surfaced on the product page convert at a 1.3% median versus 0.9% without. Second, cash-on-delivery visibility: COD remains a trust signal for UAE first-time buyers, and hiding it behind an accordion measurably lowers checkout completion. Third, accelerated checkout sequencing Shopify's data shows Shop Pay converts at 1.72x the rate of standard guest checkout, and its first-class placement at the top of the checkout now lets you test whether leading with Shop Pay, Apple Pay, and Google Pay beats leading with local methods for your specific traffic. Run these on checkout configurations, one at a time, for two to four weeks each.

Why Isn't Rollouts Enough for Most UAE Stores?

Traffic volume is the ceiling. The median UAE store we audit gets between 10,000 and 30,000 sessions per month, and at a 1.1% baseline CVR that produces roughly 110 to 330 conversions monthly. A test that needs 1,000 conversions per variant would run four to nine months before reaching significance longer than most seasonal cycles, and long enough that shopper behavior shifts mid-test. For stores below about 15,000 monthly sessions, structured A/B testing is the wrong first investment regardless of whether the tool is free. Start with qualitative work: session recordings and heatmaps through Hotjar or Microsoft Clarity, plus the checkout field-drop analytics Shopify built into the admin with the June 2026 checkout release, which show exactly where buyers abandon each form field. In our client data, the qualitative-first approach recovers about 80% of the improvement that full A/B testing delivers, and it works on any traffic level. Build the traffic or the qualitative insight first, then let Rollouts validate the winners.

FAQ

What is Shopify Rollouts and when did it launch?

Shopify Rollouts is a native A/B testing and publishing tool in the Shopify admin under Markets > Rollouts. It split-tests theme variants, checkout configurations, and customer account configurations with server-side traffic splitting and no third-party script. It launched as a scheduling tool in the Winter '26 Edition and expanded to full A/B testing on June 5, 2026 as part of the Summer '26 Editions.

Which Shopify plan do I need for Rollouts A/B testing?

Scheduling and gradual launches are available from the Basic plan up. Measured split experiments require the Grow plan or higher. Full checkout customization testing via Checkout Components is a Shopify Plus feature. For stores on Grow or Plus, native experimentation adds no monthly cost.

Can Rollouts test prices or discounts?

No. Rollouts does not test price points, discount strategies, shipping thresholds, or segment-specific offers. Those require a third-party tool like Intelligems for price and margin testing. Rollouts only covers changes built through the theme editor and checkout settings.

Should I cancel Intelligems, Shoplift, or VWO?

Cancel if you have not completed a statistically valid A/B test in the past 90 days you are paying for a subscription, not a CRO program. Keep the tool if you are actively running price tests, segment-level experiments, or post-purchase splits, which Rollouts cannot cover. Never run two overlapping splits on the same page.

How long should a Rollouts experiment run?

Run at least two full business cycles, usually two to four weeks, to capture weekday and weekend shopping patterns. Set the sample size and end date before launch, and do not stop early because one variant appears to be winning early leads reverse constantly.

What should UAE stores test first with Rollouts?

Start with checkout payment presentation: Tabby and Tamara installment messaging placement, cash-on-delivery visibility, and accelerated checkout sequencing with Shop Pay, Apple Pay, and Google Pay. Test one variable at a time for two to four weeks each. Stores below roughly 15,000 monthly sessions should run qualitative analysis first before investing in A/B tests.