This is an info Alert.
xpay
  • Product
    • Become Agent-Ready
      • Merchants
        Agentic Commerce — list your store across ChatGPT, Gemini, Claude & Perplexity
      • Publishers
        Monetize your content when AI agents read, cite, or train on it
      • SaaS Companies
        Treat AI agents as first-class customers with agent-priced checkout
    • Monetize
      • Monetize MCP Server
        Charge per call on any MCP server in 2 minutes
      • Monetize AI Agents
        Turn n8n, Zapier, Activepieces workflows into revenue
  • Resources
    • xpay Ecosystem
      • xpay✦ Tools
        1,000+ pay-per-use tools for your AI agents
      • Agent-Ready SaaS Index
        25,481 SaaS scored on agent-buyability
      • SaaS Pricing Database
        Pricing pages indexed across 1,000+ categories
      • Shopify Apps Directory
        Every Shopify app, with its full review history
      • WooCommerce Plugins Directory
        Every WooCommerce plugin, scored on how well it is maintained
      • GitHub
        Open source repositories
    • Agent Building
      • Agent Frameworks
        AI frameworks for building multi-agent systems
      • x402 Integration
        AI frameworks with x402 payment integration
      • Networks
        Blockchain networks supporting x402
    • Company
      • About xpay✦
        Our mission, products, and protocols
      • Blog
        Latest insights and updates
      • Docs
        Complete xpay documentation
  • Pricing
  • Blog
  • Docs
Get Started

  1. xpay✦ Commerce

  3. Directory

  5. Shopify apps

  7. Site optimization - Other

  9. Shogun ‑ AB Testing

Shogun ‑ AB Testing

by Shogun Labs, Inc.

4.2

(44 reviews)

65/100 · SolidBuilt for ShopifyFrom $39/month. Free trial available.
Best suited for stores comfortable with $39/mo.

Maximize Profits with Smarter A/B Testing: Themes, Templates, Pages, Prices, Shipping and More. Take conversion rate optimization to the next level with an AB testing app built specifically for Shopify. Create test variants for: Pages, sections, elements, PDPs, product prices, checkout and shipping within your theme using Shopify’s native editor. Optimize new template designs by testing changes across all associated pages, expanding the test surface area for faster insights and greater revenue potential. Segment tests based on audiences like location, referring URL and more. Take conversion rate optimization to the next level with an AB testing app built specifically for Shopify. Create test variants for: Pages, sections, elements, PDPs, product prices, checkout and shipping within your theme using Shopify’s native editor. Optimize new template designs by testing changes across all associated pages, expanding the test surface area for faster insights and greater revenue potential. Segment tests based on audiences like location, referring URL and more. more A/B test content, pricing, shipping, product pages, and checkout to grow revenue Use AI to find opportunities, build tests & analyze results w/ Claude + others Analyze revenue, profit, COGS, and conversion performance for every test Personalize experiments by behavior, location, traffic source, device & more Connect Meta & Google ad audiences to A/B test performance & conversion

Copy from the Shopify App Store listing, unedited.
View on ShopifyCompare the plansSee the scorecard
Who it’s for
Site optimization - Other
What it moves
Top line

conversion rate

What it costs
From $39/mo

3 tiers

How it’s supported
65/100

Solid · from 42 reviews

What it does

A/B test content, pricing, shipping, product pages, and checkout to grow revenue

Use AI to find opportunities, build tests & analyze results w/ Claude + others

Analyze revenue, profit, COGS, and conversion performance for every test

Personalize experiments by behavior, location, traffic source, device & more

Connect Meta & Google ad audiences to A/B test performance & conversion

Plan fit
What each step up actually buys you

This developer's plans are separated by features, not by how big your store is — so there is no volume that picks a tier for you. Here is what each one adds over the one below it.

Pro
$39/month
You start with

3 Active Tests

Template testing

Theme testing

Page testing

Split URL testing

Advanced
$119/month
Adds

Everything in Pro plus:

5 Active Tests

Test product images and titles

Advanced analytics & goals

MCP access with Claude, ChatGPT & others

Personalize test targeting

Unlimited
$499/month
Adds

Everything in Advanced plus:

Multi-variant testing

Export data

Price testing

Slack channel

Test Checkout Blocks (Shopify Plus)

Listed as “From $39/month. Free trial available.”. Plan contents change — confirm on the listing before you install.
Pricing
From $39/month. Free trial available.
Pro

$39/month

Lowest paid plan
3 Active Tests
Template testing
Theme testing
Page testing
Split URL testing
Advanced

$119/month

Everything in Pro plus:
5 Active Tests
Test product images and titles
Advanced analytics & goals
MCP access with Claude, ChatGPT & others
Personalize test targeting
Meta & Google Ad Integration
Unlimited

$499/month

Everything in Advanced plus:
Multi-variant testing
Export data
Price testing
Slack channel
Test Checkout Blocks (Shopify Plus)
Plans as listed on the Shopify App Store. Always confirm current pricing before installing.

65

/ 100
Solid
low confidence

Scored on 42 merchant reviews we hold for Shogun ‑ AB Testing, across five checks. Every figure is computed from those reviews — expand any row to see the counts behind it.

Recent reviews rate below this app's own average
Reviews in the last 12 months average 4.18, 0.27 stars below the lifetime average of 4.45. Worth reading the newest reviews before installing.
Computed from the 42 reviews we hold, with 12-month windows measured to 2026-09. This scores the public review record — it is not a review of the software itself, and a low score often means we hold little evidence rather than poor evidence.
Review history
2025–2026 · 42 reviews read
Reviews per year
202517 reviews
202625 reviews
What merchants say

4.2

44 reviews
5★81%
4★7%
3★0%
2★0%
1★12%
Developer replies
95%
of reviews get an answer
Reviewed at 12m+
3%
had used it a year when they wrote
From 42 reviews read. Typical reviewer had used the app for 1 months. Most merchants review in their first month, so this reflects when people write — not how long they stay.
Recent merchant reviews

The newest 8 of 44. Open one to read it in full, with the developer's reply where there is one.

Very quickly responded and helped in a great way. Also the app improvements show every KPI you need, with a lot of filters to filter on what actually works.

Developer repliedThank you so much for the fantastic feedback! We’re glad to hear our support team was able to help quickly and effectively. It’s also great to know you’re finding the reporting and filtering options useful for identifying what’s actually working. We’re always looking to make Shogun A/B Testing more powerful and easier to use, so we really appreciate your support! Sweetan | Client Services Shogun Show more

This app has been super useful for theme & template testing for us! The support team is very responsive and helpful when troubleshooting issues, a great team to work with!

Developer repliedHi there, thanks for sharing your feedback with us. We're thrilled to hear that Shogun A/B Testing has been valuable for your theme and template testing. It's also wonderful to know our support team has been responsive and helpful whenever you've needed assistance. We truly appreciate your kind words and look forward to continuing to support your optimization efforts. Please let us know if there is anything we can assist you with. Thanks for being a part of the Shogun community! :) Sweetan | Client Services Shogun Show more

Amazing app, good to do A/B test. Very easy to understand the metrics and setup tests

Developer repliedThank you so much for the wonderful feedback! We're delighted to hear you're enjoying the app and finding it easy to set up A/B tests and understand the results. We believe experimentation should be simple and actionable, so it's great to know the metrics and workflow are helping you make data-driven decisions. We truly appreciate your support! Sweetan | Client Services Shogun Show more

We've had a great experience using Shogun's A/B Testing app as part of a major website refresh and optimization initiative for IGK Hair. Rather than relying on assumptions, we were able to test multiple PDP layouts, content hierarchies, and merchandising strategies with real customer traffic to understand what truly resonated with our shoppers. The insights we gained were instrumental in shaping our rebuild and gave us the confidence to make data-driven decisions throughout the redesign process. The platform was easy to implement, the testing experience was seamless, and the reporting made it simple to identify winning experiences. If you're planning a website redesign or looking to optimize your PDPs, we highly recommend Shogun A/B Testing. It's been a valuable part of our CRO toolkit and helped us launch a stronger, more customer-centric shopping experience.

Developer repliedThank you so much for taking the time to share such a thoughtful review! We're thrilled to hear that Shogun A/B Testing played a valuable role in your website refresh and optimization efforts. It's especially rewarding to know the platform helped you validate decisions with real customer data rather than assumptions. Enabling teams to confidently test PDP layouts, content strategies, and merchandising experiences is exactly what we designed the app to do, so it's great to hear the insights helped shape your redesign. We're also glad you found the implementation, testing workflow, and reporting intuitive and seamless. Thank you for making Shogun a part of your CRO toolkit and for recommending us to others. We look forward to supporting your optimization journey as you continue to grow! Sweetan | Client Services Shogun Show more

WORST APP EVER - MY PRODUCT PHOTOS LAGGED AND DID NOT WORK / THE PRICE EVEN AFTER UN INSTALLING KEPT CHANGING . Subject: Re: Toothy - pricing test issue Hi Sweetan, Thanks for looking into this. Before reinstalling or sharing collaborator access, I want to share what I've found on my end, because I think it points to the actual cause. After uninstalling Shogun, I found that Shogun had written code directly into my theme files rather than only using app embeds/blocks. Specifically: 1. A snippet called `snippets/product-media-gallery.liquid` had Shogun-authored Liquid code reading from metafields under the `app--214825828353` namespace (`product-pricing`, `product-variant-pricing`, `test-media-1` through `test-media-4`). This code used `{% paginate %}` on values that aren't valid paginateable objects, which caused a "Liquid error: Array is not paginateable" on my live product pages. 2. Separately, I found a file called `snippets/shogun-optimizer.liquid`, labeled in its own comments as "Auto-generated by Shogun-AB." This file contains embedded, minified JavaScript that appears to run content-type detection across templates and includes a `cartCreate` GraphQL mutation. This file was still present and apparently still active on my storefront after I had already uninstalled the app. 3. The pricing metafields under `app--214825828353` also still exist on my products, since metafields persist independently of whether the app is installed. This is very likely why the price kept changing even after uninstall — the leftover Liquid/JS was still reading that stored data. Given this, before I reinstall anything or share collaborator access, I'd like to understand: - Does reinstalling the app actually remove or replace `shogun-optimizer.liquid` and the leftover metafields, or does it just resume using them? - Is there a clean uninstall process that removes all injected snippets and metafields, rather than leaving them orphaned in the theme? - Can you confirm whether the `cartCreate` mutation in that script ever affects the price actually charged at checkout, separate from what's displayed on the page? I want to be sure no customers were charged incorrectly during the test. I'd rather resolve this by removing the leftover code and metafields directly, since I already have a good picture of what was left behind. If there's a specific reason full collaborator access is necessary beyond what I've described above, let me know and I'm happy to discuss it further. Thanks,

Developer replied---UPDATE - July 13, 2026--- Hey there, Hamad. I just wanted to check if you received the last update and if that has helped. I am also reaching out via the email thread, and we'd appreciate it if you could reply back to us. Thanks for your patience and understanding. Sweetan | Client Services Shogun ----------------------------------------------- ---UPDATE - July 8, 2026--- Hi Hamad, Thank you again for taking the time to investigate this so thoroughly. We shared your findings with our engineering team, and I wanted to address each of the points you raised. Regarding `snippets/shogun-optimizer.liquid`, you're correct that this file can remain in the theme after the app is uninstalled. Once the app is uninstalled, Shopify immediately invalidates our API tokens, which means we're no longer able to modify the theme or remove files as part of a cleanup process. That said, the optimizer includes a safeguard where, if it hasn't received a refreshed configuration for approximately 24 hours, it automatically stands down. This is intended to prevent visitors from continuing to receive experiments after the app has been uninstalled. With respect to `snippets/product-media-gallery.liquid`, this is an existing theme snippet that's responsible for rendering your product image gallery. During an AI Theme Modification, this is one of the files that is commonly updated so that Shogun can swap in the appropriate media for products participating in a test. The behavior you described- reading the media metafields and displaying the corresponding images for the active variation is expected. You also asked whether a syntax error could have been introduced. While our AI-generated modifications go through validation, no automated process is perfect, and it's possible for an issue to slip through in certain scenarios. This is why we recommend reviewing the modified theme and verifying that key pages are functioning as expected before publishing it. We'd be happy to review your theme with you or help rerun the theme modification if necessary to check this better. Regarding the product metafields, our expectation is generally that Shopify manages app-owned data on uninstall. However, we've seen reports from other developers that, in some cases, app-owned metafields can remain after an app is removed due to Shopify platform behavior. To answer your remaining questions: * Reinstalling the app **does not** automatically restart or resume paused tests. It performs the necessary setup, but any previously paused optimization remains paused unless you explicitly start it again. * The `cartCreate` mutation does **not** modify the price charged at checkout. It simply passes the selected variant to the cart. It has no effect on the final checkout pricing. * The cart transform function is managed by Shopify and is automatically removed when the app is uninstalled, so it is no longer active after uninstall. It is possible you were seeing the cached version on your end. We really appreciate the effort you've put into documenting what you found. Collaborator access would still be valuable because it would allow our engineering team to inspect the current state of your theme and configuration directly, verify whether there are any lingering issues specific to your store, and determine whether there's anything we can improve on our side. If you're open to it, we'd be happy to continue the investigation together. Thanks again for your detailed report, and I look forward to hearing from you. Sweetan Shogun Support ------------------------------------------- Hi Hamad, we're sorry to hear about your experience. I see you reinstalled our app on July 2, ran the test for a few hours, stopped the test, and then uninstalled the app. We weren't able to locate any support conversations under the email address associated with the account. If you contacted us using a different email address, please let us know; we'd be happy to look into that correspondence. That should help us to analyze this better for you. Regarding the pricing, when a pricing test is started, paused, or stopped, Shogun updates the related data to ensure pricing remains consistent across your storefront. That said, we'd like to investigate your specific experience thoroughly to understand what happened. I've reached out to you via email to gather a few additional details, and I'll also be sending a Shopify Collaborator access request so our technical team can investigate. We'd really appreciate your reply so we can get to the bottom of this and work toward a resolution. Thanks. Sweetan | Client Services Shogun Show more

Shogun A/B testing is really poor and full of bugs. My store theme got unpublished without notice. It publish duplicated theme that created month ago. Now my entire store is messed up. This kind of sneaky conduct has caused me order losses. I paid $119 for the service, but the product detail testing function fails to work. I cannot even edit and adjust product descriptions normally.

Developer repliedHey there, Shenghai. Thanks for sharing your feedback. We apologize for the inconvenience you faced that has led to this feedback. I was looking at the use case and your account, and I want to let you know that when you start/run a product details test, we set up a duplicate theme that has all the required adjustments for you to make the changes to the variant. We then publish this modified theme. You can treat the modified theme as the new main theme moving forward, and modify that in case you wish to modify anything in the theme. However, if you want, once the test ends, you can republish the old theme again. Also, about the product details, I wanted to check if you had made changes to the description on the original. When you change the product description, you may want to re-run theme modifications if you have already run it on a previous version. This ensures the theme is set correctly and you can make changes to the variant B of your test. I see you had run a few product details tests previously, and they were all for the same product. Having said that, we would love to analyze the issue for you and help you better. I have reached out to you via the same chat thread and we would appreciate if you could reply back to us We appreciate your support. Sweetan | Client Services Shogun Show more

This app already helped us a lot with optimizing our CRO! The dashboard works easy and intuitive and gives us insights we didn't have before.

Great A/B testing and very simple to use. I had a question and Sweetan helped me out and went in depth. Thank you

Developer repliedHi there, Thanks for taking the time and leaving us a review. I am so glad that I was able to help and answer the question. We are here to help and if you bump into any issues, please don't hesitate to reach out. Thanks for your support. Sweetan | Client Services Shogun Show more

Shogun ‑ AB Testing FAQ

Answered from this app's own listing and its full review record.

Best suited for stores comfortable with $39/mo.

Paid plans start at $39/month, and run to $499/month across 3 published tiers. Prices are as listed on the Shopify App Store and can change — confirm before installing.

Shogun ‑ AB Testing's plans are separated by features rather than by store size — none of the tiers is capped on a volume you could state up front. Compare what each step adds over the one below it rather than trying to size yourself into a tier.

On our scorecard Shogun ‑ AB Testing rates 65 out of 100 — solid — read from 42 merchant reviews. The developer has published replies to 40 of 42 reviews. The score reflects what the review record shows, nothing more.

44 merchants have reviewed Shogun ‑ AB Testing, averaging 4.2 out of 5. We hold the review history from 2025 to 2026, which is what the year-by-year chart on this page is built from.

Quick facts
At a glance
Merchant rating

4.2 / 5 · 44 reviews

Rating, last 12 months

4.18 from 28 reviews

New reviews, last 12 months

28 (was 14)

Developer replies

40 of 42 reviews

Typical reviewer had used it

1 months

Pricing

From $39/month. Free trial available.

On the App Store since

March 21, 2025

Review history held

2025 – 2026

Integrates with
Checkout
Shopify Admin
Categories
Site optimization - Other
Will AI shoppers find this store?

Apps like this one convert the traffic you already have. Getting recommended by ChatGPT and Perplexity is a separate problem.

Run a free readiness check
Alternatives
Instead of Shogun ‑ AB Testing

Same categories, by review volume.

Microsoft Clarity: AI Insights
4.61,756 reviews
Lucky Orange Heatmaps & Replay
4.7804 reviews
Translate store ‑ GTranslate
4.7645 reviews
Propel Replay, Survey, Heatmap
4.9591 reviews
Plug in SEO
4.5532 reviews
MIDA Replay, Heatmap & Insight
4.9453 reviews
xpay

The agent-readiness stack for the AI shopping era — helping merchants, publishers and SaaS companies get discovered, cited and transacted with by ChatGPT, Perplexity, Claude, Gemini and the custom shopping agents underneath them.

CompanyAgentically Inc. (d/b/a xpay✦)1875 Mission St, Ste 103San Francisco, CA 94103, United Stateslegal@xpay.sh · privacy@xpay.sh
or ask your AI app
Company
About xpayAgency PartnersGitHubDiscordllms.txt
DevelopersDocumentationAPI ReferenceSDKs & LibrariesQuickstart GuideOpenAPI Spec
Stay Updated
Occasional product updates and agent-readiness playbooks from xpay (Agentically Inc.) — typically a couple of emails a month. Double opt-in: we email you a link to confirm before sending anything, and every email has one-click unsubscribe.
Social
  • For Publishers
    • News
    • Finance
    • Dev / Tech
    • Travel
    • View all verticals
  • Agent-Ready Index
    • SaaS Pricing Database
    • Agent-Ready SaaS Index
    • Verified band
    • AI & ML
    • Sales & CRM
  • Products
    • Pricing Widget
    • Monetize MCP Server
    • Paywall
    • Smart Proxy
    • Monetize AI Agents
    • xpay x402 Facilitator
  • Agentic Economy
    • Timeline
    • Resources
    • Manifesto
    • Stack
  • Agentic Commerce
    • Get listed
    • ChatGPT Ads
    • How ChatGPT Ads work
    • ChatGPT Ads · Apparel
    • ChatGPT Ads · Health & Beauty
    • xpay Listings · Amazon + Google
    • Pricing
    • Free audit
    • Shopify
    • WooCommerce
    • Apparel & Accessories
    • Health & Beauty
    • Overview
  • Commerce Index
    • Shopify apps directory
    • Agentic Commerce Ready Index
    • Methodology
    • Pet brands · WooCommerce
    • Pet brands · Shopify
  • Marketplace
    • 🛍️ xpay.deals — agentic storefront for deals
  • Protocols
    • Overview
    • x402
    • MPP
    • UCP
    • ACP
    • AP2
    • TAP
    • A2A
  • Agent Frameworks
    • Overview
    • LangChain
    • CrewAI
    • Claude MCP
    • AutoGPT
    • LangChain vs Mastra
    • LangGraph vs Pydantic AI
  • Company
    • About xpay
    • Blog
    • Docs
    • GitHub
  • Free prompts
    • Ecommerce prompts
    • Email marketing prompts
    • Product description prompts
    • Facebook ad prompts
    • Skincare prompts
    • Supplement prompts
    • Wine prompts
    • Electronics prompts
© 2025 Agentically Inc. All rights reserved.
Privacy PolicyTerms of UseAcceptable Use Policy