A/B testing

Every funnel gets a control variant + at least one experimental variant on first publish. Variants are routed at the Cloudflare edge using cookies so each visitor sees a stable experience.

Funnel A/B variants

Variant recipes are per-category. Examples:

• Urgency variant. Aggressive red accents, "Available Now" banners, sticky mobile CTAs.
• Trust variant. Calmer blue accents, prominent licensing badges, review counts.
• Short-form variant. 3 steps instead of 5 (test for emergency intent).

Emergency hybrid landing pages get their own variant recipes (red accent for higher urgency, shorter response time copy, social proof variant).

The Cloudflare Worker reads the visitor's vba_var_* cookie. New visitors get assigned to a variant based on a deterministic hash. AdsBot-Google and Googlebot always see the control to avoid disrupting ad crawls.

Statistical significance

The system uses a two-proportion z-test at 95% confidence to decide when a variant has reliably out-converted the control. Threshold:

• Minimum 100 visitors per variant
• z-score absolute value greater than 1.96
• Lift in either direction over 5 percentage points relative

Until significance, both variants continue receiving traffic. Variants are tagged in the dashboard as Running, Significant, or Winner.

Auto winner declaration

Two auto-executable funnel actions on Pro and Max:

• declare_ab_winner. When confidence is over 95% AND each variant has more than 100 visitors, the agent sets is_winner=true on the winning variant, deactivates losers, and copies the winning HTML to the base page. New visitors all get the winning experience.
• pause_losing_variant. When a variant's CVR is under 50% of the winning variant's CVR (z-test significant), the agent pauses it without yet declaring a final winner. The remaining variants keep testing.

Bandit Lab (Phase 41/42)

Beyond funnel-level A/B tests, VibeAds runs element-level Thompson sampling experiments on individual funnel elements: sticky CTA copy, hero image, step copy template. Each element has its own arms with Beta posteriors. At runtime, the edge worker samples from the posteriors and serves the winning arm.

Hierarchical priors (Phase 42) seed new arms with fleet-wide category-level evidence so low-traffic campaigns benefit from what worked across other campaigns in the same category. A plumber funnel with 30 visitors a week inherits ~20% of the plumber-category-wide prior, then quickly outweighs it as it accumulates local data.

Bandit Lab is read-only in the dashboard today. The Tracking tab on the campaign detail page (Max tier) shows live arm statistics: impressions, conversions, posterior probability of being best, locked or running status.