A/B Experimentation

Pre-defined case studies illustrating experiment design, statistical rigor, and learning culture. All data synthetic

Horror Artwork: Tension vs Face

Artwork Optimization · 14-day test · 420,000 members

Significant ✓ Positive

Hypothesis

Showing a tension-focused scene (dark environment, implied threat) as the primary artwork for horror titles will increase take rate versus face-centric artwork, because it better signals genre to browse-mode members.

Control — Control — Face Artwork
8.4% take rate
Variant — Variant — Tension Scene
10.9% take rate
Statistical Lift
+29.8%
p-value
0.003
95% CI[+1.8%, +3.2%]

Lift Over Time

Daily measured lift with 95% confidence interval

Lift stabilised in final days, confirming the result is not a novelty effect.

What we learned

Tension artwork outperformed face artwork by +29.8% on take rate for horror. This aligns with the hypothesis that genre-signaling artwork reduces cognitive load during browse mode — members don't need to recognize the IP to want to click. However, completion rate dipped slightly (−1.3pp), suggesting a small segment clicked expecting a different experience.

Follow-on experiments

  • →Test whether the completion dip is consistent across horror sub-genres (slasher vs psychological)
  • →Explore personalized artwork delivery: show face variant to returning viewers who completed similar titles
  • →Extend test to thriller genre — hypothesis likely generalizes