Bivariate (A/A) Null Controlled-Experiment Test

An Bivariate (A/A) Null Controlled-Experiment Test is a null controlled-treatment experiment that is also a two-treatment controlled experiment.

• AKA: Bivariate Null Test.
  • …
• Counter-Example(s):
  • an Multivariate Null Test.
  • an A/B Test.
• See: Null Hypothesis.

---

---

References

2007

• (Kohavi et al., 2007) ⇒ Ron Kohavi, Randal M. Henne, and Dan Sommerfield. (2007). “Practical Guide to Controlled Experiments on the Web: Listen to Your Customers Not to the Hippo.” In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. doi:10.1145/1281192.1281295
  • QUOTE: A/A Test. Sometimes called an Null Test (Peterson, 2004). Instead of an A/B test, you exercise the experimentation system, assigning users to one of two groups, but expose them to exactly the same experience. An A/A test can be used to (i) collect data and assess its variability for power calculations, and (ii) test the experimentation system (the Null hypothesis should be rejected about 5% of the time when a 95% confidence level is used).

2004

• (Peterson, 2004) ⇒ Eric T. Peterson. (2004). “Web Analytics Demystified: A Marketer's Guide to Understanding How Your Web Site Affects Your Business, s.1". :Celilo Group Media and Cafe Press. ISBN:0974358428.

---