Null Controlled-Treatment Experiment
(Redirected from null controlled-treatment experiment)
Jump to navigation
Jump to search
An Null Controlled-Treatment Experiment is a controlled-treatment experiment where all the treatments are identical.
- AKA: Null Test.
- Context:
- It can range, depending on the number of treatments, from being a Bivariate Null Test(A/A test) to being an Multivariate Null Test.
- It can be used to test an Experimentation System and its Null Hypothesis Test.
- …
- Counter-Example(s):
- See: Null Hypothesis.
References
2007
- (Kohavi et al., 2007) ⇒ Ron Kohavi, Randal M. Henne, and Dan Sommerfield. (2007). “Practical Guide to Controlled Experiments on the Web: Listen to Your Customers Not to the Hippo.” In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. doi:10.1145/1281192.1281295
- QUOTE: A/A Test. Sometimes called an Null Test (Peterson, 2004). Instead of an A/B test, you exercise the experimentation system, assigning users to one of two groups, but expose them to exactly the same experience. An A/A test can be used to (i) collect data and assess its variability for power calculations, and (ii) test the experimentation system (the Null hypothesis should be rejected about 5% of the time when a 95% confidence level is used).
2004
- (Peterson, 2004) ⇒ Eric T. Peterson. (2004). “Web Analytics Demystified: A Marketer's Guide to Understanding How Your Web Site Affects Your Business, s.1". :Celilo Group Media and Cafe Press. ISBN:0974358428.