Controlled Online Experiment

An Controlled Online Experiment is a controlled experiment for online treatments.

• Context:
  • It can range from being an Online Treatment-Controlled Experiment to being an Online Placebo-Controlled Experiment.
  • It can (often) be applied to Website Optimization Tasks.
• Example(s):
  • a Controlled Search Engine Online Experiment.
  • a Controlled Advertisement Engine Online Experiment.
  • …
• Counter-Example(s):
  • an Offline Controlled Experiment.
• See: Website Optimization, Mobile App Optimization, A/B Test.

---

---

References

2013

• (Bottou et al., 2013) ⇒ Léon Bottou, Jonas Peters, Joaquin Quiñonero-Candela, Denis X. Charles, D. Max Chickering, Elon Portugaly, Dipankar Ray, Patrice Simard, and Ed Snelson. (2013). “Counterfactual Reasoning and Learning Systems: The Example of Computational Advertising.” In: The Journal of Machine Learning Research, 14(1).

2009

• (Kohavi et al., 2009) ⇒ Ron Kohavi, Roger Longbotham, Dan Sommerfield, and Randal M. Henne. (2009). “Controlled Experiments on the Web: survey and practical guide.” In: Data Mining and Knowledge Discovery, 18(1). doi:10.1007/s10618-008-0114-1
  • QUOTE: The web provides an unprecedented opportunity to evaluate ideas quickly using controlled experiments, also called randomized experiments, A/B tests (and their generalizations), split tests, Control/Treatment tests, MultiVariable Tests (MVT) and parallel flights.

---