Experimentation Platform
Jump to navigation
Jump to search
An Experimentation Platform is an IT platform that supports the creation of experiment systems.
- Context:
- It can support A/B/N Experiments, Causal Inference Experiments, and Multi-Armed Bandit (MAB)-based Continuous Experiments.
- It can support Automated Analysis.
- It can support Concurrent Experiments.
- It can support Experiment Sizing.
- It can support Experiment Configuration.
- …
- Example(s):
- See: ML Platform.
References
2019
- https://www.microsoft.com/en-us/research/group/experimentation-platform-exp/
- QUOTE: ... Experimentation Platform (ExP) is a team of 60+ Data Scientists, Software Engineers and Program Managers. Our mission is to accelerate innovation through trustworthy experimentation. Most major products such as Bing, Cortana, Edge, Exchange, Identity, MSN, Office client, Office online, Photos, Skype, Speech, Store, Teams, Visual Studio Code, Windows, Xbox use our platform ExP to run trustworthy Online Controlled Experiments – aka A/B tests. …
2018
- https://eng.uber.com/xp/
- QUOTE: ... Uber’s Experimentation Platform (XP) plays an important role in this process, enabling us to launch, debug, measure, and monitor the effects of new ideas, product features, marketing campaigns, promotions, and even machine learning models. The platform supports experiments across our driver, rider, Uber Eats, and Uber Freight apps and is widely used to run A/B/N, causal inference, and multi-armed bandit (MAB)-based continuous experiments.
There are over 1,000 experiments running on our platform at any given time. For example, before Uber launched our new driver app, completely redesigned with our driver-partners in mind, it went through extensive hypothesis testings through a series of experiments conducted with our XP. …
- QUOTE: ... Uber’s Experimentation Platform (XP) plays an important role in this process, enabling us to launch, debug, measure, and monitor the effects of new ideas, product features, marketing campaigns, promotions, and even machine learning models. The platform supports experiments across our driver, rider, Uber Eats, and Uber Freight apps and is widely used to run A/B/N, causal inference, and multi-armed bandit (MAB)-based continuous experiments.