Facebook FBLearner Platform
A Facebook FBLearner Platform is an in-house ML platform for Facebook org..
- Context:
- It can contain an ML Training Platform, FBLearner Flow.
- …
- Counter-Example(s):
- See: Workflow, Data Preparation Platform, ML Evaluation Platform, Metrics Standardization, Experiment Management.
References
2020
- https://linkedin.com/in/adityakalro/
- QUOTE: ... We build Infra and services to make ML as a process easier for engineers at Facebook with infra to serve the entire ML process including Feature Engineering, Data Processing, Training and Inference including higher level abstractions to tie all of these pieces together.
FBlearner is used by teams across Facebook to facilitate the training and publishing of ML models including Ads, Feed, Instagram and Integrity. …
- QUOTE: ... We build Infra and services to make ML as a process easier for engineers at Facebook with infra to serve the entire ML process including Feature Engineering, Data Processing, Training and Inference including higher level abstractions to tie all of these pieces together.
2019
- (Wu et al., 2019) ⇒ Carole-Jean Wu, David Brooks, Kevin Chen, Douglas Chen, Sy Choudhury, Marat Dukhan, Kim Hazelwood et al. (2019). “Machine Learning at Facebook: Understanding Inference at the Edge.” In: Proceedings of the 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA).
- QUOTE: … Facebook products. As an example, FBLearner is an ML platform that automates many tasks such as training on clusters and is the tool of choice by many teams experimenting and developing custom ML at Face- book. In addition …
2016
- https://engineering.fb.com/2016/05/09/core-data/introducing-fblearner-flow-facebook-s-ai-backbone/
- QUOTE: ... We decided to build a brand-new platform, FBLearner Flow, capable of easily reusing algorithms in different products, scaling to run thousands of simultaneous custom experiments, and managing experiments with ease. This platform provides innovative functionality, like automatic generation of UI experiences from pipeline definitions and automatic parallelization of Python code using futures. FBLearner Flow is used by more than 25 percent of Facebook's engineering team. Since its inception, more than a million models have been trained, and our prediction service has grown to make more than 6 million predictions per second.
Eliminating manual work required for experimentation allows machine learning engineers to spend more time on feature engineering, which in turn can produce greater accuracy improvements. Engineers are able to have an impact at scale — there are fewer than 150 workflow authors, and their efforts have an impact beyond themselves and their team. FBLearner Flow provides the platform and tools to enable engineers to run thousands of experiments every day.
- Core concepts and components
Before we take a closer look at the system, there are a few key concepts to understand:
- Workflows: A workflow is a single pipeline defined within FBLearner Flow and is the entry point for all machine learning tasks. Each workflow performs a specific job, such as training and evaluation of a specific model. Workflows are defined in terms of operators and can be parallelized.
- Operators: Operators are the building blocks of workflows. Conceptually, you can think of an operator like a function within a program. In FBLearner Flow, operators are the smallest unit of execution and run on a single machine.
- Channels: Channels represent inputs and outputs, which flow between operators within a workflow. All channels are typed using a custom type system that we have defined.
- The platform consists of three core components: an authorship and execution environment for custom distributed workflows, an experimentation management UI for launching experiments and viewing results, and numerous predefined pipelines for training the most commonly used machine learning algorithms at Facebook. …
- QUOTE: ... We decided to build a brand-new platform, FBLearner Flow, capable of easily reusing algorithms in different products, scaling to run thousands of simultaneous custom experiments, and managing experiments with ease. This platform provides innovative functionality, like automatic generation of UI experiences from pipeline definitions and automatic parallelization of Python code using futures. FBLearner Flow is used by more than 25 percent of Facebook's engineering team. Since its inception, more than a million models have been trained, and our prediction service has grown to make more than 6 million predictions per second.