Floating-Point Format (bfloat16)

A Floating-Point Format (bfloat16) is a Computer Number Format occupying 16 bits in computer memory.

• Example(s):
  • 3f80 = 0 01111111 0000000 = 1,
  • c000 = 1 10000000 0000000 = −2,
  • 0000 = 0 00000000 0000000 = 0,
  • 8000 = 1 00000000 0000000 = −0,
  • …
• Counter-Example(s):
  • Half-Precision Floating-Point Format.
  • Single-Precision Floating-Point Format.
  • Double-Precision Floating-Point Format.
• See: Mixed-Precision Arithmetic, 16-Bit, Exponent Bias, 32-Bit, Binary32 Format, 8-Bit Precision.

---

---

References

2021

• (Wikipedia, 2021) ⇒ https://en.wikipedia.org/wiki/bfloat16_floating-point_format Retrieved:2021-8-31.
  • The bfloat16 (Brain Floating Point) floating-point format is a computer number format occupying 16 bits in computer memory; it represents a wide dynamic range of numeric values by using a floating radix point. This format is a truncated (16-bit) version of the 32-bit IEEE 754 single-precision floating-point format (binary32) with the intent of accelerating machine learning and near-sensor computing. It preserves the approximate dynamic range of 32-bit floating-point numbers by retaining 8 exponent bits, but supports only an 8-bit precision rather than the 24-bit significand of the binary32 format. More so than single-precision 32-bit floating-point numbers, bfloat16 numbers are unsuitable for integer calculations, but this is not their intended use. Bfloat16 is used to reduce the storage requirements and increase the calculation speed of machine learning algorithms. The bfloat16 format was developed by Google Brain, an artificial intelligence research group at Google. The bfloat16 format is utilized in Intel AI processors, such as Nervana NNP-L1000, Xeon processors (AVX-512 BF16 extensions), and Intel FPGAs, Google Cloud TPUs, and TensorFlow. ARMv8.6-A, AMD ROCm, and CUDA also support the bfloat16 format. On these platforms, bfloat16 may also be used in mixed-precision arithmetic, where bfloat16 numbers may be operated on and expanded to wider data types.

---