PaLM-E Model

A PaLM-E Model is a large neural language model.

• AKA: Pretraining-augmented Language Model with External knowledge.
• Context:
  • It can be an extension of the original PaLM model.
  • It can incorporate external knowledge sources such as knowledge graphs, dictionaries, and other structured data sources.
  • ...
• See: PaLM Model.

---

---

References

2023

• https://ai.googleblog.com/2023/03/palm-e-embodied-multimodal-language.html
  • QUOTE: Today we introduce PaLM-E, a new generalist robotics model that overcomes these issues by transferring knowledge from varied visual and language domains to a robotics system. We began with PaLM, a powerful large language model, and “embodied” it (the “E” in PaLM-E), by complementing it with sensor data from the robotic agent. This is the key difference from prior efforts to bring large language models to robotics — rather than relying on only textual input, with PaLM-E we train the language model to directly ingest raw streams of robot sensor data. The resulting model not only enables highly effective robot learning, but is also a state-of-the-art general-purpose visual-language model, while maintaining excellent language-only task capabilities.

---