Salesforce CodeGen2.5 Model
An Salesforce CodeGen2.5 Model is an CodeGen text-to-code model.
- Example(s):
- ...
- Counter-Example(s):
- See: ....
References
2023
2023
- https://blog.salesforceairesearch.com/codegen25/amp/
- QUOTE: The family of Salesforce CodeGen models is growing with CodeGen2.5 – a small, but mighty model! While there has been a recent trend of large language models (LLM) of increasing size, we show that a small model can obtain surprisingly good performance, when being trained well.
- The key contributions towards productization of these models are:
- Releasing CodeGen2.5 LLM with state-of-the-art on HumanEval for 7B parameters.
- CodeGen2.5 with 7B is on par with >15B code-generation models (CodeGen1-16B, CodeGen2-16B, StarCoder-15B), less than half the size.
- Featuring robust infill sampling, that is, the model can “read” text of both the left and right hand size of the current position.
- Optimized for fast sampling under Flash attention for optimized serving and local deployment on personal machines.
- Permissively licensed in Apache 2.0.
- Motivation
- In 2022, Salesforce Research released CodeGen [1,2], one of the first LLMs for Program Synthesis with 16B parameters. The CodeGen model allows users to “translate” natural language, such as English, into programming languages, such as Python. For such models, since the discovery of the scaling laws, power laws which relate the size of a model and the dataset, the dominant trend has been to scale up LLMs to larger sizes.