Software-Generating Language Model
(Redirected from LLMs for program synthesis)
Jump to navigation
Jump to search
A Software-Generating Language Model is a large language model designed to generate computer software code from natural language instructions.
- Context:
- It can (typically) be a type of LLM for Programming.
- It can be trained on human-written code snippets and associated natural language descriptions.
- It can take a natural language specification as input and output code in a programming language like Python, Java, etc.
- It can be evaluated on common benchmarks for code generation from language.
- It can be used to support a Software Programming System.
- ...
- Example(s):
- OpenAI Codex Model is trained to generate code from natural language.
- DeepMind AlphaCode can produce complete functions from problem statements.
- Palm by Anthropic can generate code based on instructions.
- An API Generating LLM which produces API calls.
- ...
- Counter-Example(s):
- A Chatbot LLM optimized for dialogue rather than programming.
- ...
- See: LLM for Programming, Code Generation, Program Synthesis.
References
2023
- (Patil et al., 2023) ⇒ Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. (2023). “Gorilla: Large Language Model Connected with Massive APIs.” DOI:10.48550/arXiv.2305.15334
- QUOTE: Harnessing LLMs for program synthesis has historically been a challenging task (Li et al., 2022), (Chen et al., 2021), (Xu et al., 2022), (Jain et al., 2022), (Devlin et al., 2017), (Lachaux et al., 2020). Researchers have proposed an array of strategies to prompt LLMs to perform better in coding tasks, including in-context learning (Wei et al., 2022), (Kojima et al., 2022), (Chen et al., 2021), task decomposition (Kim et al., 2023), (Yao et al., 2022), and self-debugging (Chen et al., 2023), (Shinn et al., 2023). Besides prompting, there have also been efforts to pretrain language models specifically for code generation (Nijkamp et al., 2022), (Li et al., 2023), (Nijkamp et al., 2023).
However, these strategies focus on prompting large language models or pre-training them for general program synthesis. In our research, in contrast, we focus on a much restricted domain: the synthesis of linear programs using API calls. General program synthesis, not only is complex, but is also hard to verify and evaluate. API calls, on the other hand, function more like tool usage. This allows the LLM to significantly expand its capabilities without grappling with low-level implementation details.
- QUOTE: Harnessing LLMs for program synthesis has historically been a challenging task (Li et al., 2022), (Chen et al., 2021), (Xu et al., 2022), (Jain et al., 2022), (Devlin et al., 2017), (Lachaux et al., 2020). Researchers have proposed an array of strategies to prompt LLMs to perform better in coding tasks, including in-context learning (Wei et al., 2022), (Kojima et al., 2022), (Chen et al., 2021), task decomposition (Kim et al., 2023), (Yao et al., 2022), and self-debugging (Chen et al., 2023), (Shinn et al., 2023). Besides prompting, there have also been efforts to pretrain language models specifically for code generation (Nijkamp et al., 2022), (Li et al., 2023), (Nijkamp et al., 2023).