2024 LLM2LLMBoostingLLMswithNovelIte
- (Lee, Wattawong et al., 2024) ⇒ Nicholas Lee, Thanakul Wattanawong, Sehoon Kim, Karttikeya Mangalam, Sheng Shen, Gopala Anumanchipali, Michael W. Mahoney, Kurt Keutzer, and Amir Gholami. (2024). “LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement.”
Subject Headings: LLM Fine-Tuning.
Notes
- LLM2LLM Framework Introduction**: Introduces the LLM2LLM framework, a novel approach leveraging teacher-student model dynamics for targeted and iterative data augmentation, specifically designed to enhance Large Language Models' (LLMs) performance in low-data regimes.
- Targeted Data Augmentation Strategy: Employs a unique strategy where a teacher LLM generates synthetic data based on incorrect predictions by a student LLM, focusing the augmentation efforts on areas where the student model struggles, thereby improving its learning efficiency and effectiveness.
- Significant Performance Improvements**: Demonstrates substantial improvements in LLM performance across various datasets in low-data scenarios, achieving up to 24.2% improvement on the GSM8K dataset, 32.6% on CaseHOLD, and notable gains on SNIPS, TREC, and SST-2 datasets.
- Mitigation of Data Scarcity Issues: Addresses the challenge of fine-tuning LLMs in data-constrained domains by reducing the dependence on extensive data collection and labeling, thus offering a scalable solution for enhancing LLM performance with limited data.
- Iterative and Self-Improving Process: Outlines an iterative process that evaluates the student model's performance, identifies weaknesses, and generates new data to target these specific areas, thereby creating a continuous loop of self-improvement.
- Experimental Validation and Comparisons: Provides extensive experimental results validating the effectiveness of LLM2LLM against traditional fine-tuning and other data augmentation methods, underscoring its superiority in enhancing model performance efficiently.
- Future Research Directions: Suggests avenues for future work, including optimizing the framework's hyperparameters and exploring its integration with other LLM training techniques, indicating the potential for further enhancements and applications of the LLM2LLM approach.
Cited By
Quotes
Abstract
Pretrained large language models (LLMs) are currently state-of-the-art for solving the vast majority of natural language processing tasks. While many real-world applications still require fine-tuning to reach satisfactory levels of performance, many of them are in the low-data regime, making fine-tuning challenging. To address this, we propose LLM2LLM, a targeted and iterative data augmentation strategy that uses a teacher LLM to enhance a small seed dataset by augmenting additional data that can be used for fine-tuning on a specific task. LLM2LLM (1) fine-tunes a baseline student LLM on the initial seed data, (2) evaluates and extracts data points that the model gets wrong, and (3) uses a teacher LLM to generate synthetic data based on these incorrect data points, which are then added back into the training data. This approach amplifies the signal from incorrectly predicted data points by the LLM during training and reintegrates them into the dataset to focus on more challenging examples for the LLM. Our results show that LLM2LLM significantly enhances the performance of LLMs in the low-data regime, outperforming both traditional fine-tuning and other data augmentation baselines. LLM2LLM reduces the dependence on labor-intensive data curation and paves the way for more scalable and performant LLM solutions, allowing us to tackle data-constrained domains and tasks. We achieve improvements up to 24.2% on the GSM8K dataset, 32.6% on CaseHOLD, 32.0% on SNIPS, 52.6% on TREC and 39.8% on SST-2 over regular fine-tuning in the low-data regime using a LLaMA2-7B student model.
References
;
| Author | volume | Date Value | title | type | journal | titleUrl | doi | note | year | |
|---|---|---|---|---|---|---|---|---|---|---|
| 2024 LLM2LLMBoostingLLMswithNovelIte | Michael W. Mahoney Kurt Keutzer Nicholas Lee Thanakul Wattanawong Sehoon Kim Karttikeya Mangalam Sheng Shen Gopala Anumanchipali Amir Gholami | LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement | 2024 |