Reformulation for Pretraining Data Augmentation
Xintong Hao, Ruijie Zhu, Ge Zhang +2 authors
The Massive Genre-Audience reformulation method augments training data to reduce repetition-related degradation and supports more efficient scaling of large language models.