LIMA: Less Is More for Alignment
Chunting Zhou, Pengfei Liu, Puxin Xu +12 authors
A 65B parameter LLaMa language model trained with minimal instructional data matches or outperforms models with extensive human preference modeling in most cases, indicating that pretraining is predominantly responsible for knowledge acquisition.