SingLoRA: Low Rank Adaptation Using a Single Matrix
David Bensaïd, Noam Rotstein, Roy Velich +2 authors
SingLoRA, a reformulated low-rank adaptation method, enhances parameter-efficient fine-tuning by learning a single low-rank matrix and its transpose, ensuring stable optimization and reducing parameter count.