MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
Ting Jiang, Shaohan Huang, Shengyue Luo +8 authors
MoRA, a high-rank updating method using square matrices, enhances the ability of large language models to learn and memorize new knowledge, especially in memory-intensive tasks, compared to LoRA.