CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling
Jihai Zhang, Xiaoye Qu, Tong Zhu +1 authors
Diversified Multiplet Upcycling (DMU) enhances CLIP models by fine-tuning them to capture diverse feature spaces, transforming them into CLIP-MoEs, and demonstrating superior performance in various zero-shot tasks and MLLM benchmarks.