OpenFlamingo: An Open-Source Framework for Training Large Autoregressive
  Vision-Language Models

Anas Awadalla; Irena Gao; Josh Gardner; Jack Hessel; Yusuf Hanafy; Wanrong Zhu; Kalyani Marathe; Yonatan Bitton; Samir Gadre; Shiori Sagawa; Jenia Jitsev; Simon Kornblith; Pang Wei Koh; Gabriel Ilharco; Mitchell Wortsman; Ludwig Schmidt

Paper · arXiv 2308.01390

OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models

Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, Ludwig Schmidt

34 upvotesAugust 2, 2023arXiv 预印本

AI 摘要

OpenFlamingo, an open-source family of autoregressive vision-language models, replicates DeepMind's Flamingo performance across multiple datasets.

autoregressive vision-language modelsDeepMind's Flamingovision-language datasets

Abstract

We introduce OpenFlamingo, a family of autoregressive vision-language models ranging from 3B to 9B parameters. OpenFlamingo is an ongoing effort to produce an open-source replication of DeepMind's Flamingo models. On seven vision-language datasets, OpenFlamingo models average between 80 - 89% of corresponding Flamingo performance. This technical report describes our models, training data, hyperparameters, and evaluation suite. We share our models and code at https://github.com/mlfoundations/open_flamingo.