A benchmark called Geoperception is introduced to evaluate MLLMs' geometric description accuracy, leading to the development of Euclid, a model optimized for low-level geometric perception using synthetic data and a data curriculum.
53multimodal large language modelslow-level visual perceptionHF ↗arXiv ↗
A parallel generation strategy for autoregressive models improves inference speed without significantly compromising quality in visual generation tasks.
The use of synthetic data in language model training leads to model collapse, which is mitigated by token-level editing of human-produced data to create semi-synthetic data.
A benchmark platform evaluates AI agents' autonomy in performing professional tasks in a simulated workplace environment, showing that a significant portion of simpler tasks can be automated but complex, long-term tasks remain challenging.
LG AI Research, Soyoung An, Kyunghoon Bae +30 authors
EXAONE 3.5 language models, available in three configurations, demonstrate exceptional instruction following, long-context comprehension, and competitive performance across various benchmarks.
51instruction-tuned language modelslong-context comprehensionHF ↗arXiv ↗
LatentLM integrates continuous and discrete data using causal Transformers, VAEs, and next-token diffusion, achieving superior performance in multimodal tasks like image generation, large language model integration, and text-to-speech synthesis.
Zhisheng Zhong, Chengyao Wang, Yuqi Liu +12 authors
As Multi-modal Large Language Models (MLLMs) evolve, expanding beyond
single-domain capabilities is essential to meet the demands for more versatile
and efficient AI. However, previous omni-models have insufficiently explored
speech, neglecting its integration with multi-modality. We introduce Lyra, an
efficient MLLM that enhances multimodal abilities, including advanced
long-speech comprehension, sound understanding, cross-modality efficiency, and
seamless speech interaction. To achieve efficiency and speech-centric
capabilities, Lyra employs three strategies: (1) leveraging existing
open-source large models and a proposed multi-modality LoRA to reduce training
costs and data requirements; (2) using a latent multi-modality regularizer and
extractor to strengthen the relationship between speech and other modalities,
thereby enhancing model performance; and (3) constructing a high-quality,
extensive dataset that includes 1.5M multi-modal (language, vision, audio) data
samples and 12K long speech samples, enabling Lyra to handle complex long
speech inputs and achieve more robust omni-cognition. Compared to other
omni-methods, Lyra achieves state-of-the-art performance on various
vision-language, vision-speech, and speech-language benchmarks, while also
using fewer computational resources and less training data.
48Multi-modal Large Language Models (MLLMs)omni-modelsHF ↗arXiv ↗
DiffSensei, a diffusion-based framework with a multimodal large language model, enhances manga generation by precisely controlling multi-character appearances and interactions using masked cross-attention and text cues.
48diffusion-basedmultimodal large language modelHF ↗arXiv ↗
A human-curated benchmark (CodeArena) and a large synthetic instruction corpus (SynCode-Instruct) are introduced to evaluate code LLMs based on human preference alignment, revealing performance differences between open-source and proprietary models.
48code large language modelscode generationHF ↗arXiv ↗
FLOAT method generates high-quality, temporally consistent, and emotion-enhanced talking portraits using flow matching in a learned motion latent space with a transformer-based vector field predictor.
AgoraBench evaluates language models' data generation abilities, revealing their distinct strengths and demonstrating various factors that impact effectiveness.
LiFT fine-tunes text-to-video generative models using human feedback to improve video alignment with text descriptions, demonstrating superior performance over a larger model.
A large-scale multimodal instruction-tuning dataset with intermediate rationales improves reasoning capabilities in large language models, achieving state-of-the-art performance across benchmarks.
46multimodal large language modelsinstruction-tuning datasetHF ↗arXiv ↗
Sumeet Ramesh Motwani, Chandler Smith, Rocktim Jyoti Das +6 authors
Multi-agent LLM training improves performance on reasoning tasks by assigning specialized roles and utilizing joint outcome-based rewards to enhance collaboration among models.
GRAPE improves vision-language-action models' performance by aligning policies via preference modeling, enhancing generalizability and allowing customization of objectives like safety and efficiency.
A noise-robust supervised fine-tuning framework enhances large language models' performance in downstream tasks by detecting and relabeling noisy data.
45supervised fine-tuninglarge language modelsHF ↗arXiv ↗
O1-CODER integrates reinforcement learning and Monte Carlo Tree Search to enhance coding capabilities, focusing on pseudocode and full code generation through iterative fine-tuning and standardized code testing.
45reinforcement learningMonte Carlo Tree SearchHF ↗arXiv ↗