OpenDevin is a platform for developing AI agents that interact with the world by writing code, using command lines, and browsing the web, with support for multiple agents and evaluation benchmarks.
large language modelsOpenDevinAI agentscode executionHF ↗arXiv ↗
Chain-of-Diagnosis (CoD) enhances interpretability in LLM-based medical diagnostics by providing a transparent reasoning pathway and developing DiagnosisGPT, which diagnoses a wide range of diseases with high accuracy and controllable rigor.
55large language models (LLMs)Chain-of-Diagnosis (CoD)HF ↗arXiv ↗
Enhancements to the AgentScope platform improve scalability, efficiency, and ease of use for large-scale multi-agent simulations through distributed mechanisms, flexible environments, and user-friendly tools.
LazyLLM dynamically selects important prompt tokens to accelerate the prefilling stage in transformer-based language models without sacrificing accuracy.
46transformer-based large language modelsKV cacheHF ↗arXiv ↗
A multi-modal language model using cross-attention, hierarchical ViT features, and Mixture of Experts mechanism achieves competitive performance in image and video captioning tasks with reduced computational costs.
44cross-attentionhierarchical ViT featuresHF ↗arXiv ↗
A comprehensive comparison of KAN and MLP models across diverse tasks reveals that MLP generally outperforms KAN except in symbolic formula representation where KAN's B-spline activation function provides advantage, and KAN exhibits more severe forgetting issues in class-incremental continual learning.
Diffree, a Text-to-Image model using Stable Diffusion with a mask prediction module, achieves the addition of text-guided objects while maintaining background consistency and spatial appropriateness.
A novel data augmentation approach iteratively improves visual language model data quality and performance using self-augmentation and specialist-augmentation, leading to state-of-the-art results on MMMU tasks.
41visual language modelslarge language modelsHF ↗arXiv ↗
A theoretical framework called Internal Consistency and a Self-Feedback framework with Self-Evaluation and Self-Update modules are presented to address reasoning and hallucination in large language models by assessing coherence across model layers.
40large language modelsself-consistencyHF ↗arXiv ↗
SlowFast-LLaVA captures detailed spatial and temporal features in videos using a two-stream design, outperforming existing training-free methods and matching fine-tuned Video LLMs.
LAMBDA is an open-source, code-free multi-agent system that uses advanced models and human intervention to perform iterative and generative data analysis through natural language.
Compressing large language models through strategic pruning and knowledge distillation reduces training compute costs and achieves performance comparable or better than similarly sized models trained from scratch.
Jaden Fiotto-Kaufman, Alexander R Loftus, Eric Todd +17 authors
The enormous scale of state-of-the-art foundation models has limited their
accessibility to scientists, because customized experiments at large model
sizes require costly hardware and complex engineering that is impractical for
most researchers. To alleviate these problems, we introduce NNsight, an
open-source Python package with a simple, flexible API that can express
interventions on any PyTorch model by building computation graphs. We also
introduce NDIF, a collaborative research platform providing researchers access
to foundation-scale LLMs via the NNsight API. Code, documentation, and
tutorials are available at https://www.nnsight.net.
The paper examines knowledge mechanisms in Large Language Models, focusing on utilization (memorization, comprehension, application, and creation) and evolution, addressing the fragility of parametric knowledge and potential dark knowledge.
34Large Language Modelsknowledge utilizationHF ↗arXiv ↗
BetterDepth, a conditional diffusion-based refiner, enhances zero-shot monocular depth estimation by refining pre-trained model predictions with global and local context while efficiently training on synthetic datasets.
The Android Multi-annotation EXpo (AMEX) dataset supports training and evaluating generalist mobile GUI-control agents with detailed multi-level annotations for complex task completion.
MovieDreamer integrates autoregressive models with diffusion rendering to produce high-fidelity, long-form video content with coherent narratives and consistent characters.
OutfitAnyone uses a two-stream conditional diffusion model to achieve high-fidelity garment deformation in virtual try-on scenarios, addressing challenges in control and consistency across diverse images and conditions.
An open-access text-to-audio model trained with Creative Commons data achieves competitive performance, particularly in high-quality stereo sound synthesis.
The study assesses and enhances large language models' capability to autonomously avoid harmful content generation through a benchmark and preference learning fine-tuning.
A new dataset, HumanVid, for human image animation includes high-quality real-world and synthetic video data with human and camera motion annotations, achieving state-of-the-art results in controlling both human pose and camera motions.
This study introduces T2V-CompBench, a benchmark for evaluating the compositional capabilities of text-to-video models, including metrics for consistent and dynamic attributes, spatial relationships, motion and action binding, object interactions, and generative numeracy.
ChatQA 2, a Llama3-based model, extends the context window to 128K tokens and enhances RAG performance, achieving accuracy comparable to GPT-4-Turbo on long-context tasks and surpassing it on RAG benchmarks.
The DDK framework dynamically adjusts the distillation dataset to improve performance differences between teacher and student LLMs across various domains, outperforming existing methods.
Gagan Bhatia, El Moatez Billah Nagoudi, Fakhraddin Alwajih +1 authors
Qalam, a foundation model using SwinV2 and RoBERTa architectures, achieves exceptional accuracy in Arabic OCR and HWR with low Word Error Rates and robust handling of diacritics and high-resolution inputs.
Data mixture inference using byte-pair encoding (BPE) tokenizers uncovers the distribution of training data in language models, revealing proportions of various domains and languages.
Alexey Skrynnik, Anton Andreychuk, Anatolii Borzilov +3 authors
POGEMA provides a framework for learning and evaluating multi-agent reinforcement learning (MARL) methods, enabling fair comparisons with classical and hybrid approaches in tasks such as multi-robot navigation.
Louis Castricato, Nathan Lile, Rafael Rafailov +2 authors
PERSONA is a reproducible test bed that evaluates and enhances the pluralistic alignment of language models using procedurally generated synthetic personas and a large-scale evaluation dataset.
LongVideoBench is a question-answering benchmark featuring interleaved video-language inputs up to an hour long, designed to evaluate long-term multimodal understanding and referring reasoning.