WavJourney leverages Large Language Models to generate structured audio scripts from text, which are then processed by task-specific audio models to create audio content, enabling explainable and human-controlled audio production.
Large Language ModelsWavJourneyaudio storytellingaudio scriptsHF ↗arXiv ↗
PanGu-Coder2, a model fine-tuned with RRTF framework, demonstrates superior code generation performance, outperforming existing Code LLMs on multiple benchmarks.
42RRTFpre-trained large language modelsHF ↗arXiv ↗
A new family of 3D-LLMs is introduced to perform 3D-related tasks by leveraging 3D point clouds and features, outperforming state-of-the-art baselines in tasks such as 3D question answering and captioning.
WebAgent uses Flan-U-PaLM and HTML-T5 to improve task success in real-world web navigation through decomposed instructions, HTML summarization, and Python code generation.
37pre-trained large language modelsLLMsHF ↗arXiv ↗
Chengsong Huang, Qian Liu, Bill Yuchen Lin +3 authors
LoraHub, a framework for combining LoRA modules, enables few-shot cross-task generalization and facilitates community sharing of LRA modules for large language models.
Tamera Lanham, Anna Chen, Ansh Radhakrishnan +27 authors
Larger language models may not produce faithful reasoning when using chain-of-thought despite performance improvements, depending on model size and task.
WebArena, a realistic and reproducible environment, evaluates the performance of autonomous agents performing complex tasks on websites using benchmark tasks that require human-like reasoning.
27autonomous agentsnatural language commandsHF ↗arXiv ↗
TransNormerLLM, a linear attention-based LLM, outperforms softmax attention models through advanced modifications like positional embedding, lightning attention, gating mechanisms, and tensor normalization, achieving better accuracy and efficiency.
Latent diffusion models enable zero-shot image interpolation by denoising in latent space with text embeddings and pose conditions, producing high-quality interpolations across varied subjects, styles, and content.
Marc Botet Colomer, Pier Luigi Dovesi, Theodoros Panagiotakopoulos +6 authors
HAMLET, a hardware-aware modular framework, enables real-time domain adaptation for semantic segmentation at high frame rates with low computational cost.
Tomohiro Sawada, Daniel Paleka, Alexander Havrilla +6 authors
ARB is a novel benchmark featuring advanced reasoning problems across multiple domains, demonstrating that current LLMs score poorly on more challenging tasks and introducing a rubric-based evaluation approach.
Attaching a conformer audio encoder to LLaMA-7B allows it to perform multilingual speech recognition, outperforming monolingual models and maintaining its original text capabilities, even with limited training and longer strides.
18large language modelsspeech recognitionHF ↗arXiv ↗
Subject-Diffusion, a novel open-domain personalized image generation model, uses a unified framework combining text and image semantics, coarse location control, fine-grained reference image control, and attention mechanism to generate accurate and flexible images without the need for fine-tuning or multiple reference images.
NeRF-Det uses NeRF for end-to-end 3D detection with posed RGB images, incorporating geometry priors and shared MLPs to improve performance and generalization.
The framework scales up the generation of language-labeled robot data using a large language model and sampling-based planners, then distills it into a multi-task language-conditioned visuo-motor policy, achieving improved success rates on a new multi-task benchmark.
15large language modelsampling-based robot plannersHF ↗arXiv ↗
HQTrack is a high-quality tracking framework using a video multi-object segmenter and a mask refiner to achieve competitive results in the VOTS2023 challenge.
Ansh Radhakrishnan, Karina Nguyen, Anna Chen +21 authors
Decomposition-based methods improve the faithfulness of Large Language Model reasoning in question-answering tasks, enhancing the verification of model correctness and safety.
PUMA framework accelerates secure Transformer model inference using high-quality approximations and secure procedures, achieving faster performance and maintaining accuracy compared to existing methods.
A large multimodal generative model demonstrates competitive performance across diverse biomedical tasks, including zero-shot generalization and transfer learning, suggesting potential clinical utility.
A new diagnostic benchmark evaluates fact editing methods' impact on related facts, revealing inconsistency and the effectiveness of in-context editing.
A method is proposed to protect NeRF models' copyright by watermarking the color representation with a distortion-resistant rendering scheme, maintaining high rendering quality and bit accuracy.
Kevin Yang, Dan Klein, Asli Celikyilmaz +2 authors
Reinforcement Learning from Contrast Distillation (RLCD) aligns language models to natural language principles using simulated preference pairs without human feedback, outperforming existing methods across various alignment tasks.
11Reinforcement Learning from Contrast Distillation (RLCD)preference modelHF ↗arXiv ↗
A new network architecture optimizes GPU communication for Large Language Model training by partitioning GPUs into high-bandwidth domains, reducing network costs.
11Large Language Models (LLMs)any-to-any networksHF ↗arXiv ↗
Dan Busbridge, Jason Ramapuram, Pierre Ablin +4 authors
A scaling rule for optimization with Exponential Moving Average (EMA) improves performance and enables efficient training of pseudo-labeling and SSL methods, including BYOL, at various batch sizes.
9stochastic gradient descentExponential Moving Average (EMA)HF ↗arXiv ↗
A new Points-to-3D framework bridges the gap between sparse 3D points and realistic text-to-3D generation by integrating knowledge from 2D and 3D diffusion models.
Scott Sanner, Krisztian Balog, Filip Radlinski +2 authors
LLMs demonstrate competitive recommendation performance for language-based preferences, even in zero-shot and few-shot scenarios, compared to item-based collaborative filtering methods.
9large language modelsprompting paradigmsHF ↗arXiv ↗
Code coverage is a widely used metric for quantifying the extent to which
program elements, such as statements or branches, are executed during testing.
Calculating code coverage is resource-intensive, requiring code building and
execution with additional overhead for the instrumentation. Furthermore,
computing coverage of any snippet of code requires the whole program context.
Using Machine Learning to amortize this expensive process could lower the cost
of code coverage by requiring only the source code context, and the task of
code coverage prediction can be a novel benchmark for judging the ability of
models to understand code. We propose a novel benchmark task called Code
Coverage Prediction for Large Language Models (LLMs). We formalize this task to
evaluate the capability of LLMs in understanding code execution by determining
which lines of a method are executed by a given test case and inputs. We curate
and release a dataset we call COVERAGEEVAL by executing tests and code from the
HumanEval dataset and collecting code coverage information. We report the
performance of four state-of-the-art LLMs used for code-related tasks,
including OpenAI's GPT-4 and GPT-3.5-Turbo, Google's BARD, and Anthropic's
Claude, on the Code Coverage Prediction task. Finally, we argue that code
coverage as a metric and pre-training data source are valuable for overall LLM
performance on software engineering tasks.
9Large Language Models (LLMs)Code Coverage PredictionHF ↗arXiv ↗
The proposed Position-conditional Anchor Compositor (PAC) enables text-driven manipulation of NeRF-reconstructed faces using spatially varying latent codes and CLIP embedding optimization, requiring minimal human input.