Reinforcement Pre-Training
Qingxiu Dong, Li Dong, Yao Tang +4 authors
Reinforcement Pre-Training (RPT) improves language model accuracy through reinforcement learning and offers a scalable method for leveraging text data for general-purpose RL.
Trends · 研究趋势
数据来自 Hugging Face 论文的 AI 提取关键词,按月统计研究方向的增长与热度。
NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction
NCP Team, Jiaqi Cao, Chiyu Chen +25 authors
NCP-ArchPreview is a large latent-space language model that jointly trains next-token and next-concept prediction to improve pretraining efficiency and downstream performance.
LongCat-Next: Lexicalizing Modalities as Discrete Tokens
Meituan LongCat Team, Bin Xiao, Chao Wang +86 authors
Discrete Native Autoregressive framework enables unified multimodal processing by representing diverse modalities in a shared discrete space through a novel visual transformer architecture.
NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scale
NextStep Team, Chunrui Han, Guopeng Li +47 authors
NextStep-1, a 14B autoregressive model with a 157M flow matching head, achieves state-of-the-art performance in text-to-image generation and image editing by processing discrete text tokens and continuous image tokens.
Unlocking Lossless Speedups in LLMs via Discrete Diffusion
Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham +14 authors
Diffusion-augmented autoregressive language models use parallel token sampling via distilled diffusion weights and a specialized sampler to accelerate inference without quality loss or draft models.
Emu3.5: Native Multimodal Models are World Learners
Yufeng Cui, Honghao Chen, Haoge Deng +20 authors
Emu3.5, a large-scale multimodal world model, predicts next states in vision and language, enhanced with reinforcement learning and Discrete Diffusion Adaptation for efficient inference, achieving strong performance in various multimodal tasks.
Beyond Language Modeling: An Exploration of Multimodal Pretraining
Shengbang Tong, David Fan, John Nguyen +18 authors
Controlled multimodal pretraining experiments reveal key insights about unified visual representations, data complementarity, world modeling emergence, and efficient scaling through mixture-of-experts architectures.
Emu3: Next-Token Prediction is All You Need
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo +22 authors
Emu3, a transformer-based multimodal model trained exclusively with next-token prediction, outperforms existing diffusion and compositional models in generation and perception tasks.
Rho-1: Not All Tokens Are What You Need
Zhenghao Lin, Zhibin Gou, Yeyun Gong +8 authors
Rho-1, a novel language model using Selective Language Modeling, improves efficiency and performance by selectively training on useful tokens rather than all tokens in the corpus.
Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation
Peize Sun, Yi Jiang, Shoufa Chen +4 authors
LlamaGen applies the next-token prediction paradigm from large language models to image generation, achieving state-of-the-art performance with various model sizes and conditions.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Machel Reid, Nikolay Savinov, Denis Teplyashin +668 authors
Gemini 1.5 Pro, a multimodal mixture-of-experts model, achieves near-perfect recall and state-of-the-art performance in long-context tasks, including QA and ASR, surpassing previous models in token prediction and translation capabilities.
UniAudio: An Audio Foundation Model Toward Universal Audio Generation
Dongchao Yang, Jinchuan Tian, Xu Tan +9 authors
UniAudio, a unified language model system, generates various audio types effectively using tokenization, sequence concatenation, and next-token prediction, with a multi-scale Transformer model handling extended sequences.
上升最快
近 6 个月1
35 篇论文
2
llmNEW
34 篇论文
3
mllmsNEW
29 篇论文
4
26 篇论文
5
ditNEW
12 篇论文
6
quantizationNEW
12 篇论文
7
12 篇论文
8
11 篇论文
9
distillationNEW
11 篇论文
10
zero-shotNEW
11 篇论文
11
11 篇论文
12
10 篇论文
13
10 篇论文
14
10 篇论文
15
10 篇论文
16
26 篇论文
17
diffusion models+200%
74 篇论文
18
19
rlvr+200%
13 篇论文
20
autonomous agents+200%
12 篇论文
最热方向
按总量1
319 篇论文
2
199 篇论文
3
167 篇论文
4
86 篇论文
5
75 篇论文
6
74 篇论文
8
9
54 篇论文
10
49 篇论文
11
39 篇论文
12
38 篇论文
13
37 篇论文
14
35 篇论文
15
35 篇论文
16
34 篇论文
17
33 篇论文
18
29 篇论文
19
29 篇论文
20
29 篇论文
21
28 篇论文
22
23
27 篇论文
24
27 篇论文
25
26 篇论文
26
26 篇论文
27
26 篇论文
28
25 篇论文
29
25 篇论文
30
24 篇论文
31
23 篇论文
32
23 篇论文
33
23 篇论文
34
22 篇论文
35
22 篇论文
36
20 篇论文
37
20 篇论文
38
20 篇论文
39
20 篇论文
40
20 篇论文
41
19 篇论文
42
19 篇论文
43
19 篇论文
44
46
18 篇论文
47
48
18 篇论文
49
17 篇论文
50
51
17 篇论文
53
16 篇论文
54
16 篇论文
55
16 篇论文
56
16 篇论文
57
16 篇论文
58
16 篇论文
59
16 篇论文
60
15 篇论文