Autoregressive Video Generation without Vector Quantization
Guardado en:
| Publicado en: | arXiv.org (Dec 18, 2024), p. n/a |
|---|---|
| Autor principal: | |
| Otros Autores: | , , , , , , , |
| Publicado: |
Cornell University Library, arXiv.org
|
| Materias: | |
| Acceso en línea: | Citation/Abstract Full text outside of ProQuest |
| Etiquetas: |
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
MARC
| LEADER | 00000nab a2200000uu 4500 | ||
|---|---|---|---|
| 001 | 3147264472 | ||
| 003 | UK-CbPIL | ||
| 022 | |a 2331-8422 | ||
| 035 | |a 3147264472 | ||
| 045 | 0 | |b d20241218 | |
| 100 | 1 | |a Deng, Haoge | |
| 245 | 1 | |a Autoregressive Video Generation without Vector Quantization | |
| 260 | |b Cornell University Library, arXiv.org |c Dec 18, 2024 | ||
| 513 | |a Working Paper | ||
| 520 | 3 | |a This paper presents a novel approach that enables autoregressive video generation with high efficiency. We propose to reformulate the video generation problem as a non-quantized autoregressive modeling of temporal frame-by-frame prediction and spatial set-by-set prediction. Unlike raster-scan prediction in prior autoregressive models or joint distribution modeling of fixed-length tokens in diffusion models, our approach maintains the causal property of GPT-style models for flexible in-context capabilities, while leveraging bidirectional modeling within individual frames for efficiency. With the proposed approach, we train a novel video autoregressive model without vector quantization, termed NOVA. Our results demonstrate that NOVA surpasses prior autoregressive video models in data efficiency, inference speed, visual fidelity, and video fluency, even with a much smaller model capacity, i.e., 0.6B parameters. NOVA also outperforms state-of-the-art image diffusion models in text-to-image generation tasks, with a significantly lower training cost. Additionally, NOVA generalizes well across extended video durations and enables diverse zero-shot applications in one unified model. Code and models are publicly available at https://github.com/baaivision/NOVA. | |
| 653 | |a Diffusion rate | ||
| 653 | |a Autoregressive models | ||
| 653 | |a Image processing | ||
| 653 | |a Modelling | ||
| 653 | |a Raster scanning | ||
| 653 | |a Efficiency | ||
| 700 | 1 | |a Pan, Ting | |
| 700 | 1 | |a Diao, Haiwen | |
| 700 | 1 | |a Luo, Zhengxiong | |
| 700 | 1 | |a Cui, Yufeng | |
| 700 | 1 | |a Lu, Huchuan | |
| 700 | 1 | |a Shan, Shiguang | |
| 700 | 1 | |a Qi, Yonggang | |
| 700 | 1 | |a Wang, Xinlong | |
| 773 | 0 | |t arXiv.org |g (Dec 18, 2024), p. n/a | |
| 786 | 0 | |d ProQuest |t Engineering Database | |
| 856 | 4 | 1 | |3 Citation/Abstract |u https://www.proquest.com/docview/3147264472/abstract/embedded/6A8EOT78XXH2IG52?source=fedsrch |
| 856 | 4 | 0 | |3 Full text outside of ProQuest |u http://arxiv.org/abs/2412.14169 |