GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for
  Real-time Human Novel View Synthesis

Shunyuan Zheng; Boyao Zhou; Ruizhi Shao; Boning Liu; Shengping Zhang; Liqiang Nie; Yebin Liu

返回文献探索

Paper · arXiv 2312.02155

GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis

Shunyuan Zheng, Boyao Zhou, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie, Yebin Liu

14 upvotesDecember 4, 2023arXiv 预印本

AI 摘要

A GPS-Gaussian approach synthesizes 2K-resolution views in real-time using Gaussian parameter maps and depth estimation, outperforming existing methods.

Gaussian parameter mapsGaussian Splattingneural implicit renderingdepth estimation module2D parameter maps3D spacedifferentiable frameworknovel view synthesis

Abstract

We present a new approach, termed GPS-Gaussian, for synthesizing novel views of a character in a real-time manner. The proposed method enables 2K-resolution rendering under a sparse-view camera setting. Unlike the original Gaussian Splatting or neural implicit rendering methods that necessitate per-subject optimizations, we introduce Gaussian parameter maps defined on the source views and regress directly Gaussian Splatting properties for instant novel view synthesis without any fine-tuning or optimization. To this end, we train our Gaussian parameter regression module on a large amount of human scan data, jointly with a depth estimation module to lift 2D parameter maps to 3D space. The proposed framework is fully differentiable and experiments on several datasets demonstrate that our method outperforms state-of-the-art methods while achieving an exceeding rendering speed.