In novel view synthesis of scenes from multiple input views, 3D Gaussian splatting emerges as a viable alternative to existing radiance field approaches, delivering great visual quality and real-time rendering. While successful in static scenes, the present advancement of 3D Gaussian representation, however, faces challenges in dynamic scenes in terms of memory consumption and the need for numerous observations per time step, due to the onus of storing 3D Gaussian parameters per time step. In this study, we present an efficient 3D Gaussian representation tailored for dynamic scenes in which we define positions and rotations as functions of time while leaving other time-invariant properties of the static 3D Gaussian unchanged. Notably, our representation reduces memory usage, which is consistent regardless of the input sequence length. Additionally, it mitigates the risk of overfitting observed frames by accounting for temporal changes. The optimization of our Gaussian representation based on image and flow reconstruction results in a powerful framework for dynamic scene view synthesis in both monocular and multi-view cases. We obtain the highest rendering speed of 118 frames per second (FPS) at a resolution of 1352×1014 with a single GPU, showing the practical usability and effectiveness of our proposed method in dynamic scene rendering scenarios.
在从多个输入视图合成场景的新视角中，3D高斯溅射作为现有辐射场方法的一种可行替代方案出现，提供了出色的视觉质量和实时渲染。虽然在静态场景中取得了成功，但目前3D高斯表示在动态场景中面临着挑战，主要是由于需要存储每个时间步的3D高斯参数，导致内存消耗大和每个时间步需要大量观测。在这项研究中，我们提出了一种针对动态场景量身定制的高效3D高斯表示，我们将位置和旋转定义为时间的函数，同时保持静态3D高斯的其他时间不变属性不变。值得注意的是，我们的表示减少了内存使用，这与输入序列的长度无关。此外，它通过考虑时间变化，减轻了过度拟合观测帧的风险。我们基于图像和流重建优化的高斯表示形成了一个强大的框架，用于在单目和多视图情况下进行动态场景视图合成。我们在单个GPU上以1352×1014的分辨率实现了最高118帧每秒（FPS）的渲染速度，显示了我们提出的方法在动态场景渲染场景中的实用性和有效性。