Recently, 3D Gaussian Splatting (3DGS) has garnered attention for its high fidelity and real-time rendering. However, adapting 3DGS to different camera models, particularly fisheye lenses, poses challenges due to the unique 3D to 2D projection calculation. Additionally, there are inefficiencies in the tile-based splatting, especially for the extreme curvature and wide field of view of fisheye lenses, which are crucial for its broader real-life applications. To tackle these challenges, we introduce Fisheye-GS.This innovative method recalculates the projection transformation and its gradients for fisheye cameras. Our approach can be seamlessly integrated as a module into other efficient 3D rendering methods, emphasizing its extensibility, lightweight nature, and modular design. Since we only modified the projection component, it can also be easily adapted for use with different camera models. Compared to methods that train after undistortion, our approach demonstrates a clear improvement in visual quality.
近期，3D高斯分裂（3DGS）因其高保真度和实时渲染性能备受关注。然而，将3DGS应用于不同的相机模型，尤其是鱼眼镜头，面临挑战，主要是由于独特的3D到2D投影计算。此外，基于平铺的高斯分裂在鱼眼镜头的极端曲率和广视角条件下效率不高，这对于其更广泛的实际应用至关重要。为了解决这些问题，我们提出了Fisheye-GS⋆。该创新方法重新计算了鱼眼相机的投影变换及其梯度。我们的方法可以作为一个模块无缝集成到其他高效的3D渲染方法中，强调其可扩展性、轻量化和模块化设计。由于我们只修改了投影部分，该方法也可以轻松适配不同的相机模型。与通过图像去畸变后进行训练的方法相比，我们的方法在视觉质量上有明显提升。