Show us some cool CUDA tricks. Shorten the training time.

[MrNeRF]

As a potential user of this project, I am interested in training 3D Gaussian splatting models on my own dataset. However, I am using a moderately capable graphics card, and I want to avoid long training times that span several hours.

To improve the user experience for people with similar hardware, I kindly request the following:

1. Code Profiling: Please conduct a detailed profiling of the training process to identify performance bottlenecks. This will help in understanding which parts of the code are consuming the most time and resources.
2. Optimization: After identifying the bottlenecks, please optimize the code to reduce the computational load. This might involve algorithmic improvements, efficient memory usage, or leveraging specific GPU capabilities (without requiring top-end GPUs).
3. Documentation Update: Once the optimizations are complete, it would be beneficial to update the README or a separate performance guide that explains the optimizations that were made and how users with different hardware configurations can expect the code to perform.

My goal as a user is to have an efficient and swift training process, even when not using a top-tier GPU. Implementing these optimizations will significantly enhance the project's accessibility and usability for a wider audience.

[MrNeRF]

This might be relevant:
https://github.com/InternLandMark/FlashGS
https://arxiv.org/abs/2408.07967

https://github.com/humansensinglab/taming-3dgs