README.md

GuardSplat: Efficient and Robust Watermarking for 3D Gaussian Splatting

Zixuan Chen, Guangcong Wang, Jiahao Zhu, Jian-Huang Lai, Xiaohua Xie.

TL;DR: GuardSplat is an efficient watermarking framework to protect the copyright of 3DGS assets. It presents superior performance to state-of-the-art watermarking approaches in capacity, invisibility, robustness, security, and training efficiency.

---

Application scenarios of GuardSplat. To protect the copyright of 3D Gaussian Splatting (3DGS) assets, (a) the owners (Alice) can use our GuardSplat to embed the secret message (blue key) into these models. (b) If malicious users (Bob) render views for unauthorized uses, (c) Alice can use the private message decoder to extract messages (purple key) for copyright identification.

Framework

Overview of GuardSplat. (a) Given a binary message $M$, we first transform it into CLIP tokens $T$ using the proposed message tokenization. We then employ CLIP's textual encoder $\mathcal{E _T}$ to map $T$ to the textual feature $F _\mathcal{T}$. Finally, we feed $F _\mathcal{T}$ into message decoder $\mathcal{D _M}$ to extract the message $\hat{M}$ for optimization. (b) For each 3D Gaussian, we freeze all the attributes and build a learnable spherical harmonic (SH) offset $\boldsymbol{h}^o _i$ as the watermarked SH feature, which can be added to the original SH features as $\boldsymbol{h} _i + \boldsymbol{h}^o _i$ to render the watermarked views. (c) We first feed the 2D rendered views to CLIP's visual encoder $\mathcal{E _V}$ to acquire the visual feature $F _{\mathcal{V}}$ and then employ the pre-trained message decoder to extract the message $\hat{M}$. A differentiable distortion layer is used to simulate various visual distortions during optimization. $\mathcal{D _M}$ and $\boldsymbol{h}^o _i$ are optimized by the corresponding losses, respectively.

An Evaluation Example of Our GuardSplat

We provide a demo code "eval_watermark.ipynb" for evaluation. It is recommended to check the evaluation results recorded in the Jupyter Notebook and run the code by the following steps:

1. Install the 3D gaussian splatting (see https://github.com/graphdeco-inria/gaussian-splatting).

2. Install the CLIP (see https://github.com/openai/CLIP).

3. Download the "eval_examples" folder from Google Driver / Baidu Cloud as:

4. Copy the file "eval_watermark.ipynb" and the "eval_examples" folder into the "gaussian-splatting" folder as:

gaussian-splatting/
    ├── eval_examples/
    │   ├── CLIP_visual+msg_decoder-32.pt (CLIP Visual Encoder + Our Message Decoder)
    │   ├── original-lego.ply (original 3DGS asset)
    │   ├── watermarked-lego.ply (watermarked file)
    │   ├── message-lego.txt (32-bit secret message)
    │   └── cameras-lego.json (3DGS camera pose file)
    ├── eval_watermark.ipynb
    └── ...

4. Run the code in "eval_watermark.ipynb".

Citation

@article{Chen_2024_GuardSplat,
    author  = {Chen, Zixuan and Wang, Guangcong and Zhu, Jiahao and Lai, Jian-Huang and Xie, Xiaohua},
    title   = {GuardSplat: Efficient and Robust Watermarking for 3D Gaussian Splatting},
    year    = {2024},
    journal = {arXiv preprint},
}

Acknowledgement

We build our project based on gaussian-splatting and CLIP. We sincerely thank them for their wonderful work and code release.