semantic_adversarial_examples

Semantic Adversarial Examples

Images that are arbitrarily perturbed to fool the model, but in such a way that the modified image semantically represents the same object as the original image.

— Semantic Adversarial Examples, Hosseini et al., CVPRW 2018.

These attacks are discussed in the following papers:

1. Semantic Adversarial Examples (CVPR Workshops 2018)
2. Towards Compositional Adversarial Robustness: Generalizing Adversarial Training to Composite Semantic Perturbations (CVPR 2023)

This repository implements only single semantic attacks discussed in [2]. The attacks are constructed using Projected Gradient Descent (PGD) on the following components:

• Hue
• Saturation
• Rotation
• Brightness
• Contrast

See section 3.2. of [2] for the math behind these attacks.

Dependencies

• Python3
• PyTorch
• Kornia (A differentiable computer vision library for PyTorch)
• OpenCV
• NumPy
• tqdm

Contents

attacks.py: contains both gradient-based search and random search of the five semantic perturbation parameters.
main.py: computes the robust accuracy of a model against single attacks. Need to manually change the model and the attack.
save_examples.py: saves some adversarial images to disk, along with their original images.
examples: contains some adversarial examples.
models: VGG16 and ResNet models.
weights: VGG16 and ResNet50 weights, trained on CIFAR-10.
visualizations: codes to visualize the attacks.

Experiments

Robust accuracy of single semantic attacks on ResNet50 trained on CIFAR-10.

	Clean	Hue	Saturation	Rotation	Brightness	Contrast
Mine	92.72	81.65	92.37	88.49	90.04	91.40
Paper	95.2	81.8 ± 0.0	94.0 ± 0.0	88.1 ± 0.1	92.1 ± 0.1	93.7 ± 0.1

Examples

Hue

Clean
Perturbed
Pred before	automobile	ship	horse	dog	bird
Pred after	dog	frog	truck	deer	dog
Hue	1.836	-0.193	-1.913	3.142	2.274

 

Saturation

Clean
Perturbed
Pred before	truck	cat	cat	horse	bird
Pred after	ship	dog	automobile	bird	dog
Saturation	0.700	1.156	0.700	1.159	0.874

 

Rotation

Clean
Perturbed
Pred before	ship	automobile	ship	horse	bird
Pred after	automobile	cat	frog	cat	cat
Rotation	-6.318	9.400	0.383	-9.586	10.000

 

Brightness

Clean
Perturbed
Pred before	ship	horse	cat	bird	airplane
Pred after	automobile	dog	frog	frog	bird
Brightness	-0.200	-0.150	0.125	-0.134	-0.023

 

Contrast

Clean
Perturbed
Pred before	ship	horse	bird	cat	frog
Pred after	automobile	cat	cat	frog	cat
Contrast	0.773	0.836	1.109	0.700	0.711

 

Citations

@inproceedings{hosseini2018semantic,
  title={Semantic adversarial examples},
  author={Hosseini, Hossein and Poovendran, Radha},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops},
  pages={1614--1619},
  year={2018}
}

@article{tsai2022towards,
  title={Towards compositional adversarial robustness: Generalizing adversarial training to composite semantic perturbations},
  author={Tsai, Yun-Yun and Hsiung, Lei and Chen, Pin-Yu and Ho, Tsung-Yi},
  journal={arXiv preprint arXiv:2202.04235},
  year={2022}
}