Research into physically realizable adversarial attacks on computer vision systems, focusing on patch-based perturbations that can fool models in real-world scenarios.
A comprehensive study of adversarial patch attacks, from 2D static patches to 3D dynamic attacks on autonomous systems.
Unlike digital perturbations, patch attacks are physical stickers that can be placed in the real world to fool AI vision systems.
Imagine a small, specially designed sticker that, when viewed by a camera, causes an AI model to misclassify what's in the image. Unlike traditional adversarial attacks that require pixel-level modifications to digital images, patch attacks work in the physical world—think of them as "digital camouflage" that exploits vulnerabilities in how AI systems process visual information. These attacks are particularly concerning for autonomous vehicles, where a patch on a stop sign could make it appear as a speed limit sign to the car's computer vision system.
Systematic evaluation of patch attack effectiveness under geometric transformations using the Expectation over Transformation (EoT) framework.
Analyzed how rotation, scaling, and positioning affect attack success rates. Findings showed that scale has the most significant impact on performance, with training distribution support directly influencing robustness to out-of-distribution transformations.
Extending patch attacks to 3D space with adaptive active attacks (AAA) that optimize for observer pose and kinematic motion.
Developed AAA attacks that consider the full 3D trajectory of an observer, demonstrating improved success rates (up to 15% for specific classes) over static patches. Introduced risk-based metrics to evaluate attack success probability while accounting for detection likelihood.
Insights into patch attack limitations and design principles for more robust computer vision systems.
Identified fundamental cutoff limits in attack effectiveness based on out-of-plane rotation angles. Demonstrated that increasing training distribution support improves invariance to transformations like rotation and looming motion. These results inform both attack development and defense strategies for real-world AI safety.
Review the project papers and PDF previews that document our work on patch robustness, geometric transformation analysis, and adaptive attacks.