The explosive adoption of artificial intelligence (AI) and machine learning (ML) across defense, communications, health care, intelligence, finance, transportation, and other markets makes adversarial action a significant and growing threat. Successful attacks can have disastrous consequences from vehicular crashes, cyber breaches, and stolen identities, to missed diagnoses and failures in financial risk management, and ISR (intelligence, surveillance, and reconnaissance).
Hacking AI, our first article in this two-part series, focused on the threats, vulnerabilities, and impacts of adversarial AI/ML. This article discusses securing AI, with information on ML assurance and snapshots of some of our innovative work in adversarial AI/ML. Contact us at [email protected] to learn more.
Both the vulnerabilities of AI/ML systems and the severe consequences of successful attacks are increasingly well recognized. The National Institute of Standards and Technologies (NIST) has issued a draft report (NISTIR 8269 (draft) on Adversarial ML) aimed at informing future standards and best practices for ML security. The Department of Health and Human Services’ Trustworthy AI playbook focuses on risk mitigation to ensure AI systems are ethical, effective, and secure and includes requirements to “develop defenses against adversarial attacks and scan for vulnerabilities.”
Peraton Labs is a leader in adversarial AI including innovative projects aimed at:
Below are a few examples of our work.
Disruptive Techniques for Machine Vision: We have developed new techniques for disrupting machine vision systems so that they cannot identify objects. The techniques can disrupt diverse neural networks and architectures and are effective across a wide variety of images at different distances, orientations, viewing conditions, degrees of occlusion, and resolution.
AI Robustness for Multi-domain Operations: We have developed and analyzed a new method to provide robustness against adversarial attacks to ML systems in multi-sensor, multi-modal image and audio domains. Our work includes techniques to discover factors of vulnerability and robustness, analyze the defensibility of algorithms, measure the attack surface, and calculate the performance penalty of defenses.
Finding Trojans in AI: In current research, we are developing software to automatically inspect a neural-network classifier and predict if it has a “Trojan.” A Trojan means the ML was successfully hacked during training, such that it can be triggered to give incorrect results; e.g., misidentifying a sign or verbal command. Our solution is a multi-pronged detection mechanism to significantly reduce Trojan risk.
Adversarial ML in the Cyber Domain: PDF documents can be infected with malware, offering attackers a convenient cyberattack vector. In current work we have developed a countermeasure to circumvent a commercial ML-based anti-virus protection system. Our technique involves perturbing an infected PDF, so that it is not recognized as infected. The detection rate for infected PDFs dropped significantly after our perturbations.
Protecting ML from Privacy Leakages: Privacy leakage attacks on ML are a serious concern for the DoD, IC, and health sector given the sensitivity of training data. In published work, we developed and evaluated a prototype software solution that successfully thwarts standard membership inference attacks against trained ML models, without impacting the accuracy or utility of the protected ML model.
For more information on these projects and Peraton Labs’ capabilities in adversarial AI/ML contact us at [email protected].
Hacking AI, our first article in this two-part series, focused on the threats, vulnerabilities, and impacts of adversarial AI/ML. This article discusses securing AI, with information on ML assurance and snapshots of some of our innovative work in adversarial AI/ML. Contact us at [email protected] to learn more.
Both the vulnerabilities of AI/ML systems and the severe consequences of successful attacks are increasingly well recognized. The National Institute of Standards and Technologies (NIST) has issued a draft report (NISTIR 8269 (draft) on Adversarial ML) aimed at informing future standards and best practices for ML security. The Department of Health and Human Services’ Trustworthy AI playbook focuses on risk mitigation to ensure AI systems are ethical, effective, and secure and includes requirements to “develop defenses against adversarial attacks and scan for vulnerabilities.”
Peraton Labs is a leader in adversarial AI including innovative projects aimed at:
- Understanding the AI/ML attack surface
- Identifying vulnerabilities
- Characterizing attack vectors and kill chains
- Developing defense and attack mechanisms
- Testing, evaluation, and training on attack and counterattack mechanisms
- Research and development to stay ahead of evolving exploits
Below are a few examples of our work.
Disruptive Techniques for Machine Vision: We have developed new techniques for disrupting machine vision systems so that they cannot identify objects. The techniques can disrupt diverse neural networks and architectures and are effective across a wide variety of images at different distances, orientations, viewing conditions, degrees of occlusion, and resolution.
AI Robustness for Multi-domain Operations: We have developed and analyzed a new method to provide robustness against adversarial attacks to ML systems in multi-sensor, multi-modal image and audio domains. Our work includes techniques to discover factors of vulnerability and robustness, analyze the defensibility of algorithms, measure the attack surface, and calculate the performance penalty of defenses.
Finding Trojans in AI: In current research, we are developing software to automatically inspect a neural-network classifier and predict if it has a “Trojan.” A Trojan means the ML was successfully hacked during training, such that it can be triggered to give incorrect results; e.g., misidentifying a sign or verbal command. Our solution is a multi-pronged detection mechanism to significantly reduce Trojan risk.
Adversarial ML in the Cyber Domain: PDF documents can be infected with malware, offering attackers a convenient cyberattack vector. In current work we have developed a countermeasure to circumvent a commercial ML-based anti-virus protection system. Our technique involves perturbing an infected PDF, so that it is not recognized as infected. The detection rate for infected PDFs dropped significantly after our perturbations.
Protecting ML from Privacy Leakages: Privacy leakage attacks on ML are a serious concern for the DoD, IC, and health sector given the sensitivity of training data. In published work, we developed and evaluated a prototype software solution that successfully thwarts standard membership inference attacks against trained ML models, without impacting the accuracy or utility of the protected ML model.
For more information on these projects and Peraton Labs’ capabilities in adversarial AI/ML contact us at [email protected].