INTERVIEWS Vahid Behzadan, Director of Secured and Assured Intelligent Learning (SAIL) Lab – Interview Series
Vahid is an Assistant Professor of Computer Science and Data Science at the University of New Haven. He is also director of the Secure and Assured Intelligent Learning (SAIL) Lab
His research interests include safety and security of intelligent systems, psychological modeling of AI safety problems, security of complex adaptive systems, game theory, multi-agent systems, and cyber-security.
You have an extensive background in cybersecurity and keeping AI safe. Can you share your journey in how you became attracted to both fields?
My research trajectory has been fueled by two core interests of mine: finding out how things break, and learning about the mechanics of the human mind. ...
You recently led an effort to propose the modeling of AI safety problems as psychopathological disorders. Could you explain what this is?
This project addresses the rapidly growing complexity of AI agents and systems: it is already very difficult to diagnose, predict, and control unsafe behaviors of reinforcement learningagents in non-trivial settings by simply looking at their low-level configurations. In this work, we emphasize the need for higher-level abstractions in investigating such problems. Inspired by the scientific approaches to behavioral problems in humans, we propose psychopathology as a useful high-level abstraction for modeling and analyzing emergent deleterious behaviors in AI and AGI. ...
One of the areas you closely follow is counterterrorism. Do you have concerns with terrorists taking over AI or AGI systems?
...One of the main problems with AI-enabled weaponry is in the difficulty of controlling the underlying technology: AI is at the forefront of open-source research, and anyone with access to the internet and consumer-grade hardware can develop harmful AI systems. I suspect that the emergence of autonomous weapons is inevitable, and believe that there will soon be a need for new technological solutions to counter such weapons. This can result in a cat-and-mouse cycle that fuels the evolution of AI-enabled weapons, which may give rise to serious existential risks in the long-term.
What can we do to keep AI systems safe from these adversarial agents?
...training machine learning agents in adversarial settings can improve their resilience and robustness against evasion and policy manipulation attacks (e.g., see my paper titled “Whatever Does Not Kill Deep Reinforcement Learning, Makes it Stronger“). Another solution is to directly account for the risk of adversarial attacks in the architecture of the agent (e.g., Bayesian approaches to risk modeling). There is however a major gap in this area, and it’s the need for universal metrics and methodologies for evaluating the robustness of AI agents against adversarial attacks. Current solutions are mostly ad hoc, and fail to provide general measures of resilience against all types of attacks.
See the full long interview here: https://www.unite.ai/vahid-behzadan-director-of-secured-and-assured-intelligent-learning-sail-lab-interview-series/