Cooperative Perception in Multi-agent Systems

This thesis presents work and simulations containing the use of Artificial Intelligence for Unmanned Aerial Vehicles in search and rescue and/or surveillance operations. The goal is to create a vision system that leverages Artificial Intelligence, mainly Deep Learning techniques to build a pipeline that enables fast and accurate classification of the environment of the robot. Deep Neural Networks are trained and tested on ’emergency situational data. Further, the power of this vision system is leveraged to extend the problem onto a multiagent system to handle fault tolerance. The multi-agent system is also made resilient to Byzantine malicious attacks to help improve the reliability of the system.

This thesis also shows the use of Artificial Intelligence for effective surveillance for defense related purposes. Tracking the GPS coordinates of a boat using only the video of the boat captured by a camera and the GPS coordinates of the camera itself is demonstrated. The solution was tested by the Department of Defense - Department of the Navy, Naval Information Warfare Center Pacific.