TeamCollab: A Framework for Collaborative Perception-Cognition-Communication-Action

Teams of embodied AI-enabled agents are critical for applications in extreme and highly dynamic environments. Developing robust controllers for such agents requires a deep understanding of the challenges encountered when attempting to coordinate and synchronize their individual perception-cognition-communication-action (PCCA) loops for team-wide mission objectives. We introduce a framework to explore the coordination of the PCCA loops across multiple agents in a new simulated physical environment designed to explore collaboration in each PCCA stage. This environment tasks teams of agents with the correct disposal of dangerous objects in an area and forces careful coordination of sensing, communication, movement, and manipulation actions by providing spatially-bounded communication, incorporating situations that require concerted effort by groups of agents, and introducing uncertainty into agents' sensing capabilities. We provide a set of heuristic controllers, an offline oracle model, and an initial exploration of a Reward Machine-based controller that learns its policies from training. Together these approaches serve to provide insights into the complexity of the multi-agent PCCA loop coordination problem. The multiagent PCCA simulation environment, which supports AI and human-controlled agents, and the code for various agent controllers are available at https://github.com/nesl/AI-Collab.