Research Environments for Human-Machine Teaming

Humans and intelligent machines such as robots are increasingly working together as teammates in various domains. For example, the fire department of Rotterdam is already using explore and extinguish robots for situations too dangerous for firefighters. The ultimate goal of these teams is to harness the combination of strengths of both humans and machines, to accomplish what neither can do alone.

Such an integration of machines that augment rather than replace humans can improve efficiency and safety in various domains. Unfortunately, there is still a long way to go before machines will be effective teammates for humans. For example, intelligent machines are often black boxes whose inner workings and behavior are hard to understand and appropriately trust. To further identify and optimize the critical success factors of human-machine teams, we need realistic and reusable research environments for human-machine teaming.

Since studying human-machine teams in practice can be both time and cost expensive, it is common to conduct experiments in simulated and/or controlled environments. Unfortunately, it has proven challenging to design realistic and reusable human-machine collaboration studies in these environments. For example, creating scenarios and collaborations that align with the research questions and appropriately reflect and allow measures of teamwork is extremely difficult. The field of human-machine teaming would greatly benefit from the identification of such realistic and reusable scenarios, collaborations, and measures. Therefore, the aim of this workshop is to discuss and address the following topics and challenges:

• What are realistic human-machine teaming scenarios to study?
• Which measures are reliable and robust when evaluating human-machine teamwork?
• What are realistic forms of human-machine collaboration to study (i.e., role and task distributions)?
• How can we design effective baseline conditions for human-machine teaming studies?
• When are certain methods (2/3D, A/VR) more suitable for studying human-machine collaboration?

During the workshop, we want to bring together people from various disciplines who face the same challenges, to discuss and identify reusable solutions and methods that can help overcome these challenges. This way, we want to build a close network and stimulate knowledge gathering and sharing on the further development of appropriate tools for human-machine teaming research. The workshop should lead to reusable and realistic scenarios, collaborations, and measures for human-machine teaming research, as well as guidelines and templates for describing them. Eventually, we want to combine these outcomes into a paper. Additionally, we want to work towards a Wiki that allows and stimulates the reuse of realistic research environments for human-machine teaming.