Adaptive training environments that can provide reliable insight into team communication offer great potential for team training and assessment. However, traditional techniques are especially resource-intensive without machine assistance.
There is a great deal of interest in developing AI-powered training technologies that can understand team dynamics and modify their training to foster improved collaboration among team members. However, previous AI architectures have struggled to accurately assess the content of what team members are sharing with each other when they communicate.
Now, researchers at North Carolina State University have developed a new artificial intelligence (AI) framework that significantly improves the ability of AI to analyze communication between team members. It is better than previous technologies at analyzing and categorizing dialogue between individuals, with the goal of improving team training technologies.
The new AI framework builds on a powerful deep learning model that was trained on a large, text-based language dataset. Called the Text-to-Text Transfer Transformer (T5), this model was then customized using data collected during squad-level training exercises conducted by the U.S. Army.
“We modified the T5 model to use contextual features of the team – such as the speaker’s role – to more accurately analyze team communication,” says Wookhee Min, co-author of a paper on the work. “That context can be important. For example, something a team leader says may need to be viewed differently than something another team member says.”
Researchers compared the new framework to two previous AI technologies to test its performance. They tested the ability of all three AI technologies to understand the dialogue within a squad of six soldiers during a training exercise.
The AI framework was tasked with classifying what sort of dialogue was taking place and following the flow of information within the squad. Classifying the dialogue refers to determining the purpose of what is being said. And they found that the new framework performed substantially better than the previous AI technologies.
“One of the things that was particularly promising was that we trained our framework using data from one training mission but tested the model’s performance using data from a different training mission,” Min says. “And the boost in performance over the previous AI models was notable – even though we were testing the model in a new set of circumstances.”
The team was able to achieve these results using a relatively small version of the T5 model. That’s important because it means that they can get analysis in fractions of a second without a supercomputer.
The next step will be to explore the extent to which the new framework can be applied to a variety of other training scenarios.
“We tested the new framework with training data that was transcribed from audio files into text by humans,” Min says. “Another next step will involve integrating the framework with an AI model that transcribes audio data into text so that we can assess the ability of this technology to analyze team communication data in real time. This will likely involve improving the framework’s ability to deal with noises and errors as the AI transcribes audio data.”
