Dancing a Duet with Virtual Reality

Innovative art created by Deakin Motion.Lab - Centre for Creative Arts Research (DML-CCAR) premiered in May at Melbourne Knowledge Week and Geelong After Dark, putting on show Deakin researchers’ virtual reality interactive-movement mapping work.

 “Duet,” a conceptual art installation that advances research in artificial intelligence and participatory performance, was developed by DML-CCAR’s Dr John McCormick, PhD candidate Stephanie Andrews and DML-CCAR researcher Dr Jordan Beth Vincent and allowed participants to don a VR headset and physically interact with an artificially intelligent entity.

With the headset on, participants could see themselves moving in the visual environment before their movements transferred to the body of the AI entity.

“First, you see what it is like to be an avatar yourself, then the entity comes in and there is a transfer of movement to symbolise the flow of information from the participant to the digital agent,” Ms Andrews says.

“Both the human participant and the AI avatar become performers, spontaneously improvising with each other and exploring wordless communication.” 

As conceptual art, “Duet” incorporates VR, full body motion capture and the AI entity to “explore concepts of embodied knowledge, shared movement poetics and distortions of personal identity”.

As technology, it adds to knowledge about machine learning, neural networks and how virtual reality might expand the possibilities of human-computer interaction.

“The AI entity is able to accumulate movement knowledge in a way that mimics human learning,” says Dr McCormick, a leading researcher in the area of computer-based gesture recognition.

“’Duet’ was an installation looking at how we interact with digital systems. Systems are usually more calculated and pragmatic – there are buttons to push, there’s a cause and effect.

“But the research behind the installation is looking more at how we interact as a natural interface and how we might relate in the future to digital systems on a more human scale.”

In the past, digital avatars had to be pre-programmed with human movements. Using neural networks allows the systems to be more responsive and to respond in less predictable ways.

“Using this technology, we can train the avatar to understand similar movements to humans,” Dr McCormick says.

“We can examine how digital creatures learn to move and how they learn to respond physically to the environment without having to be preprogramed.”

He says the technology has the potential to be used in training robots to move more like humans and also in other areas such as computer generated movie characters.