For patients recovering from stroke, the leading cause of adult disability in the developed world, the rehabilitation process is monotonous and solitary.

Researchers at Waipapa Taumata Rau, University of Auckland are working to change that with an augmented reality game that strengthens the body, stimulates the brain and encourages social connection.

Associate Professor Danielle Lottridge, a human-computer interaction expert with the School of Computer Science, along with colleagues including neurorehabilitation experts Professor Cathy Stinear, Professor Winston Byblow and Research Fellow Harry Jordan, wanted to develop a rehab tool that would motivate patients to keep going.

With collaborators including Amber Taylor (Ngāti Whātua, Ngāpuhi, Te Rarawa, Tainui, Ngāti Mutunga, Te Ātiawa), CEO of ARA Journeys, an award-winning Māori-owned game development company, honours student Maya Gibson (Ngāti Apa) and hospital physiotherapist Jon Bagnall, the researchers are prototyping an augmented reality game for stroke rehab patients.

“There’s a massive problem in that people cut their rehab too early and never finish their programme,” says Taylor. “We’re really hoping the gameplay we’ve come up with encourages daily use and keeps up engagement levels so patients keep improving their movement and coordination.”

“The goal is for it to be self-explanatory,” says Lottridge. “With human-computer interaction, though, the simpler the interface, the harder it is to design.”

As part of the process, Lottridge and her colleagues interviewed Māori, Pacific, South Asian and Pākehā stroke patients in their homes. They asked about their experiences of healing from stroke, how they use technology and what they find motivating. From these sessions came the message that stroke survivors need the therapy and tech to fit into their everyday lives.

“For me as a researcher in computer science, I sometimes feel pushed to use the latest gadgets and technologies,” says Lottridge. “Co-design pushes me to look at what technology works best for people.”

Another clear message from participants was that they experienced health and healing as multidimensional, comprising physical and mental health as well as social connection and, for some, a spiritual dimension. This fit with the te ao Māori concept of te whare tapa whā – that there are four cornerstones to health: physical, spiritual, family and mental.

While the game is informed by te whare tapa whā, it’s designed to help anybody recovering from a stroke.

“The idea is to create a multiplayer environment so people can rehab with whānau,” says Taylor. “Whānau is important to everyone, so it really contributes to rehab if you can interact with your mokopuna even if they’re not in the same house as you. That connection has significant health and wellbeing benefits.”

“Once the patient has done the puzzle, they can add virtual stickers or overlays and send the image back to the friend who sent it to them, so it comes full circle and encourages the friend to send the next picture.”

The app does remind users to practice so they can stay on track with their physiotherapy goals, but the researchers hope the game itself is motivating enough to keep patients coming back.

“Being creative, being social and having moments of respite and enjoyment all rolled into physical therapy will really help people heal,” says Lottridge.

The team is now seeking funding to do a feasibility study in a hospital ward, with patients receiving initial instruction on the game in hospital and given a kit to take home. With participant permission, their gameplay would be tracked and their upper limb function assessed at regular intervals. Through questionnaires and interviews, participants could provide feedback about the game as well as their mood and social support levels.

After that would come randomised control trials, the gold standard for establishing effectiveness.

Eventually, Lottridge says, she’d like the tool to be available to patients anywhere, as a low-cost intervention, with kits shipped around the world.

“This is very different from other interventions that are out there, so it can make a real difference.”