Using virtual reality software and then goggles, drawing this fish figure can help a patient and physical therapist alike.

Virtual reality for physical therapy

Article by Julie Stewart Photos courtesy of Roghayeh (Leila) Barmaki | Illustrations by Joy Smoker March 20, 2019

Computer science professor develops award-winning tool for improving upper body mobility

A University of Delaware professor was part of a team that won the Best Virtual Reality Prize at the Reality Virtually Hackathon, held at the Massachusetts Institute of Technology Media Lab from Jan. 17 to 21, 2019.

This hackathon, which asks experts to create a product from scratch within five days, is widely considered the most prestigious event of its kind. More than 400 attendees were selected from approximately 1,600 applicants, and participants represented more than 35 countries.

Prof. Roghayeh (Leila) Barmaki

The award-winning team, Move2Improve, included Roghayeh (Leila) Barmaki, an assistant professor of computer and information sciences and resident faculty member in UD’s Data Science Institute. In addition to winning the Best Virtual Reality Prize, Move2Improve was selected as one of four finalists to compete for a $30,000 grand prize and join a summer accelerator program. Four of Barmaki’s students also participated in the hackathon — Yan-Ming Chiou, Alina Christenbury, Matthew Bonham, and Zhang Guo.

Move2Improve is a physical therapy game designed to make treatment more fun and immersive for patients with upper limb injuries, such as post-stroke individuals. Patients wear a virtual reality headset and are asked to complete drawings with the hand affected by the injury. For example, a patient might be asked to draw a fish with a single line without changing his or her path. Once that’s complete, the drawing becomes a vivid fish, and happily starts moving around in the virtual world: a reward of giving life to an animal after patient accomplishes the task. As the patient progresses the drawing exercises, the game become more challenging, with more detailed scenes to be drawn. The physical therapist can also assess how the patient’s upper body moves as he or she slowly completes the drawings. These amusing exercises could help patients avoid boredom during the repetitive movements sometimes required in physical therapy.

Barmaki is particularly motivated by healthcare applications of virtual and augmented reality. Two years ago, while she was a postdoctoral associate at Johns Hopkins University studying virtual reality applications for education, she broke her clavicle. The injury required two surgeries and many hours of physical therapy and inspired her to do research on the ways immersive technologies such as virtual reality could benefit patients and healthcare providers.

The Move2Improve team won the Best Virtual Reality Prize at the Reality Virtually Hackathon.

Barmaki worked with collaborators from several universities and industry on Move2Improve. Her team included not only experts in virtual reality, such as Barmaki herself, but also an artist, a user experience designer, and more. “We all wanted to make something elegant, entertaining, and very applied,” she said.

One-line animal drawings: With Move2Improve, patients can draw these animals as part of their physical therapy.

Barmaki’s students also joined groups and created applications. Christenbury’s group created Escape the Witch's Grotto; Guo’s group created Logos Pathos Ethos, and Chiou’s group created FIT. These experiences were almost like internships, because each day was jam-packed with valuable information and skill-building in mixed reality — an important area for future computer scientists. Various workshop sessions with mentors also helped students learn at a fast pace.

“Students made friends, went beyond their comfort zone in programming and design, made applications, and had fun,” Barmaki said. “So it has been a very impressive and positive experience for the students.”

Christenbury, one of the UD students, said that while she has both been to and hosted hackathons, this was one of the biggest and most well-organized ones.

“I met a ton of people from all sorts of different disciplines and ways of life and look forward to working with them on projects in the future,” she said. “The organizers put a lot of effort into bringing in diverse people from all around the world, and it shows.”

There is a growing community of research around virtual and augmented reality at UD — and around the world.

“Virtual reality is a very emerging field right now,” said Barmaki. “There are a lot of startups that are currently working on virtual reality and augmented reality. It is the future of technology. It’s becoming mainstream in healthcare, pain management, education, communication, and more.”