SocialXR
An exergame using XR tech to simulate social companionship in fitness
2020- 2021
Solo Project
The use of video conferencing for group exercise sessions has become popular and offers emotional support but is limited to static activities in confined spaces. Dynamic activities like running or cycling pose challenges in representing participants' positions in 3D through video conferencing.
Additionally, constantly looking at the screen hinders the realistic and immersive experience compared to physically exercising together. I developed SocialXR to explore how might we enhance remote exercise experiences through simulated social companionship during exercise using Mixed Reality on a Microsoft HoloLens 2.
SocialXR
An exergame using XR tech to simulate social companionship in fitness
2020- 2021
Solo Project
The use of video conferencing for group exercise sessions has become popular and offers emotional support but is limited to static activities in confined spaces. Dynamic activities like running or cycling pose challenges in representing participants' positions in 3D through video conferencing.
Additionally, constantly looking at the screen hinders the realistic and immersive experience compared to physically exercising together. I developed SocialXR to explore how might we enhance remote exercise experiences through simulated social companionship during exercise using Mixed Reality on a Microsoft HoloLens 2.
Overview
Due to the Covid-19 pandemic, the project had to be scaled down. The prototype, originally planned for an outdoor exercise setting, was adapted for indoor use with a stationary bicycle.
To simulate an outdoor experience, a video was projected onto a screen, showing a first-person cycling perspective.
In this prototype, the user cycles indoors while a virtual avatar runs alongside, and the goal is to maintain a distance or chase the avatar. A holographic modal shows workout statistics like speed and distance from the avatar.
Due to the Covid-19 pandemic, the project had to be scaled down. The prototype, originally planned for an outdoor exercise setting, was adapted for indoor use with a stationary bicycle.
To simulate an outdoor experience, a video was projected onto a screen, showing a first-person cycling perspective.
In this prototype, the user cycles indoors while a virtual avatar runs alongside, and the goal is to maintain a distance or chase the avatar. A holographic modal shows workout statistics like speed and distance from the avatar.
Design Considerations
Using hand-tracking abilities on the Microsoft Mixed Reality Tool Kit (MRKT), I designed a modal showing the user's workout statistics to enable users to easily trigger and adjust the position and size of the modal, enabling users to have this in a convenient location in their field of view during their workout.
Using hand-tracking abilities on the Microsoft Mixed Reality Tool Kit (MRKT), I designed a modal showing the user's workout statistics to enable users to easily trigger and adjust the position and size of the modal, enabling users to have this in a convenient location in their field of view during their workout.
For the use case of SocialXR, where users frequently move their heads during gameplay while cycling, keeping the holograms world-locked ensures that the avatar friend remains steady and in a fixed plane when viewed by the user, improving usability in remembering its position within gameplay.
See the pictures below showing how the avatar remains in the same position regardless of the player's movement.
For the use case of SocialXR, where users frequently move their heads during gameplay while cycling, keeping the holograms world-locked ensures that the avatar friend remains steady and in a fixed plane when viewed by the user, improving usability in remembering its position within gameplay.
See the pictures below showing how the avatar remains in the same position regardless of the player's movement.
Furthermore, by implementing a spatial coordinate system, the application can accurately position the avatar in relation to the user, effectively depicting the distance between them and their exercise companion. The HoloLens 2 headset serves as a fixed reference point with coordinates set at startup.
When a user overtakes an avatar, they can see it behind them as seen below, due to the system placing it based on their headset position at startup.
Furthermore, by implementing a spatial coordinate system, the application can accurately position the avatar in relation to the user, effectively depicting the distance between them and their exercise companion. The HoloLens 2 headset serves as a fixed reference point with coordinates set at startup.
When a user overtakes an avatar, they can see it behind them as seen below, due to the system placing it based on their headset position at startup.
Implementation
Overall, prototype consists of three main parts, the SocialXR Unity application, Raspberry Pi 4 (RPi 4), and the reed switch on the stationary bicycle.
The user wears the Microsoft HoloLens 2 device to view the avatar and the game statistics. Holograms will appear based on specific hand gestures detected by the hand-tracking capabilities of the HoloLens 2.
To calculate cycling speed, a Python script running on RPi 4 uses a reed switch sensor attached to the stationary bicycle and a magnet attached to the pedal that registers a HIGH signal each time the magnet passes the reed switch. This measures the time taken for one revolution and sends it to a TCP server hosted on RPi.
The cycling speed data will then be pushed to the HoloLens client and then used to further calculate the distance that the user is away from the avatar.
Overall, prototype consists of three main parts, the SocialXR Unity application, Raspberry Pi 4 (RPi 4), and the reed switch on the stationary bicycle.
The user wears the Microsoft HoloLens 2 device to view the avatar and the game statistics. Holograms will appear based on specific hand gestures detected by the hand-tracking capabilities of the HoloLens 2.
To calculate cycling speed, a Python script running on RPi 4 uses a reed switch sensor attached to the stationary bicycle and a magnet attached to the pedal that registers a HIGH signal each time the magnet passes the reed switch. This measures the time taken for one revolution and sends it to a TCP server hosted on RPi.
The cycling speed data will then be pushed to the HoloLens client and then used to further calculate the distance that the user is away from the avatar.
User Trial
I conducted a user trial comparing a 10-minute cycling session without the HoloLens as a control, and a 10-minute cycling session with the HoloLens using SocialXR. Heart rate was monitored during both cycling sessions.
After the control cycling session, participants completed a post-exercise survey. Following that, they cycled with the HoloLens, optionally triggering a modal during gameplay. Another post-exercise survey was conducted to assess engagement and perceived exertion levels.
I conducted a user trial comparing a 10-minute cycling session without the HoloLens as a control, and a 10-minute cycling session with the HoloLens using SocialXR. Heart rate was monitored during both cycling sessions.
After the control cycling session, participants completed a post-exercise survey. Following that, they cycled with the HoloLens, optionally triggering a modal during gameplay. Another post-exercise survey was conducted to assess engagement and perceived exertion levels.
Results
It was observed that all participants showed a significant increase in mean heart rate by 26% while cycling with the SocialXR.
Based on the survey data collected, all participants reported feeling more tired when exercising with SocialXR. There was an average increase of two levels in tiredness across all responses from the participants.
Participants were also unanimous in reporting that they perceived themselves pushing harder while cycling with SocialXR compared to cycling.
It was observed that all participants showed a significant increase in mean heart rate by 26% while cycling with the SocialXR.
Based on the survey data collected, all participants reported feeling more tired when exercising with SocialXR. There was an average increase of two levels in tiredness across all responses from the participants.
Participants were also unanimous in reporting that they perceived themselves pushing harder while cycling with SocialXR compared to cycling.
Future Works and Conclusion