In this section, the results of each step in the method, including (1) Analyze and Compare Relevant Literature to identify game elements. (2) Decomposition of clinical rehabilitation exercises and mapping into the game elements. (3) Impact of VR on motor rehabilitation serious games. (4) A reward cycle for long-term motivation, are introduced in detail. The framework is also introduced at the end of this section.

Analyze and compare relevant literature to identify game elements

The designed game should adhere to the principles of rehabilitation gaming, which must be considered in the proposed rehabilitation serious game design framework. Therefore, relevant literatures are analyzed and compared to identify the elements that games should encompass.

The game elements identified in relevant literatures are listed in Table 1. Furthermore, these elements could be further generalized into following elements: Feedback, Difficulty, Reward, Goal, Choice, Immersion, Game length, therapeutic principles, Guide and Uncertainty.

Table 1 Elements in relevant literature

Feedback is a common element across all related literature (e.g. Scoring in [25]). It includes the KP and KR feedback, which provide patients with information about their motion, enabling them to become aware of their state. Thus, the Awareness in [27, 28] could also be categorized under this game element. Additionally, multimodal feedback that response to patient’s motion is also a kind of feedback, presenting the Interaction mentioned in [27, 28, 51].

Difficulty is also a prominent element across all related literature (e.g. Beating the game in [53]). According to the flow theory [59], patients could achieve a state of deep engagement, leading to efficient learning, which requires a balancing personal skill and challenge. Since patients have varying degree of motion function, this balance point differs among individuals. Therefore, Difficulty is crucial in rehabilitation serious game to accomplish personalized rehabilitation. As such, Difficulty, Challenge, Adaptivity and Customizable all refer to the same concept. The method by which Difficulty provides suitable difficult to different individuals involves adjusting parameters such as Speed, Time, Life, Objects/Obstacles behavior and sensitivity.

Reward is referenced in several sources [25, 27, 28, 52], with examples including Achievements and badges in [25]. It serves as an incentive structure, giving patient positive feedback in time to encourage continued engagement with training. Therefore, Motivation and Effort mentioned in [27, 28] can also be categorized under this game element.

Goal is mentioned in various sources [25, 27, 28, 52], such as Purpose in [27, 28]. The Goal of the game represents what the designed game intends for patients to strive towards. It could make task (rehabilitation training) execution efficient [52]. Additionally, the achievement of the game goal can offer patients achievement-based satisfaction.

Choice, as mentioned in [25, 52]. Choice offers patients a sense of control, strengthening their connection to the virtual environment and facilitating active participation in the game.

Immersion is discussed in [25, 60], with references to First-Person View in [25]. The immersion is a kind of feeling, which could be maximally enhanced through VR.

There are additional elements proposed by individual research: Game length (Number of levels), the total duration time of the game. Therapeutic principles, the of rehabilitation approached employed in game. Plausibility, the patient could have a rest in time. Guide (Help), tell patient what and how to achieve the task. Uncertainty (Operant Conditioning), to keep a certain of change in game, which is proved to improve to the motor learning [13].

Decomposition of clinical rehabilitation exercises and mapping into the game elements

As depicted in Fig. 2a, the clinical rehabilitation exercises (Rehabilitation Training) are represented as a black box (left). Its inputs include Patient with Low Motor Function, Patient’s Energy, and Rehabilitation Stage Goal, while the output is Patient with Higher Motor Function.

Fig. 2

Decomposition diagram of clinical rehabilitation exercises. The solid blue line denotes material and energy, the dashed blue line represents information, and the orange line signifies the mapping

The decomposition result of clinical rehabilitation exercises is depicted in Fig. 2b. For clarity, it is presented in a functional decomposition structure diagram. Before commencing the rehabilitation training process, the therapist conducts a clinical evaluation during the initial visit to assess the patient's stage and motor ability. Thus, during the rehabilitation training, the therapist initiates by proposing rehabilitation training actions and plans based on the patient's rehabilitation stage goals. The requirements for rehabilitation training movements are then personally adjusted, accounting for the patient's motor function level. Following this, the patient actively engages in limb movement. If, during the movement, the patient perceives that their motion has not yet met the specified requirements, they persist in driving the limb. In cases where further movement toward the required position proves unattainable, therapists may reduce the training requirements. Upon the completion of a training session, the therapist records the patient's training performance and adjusts the training requirements for subsequent sessions. After the conclusion of the training course, a reassessment is conducted on the patient’s condition, encompassing motor function and ability information.

VR rehabilitation serious game is expected to achieve comparable effectiveness to clinical rehabilitation. Hence, as depicted in Fig. 2a, it should have similar structure to the clinical rehabilitation exercises. Although we cannot achieve a complete functional decomposition result of the VR rehabilitation serious game all at once, it can be partially confirmed by mapping the function blocks of clinical rehabilitation exercises to the game elements identified previously, as illustrated in Fig. 2b.

VR factors in game design

In addition to the basic functions of clinical rehabilitation training, VR rehabilitation serious games are more important in enhancing patients’ active participation. Therefore, VR rehabilitation serious games utilize head-mounted displays (HMDs) to deliver rehabilitation training. HMDs present scenes in a 3D surround format around the patient, significantly enhancing the aesthetic effect and increasing the patient's immersion. The aesthetic effect primarily involves designing content that patients can directly perceive through their senses in games, including art design and stories, which directly affect the patient’s gaming experience [61]. Therefore, compared with non-VR rehabilitation games, VR rehabilitation games are more likely to provide an enjoyable experience. Moreover, the increased immersion allows patients to better immerse themselves in the game, enabling them to more easily to get emotional changes, such as a sense of purpose, accomplishment, and control, among other emotions. These feelings encourage active participation in rehabilitation serious game.

However, it also introduces side effects [62]. In VR rehabilitation, patients may encounter visual fatigue and motion sickness. HMDs leverage the disparity between the left and right eyes to present a three-dimensional image. The magnitude of this disparity influences the depth of the image, and a larger disparity can lead to a more pronounced inconsistency in the patient's binocular focus and adjustment, contributing to visual fatigue. Additionally, the HMD’s display is in close proximity to the patient’s eyes, and prolonged focus on the screen can necessitate constant adjustments, further intensifying visual fatigue [63]. Therefore, it is crucial to control the duration of single-round games, allowing patients’ eyes to rest periodically.

The primary cause of motion sickness in VR rehabilitation serious games is sensory conflict. As all mapped motions come from the patient’s movements instead of a controller in VR rehabilitation, the primary factor contributing to motion sickness is latency caused by the device. Studies indicate that delays of less than 15 ms can effectively mitigate the occurrence of motion sickness [64]. Hence, in HMD-VR rehabilitation serious games, it is essential to prioritize the use of preset models (such as cylinders and cubes) to minimize complex models. This approach could alleviate the CPU computation and GPU rendering burden, consequently reducing latency, and addressing motion sickness concerns.

A reward cycle for long-term motivation

However, as rehabilitation training spans a considerable duration, the enjoyable aspects provided by HMDs alone may not be sufficient to sustain enduring interest among patients. Consequently, a long-term motivation promotion mechanism is necessary.

A reward mechanism is popular among rehabilitation serious games. It emphasizes the significance of the reward, as it could encourage patients to engage to get the reward in short-term. However, in a long-term, as patients receive numerous rewards, the allure of these rewards may diminish, leading to a decline in motivation.

Therefore, we propose a reward cycle, emphasizing the consumption part to promote the long-term motivation. As illustrated in Fig. 3, after the patient’s exercise performance, game results (success or failure) are determined based on established rules. The reward system gives patients resources, such as points and coins, for the patient’s successful game results. Upon accumulating a certain level of resources, patients can consume these resources to make selections or engage socially with others. This includes spending points on new interaction goals, entering rankings, and competing with others. This establishes a cycle of “patient movement \(\stackrel\) Result \(\stackrel\) Gain Sources \(\stackrel\) Choice/Social \(\stackrel\) patient movement”. In this cycle, if the patient fails to reach the next step—such as failing to achieve success in exercise or having insufficient resources for consumption—it amplifies the patient’s desire to progress to the subsequent stage. This represents external motivation, motivating them to exert greater effort. Once a cycle is completed, the choices and social interactions meet the patient's individual or relational needs, enhancing the patient's intrinsic motivation and fostering a commitment to long-term rehabilitation training.

Fig. 3

Reward cycle designed to sustain the patient’s long-term training cycle. Following the patient's exercise, game results (success or not) are determined based on the game rules. The reward system then offers resource rewards for the patient’s success. Upon accumulating a certain level of resources, these can be consumed, empowering patients to make independent choices or engage in social with others, thereby further encouraging the patient’s exercise

Indeed, the proposed loop builds upon a single reward by introducing another that necessitates prolonged effort from the patient. Thus, akin to a single reward, the attractiveness of the proposed cycle may diminish with many times repeated iterations. Hence, the cycle period should not be excessively short. However, if the time required to achieve the “long-term reward” is excessively long, it may seem unattainable for patients, thus failing to produce the desired effect. Hence, it’s crucial to appropriately control the length of the cycle period, with the optimal period possibly varying depending on individual personality traits. Additionally, as rehabilitation aspects required time is varies, the design of this cycle should ensure that (Number of cycles) * (Cycle period) ≥ (Rehabilitation duration).

VR rehabilitation serious game design framework

Combining the game function blocks (for rehabilitation function), VR factors, and reward cycle for long-term motivation, this paper presents a comprehensive design framework: Clinical Function Fun (CFI), as depicted in Fig. 4.

Fig. 4

VR Rehabilitation serious game Design Framework. a The overall structure of the VR rehabilitation serious game design framework. b Details the specific process of designing VR rehabilitation serious games. Clinical information, serving as a clinical representation, is positioned closer to the designer and rehabilitation therapist. “Interesting” represents the content directly experienced by players and is closest to them. “Function” signifies the mapping of the functions of rehabilitation to game function, acting as a bridge between the “clinical” and “interesting” aspects

Giving priority to clinical rehabilitation is essential in VR rehabilitation serious games. This process involves initially considering clinical rehabilitation motions, then determining the mapping of patient movements in the game based on these clinical movements, and ultimately structuring the game around patient movements. The MDA framework, where mechanics take precedence in game design, proves inadequate for meeting the specific design requirements of VR rehabilitation serious games. In response to this challenge, this paper introduces a novel VR rehabilitation serious game design framework: Clinical Function Fun (CFI). “Clinical” is the closest to the designers and therapists, serving to determine clinical information to guide game design. Subsequently, guided by this clinical information, the game “Function” is designed, to align with the fundamental functions of rehabilitation. Ultimately, “Interesting” is designed to sustain patients’ engagement in long-term rehabilitation training.

Clinical

VR rehabilitation serious games must be designed with a clinical perspective to ensure alignment with clinical requirements. The term “clinical” here pertains to providing clear information relevant to clinical practice, encompassing the patient’s stage goals, features of clinical rehabilitation exercises, evaluation of the patient's motor function and the assessment of patient’s state. Within this, the patient’s stage goal primarily aims to clarify the rehabilitation content needed by the patient at the current stage, providing direction for game rule design in “function.” For instance, in patients with mild to moderate stroke (patient stage), the goal may be to train the overall coordination function of the upper limbs (objective). The features of clinical rehabilitation motions involve breaking down the patient's rehabilitation motions into specific indicators, such as joint angle and posture retention time, which are instrumental for the later design of motion mapping in “function.” The assessment of patient motor function is to address the requirements of “difficulty” within the “function” aspect to accomplish personalized rehabilitation for patients. The state assessment method is employed to identify the patient’s current condition, such as using a pain scale, to determine the duration of the rehabilitation training within the game. Unlike motor function assessment, which is conducted only before and after the game, state assessment needs to be performed multiple times during the game process to determine when to conclude the game session.

Function

“Function” signifies the functionality of VR rehabilitation serious games, constituting the mapping of fundamental rehabilitation functions within the game. While “Clinical” encompasses information derived from therapists’ diagnoses and “Interest” represents the patient’s direct experience, the purpose of “Function” is to establish a relationship between them. Within “Function,” various components are incorporated, including the design of motion mapping, duration design, game rules, feedback mechanisms, and methods for setting and adjusting difficulty.

Movement mapping translates the rehabilitation motions that patients need to perform in a clinical setting into the content of VR rehabilitation serious games. More specifically, it is based on previously identified features of clinical rehabilitation motions, determining which patient movements will affect the virtual gaming world. In VR rehabilitation serious games, motion mapping plays a foundational role in the functional layer, influencing the design of game rules, and serving as the basis for KP feedback. Indeed, in the movement mapping process, ensuring adequate space for patients’ movements is paramount. In a virtual environment where patients cannot perceive the real world, there’s a risk of collisions or other hazards if the context is not carefully monitored and controlled.

The duration serves for goal in game rules. It can be determined by evaluating the patient’s current state or by setting predefined criteria, aiming to fulfill the required training volume as practiced clinically, such as “as often as possible,” “x times,” or “only if no pain is experienced.” Additionally, considering the potential side effects of VR devices, such as visual fatigue, it is essential to set the duration of single round games to allow patients to take timely breaks when needed.

The game rules constitute the fundamental content of game operation, primarily focusing on establishing the goal and gameplay. The goal is to gamify the duration of the game, specifically by translating it into in-game parameters such as the number of targets or the remaining time. The gameplay entails gamifying the previously mapped motion content while considering the stage goals of clinical, thereby determining relevant game parameters concurrently. Additionally, uncertainty, which means a certain of change should be considered in gameplay.

The feedback system in the game serves as the response to a patient’s actions, providing information about the impact of their actions on the virtual world. Feedback comprises performance-based (KP) feedback and result-based (KR) feedback. Knowledge of Performance (KP) feedback provides feedback about the motion itself; it focus on process; therefore, it needs to be designed based on motion mapping. Knowledge of Results (KR) feedback, on the other hand, refers to feedback that informs patient about the outcome or results of their motion, necessitating design considerations based on the game objectives outlined in the game rules. Feedback can be achieved through multiple modalities including visual, auditory, and tactile senses.

According to the flow theory [59], individuals can achieve a state of deep engagement and forgetfulness, leading to efficient learning, which requires a balancing personal skill and challenge. This theory led to the development of Dynamic Difficulty Adjustment technology (DDA), which is incorporated as a component of Difficulty in the proposed framework. The design of difficulty encompasses both the initial difficulty setting and adjustment, playing a crucial role in personalized rehabilitation for patients with varying degrees of injury. This element ensures that the difficulty levels always align with patients’ motor abilities, preventing them from feeling discouraged or bored. The difficulty setting and adjustment methods primarily revolve around the relevant parameters outlined in the game rules, making the design of difficulty inherently dependent on the design of these rules. The initial difficulty setting is established based on the clinical evaluation results of the patient’s physical ability. On the other hand, dynamic difficulty adjustment is rooted in the patient’s performance during the training process. This dynamic adjustment involves modifying the parameters defined in the game rules to ensure that the difficulty level aligns with the patient's physical and mental state.

Interesting

Like the aesthetics in MDA, the initial impression patients have in VR rehabilitation serious games is centered around whether the game is interesting. Consequently, “Interesting” becomes the element closest to the patient. This part primarily encompasses reward mechanisms, game stories, art design and music & sound design.

The reward mechanism is an incentive structure to sustain long-term training for patients, encompassing resources, consumption, and choices/socializing. Resources, often represented as points or coins, are acquired by patients upon achieving their goals, dependent on the design of game rules. Consumption involves utilizing acquired resources to modify game content, with choices or social interactions serving as specific means. In the context of choices, patients can independently make decisions that impact the game content, fulfilling their need for self-expression. Social interaction allows patients to engage with others in the game, fostering collaboration or competition and meeting their relational needs. By addressing the patient's self or relationship needs, intrinsic motivation is enhanced, facilitating long-term engagement in gaming training.

The game story is an extension of the game rules, presenting them to patients in the narrative. This allows patients to perceive the meaning and mission of the motions, immersing themselves in the experience. The game story incorporates various elements, with missions and objectives derived from the goals and gameplay outlined in the game rules. Missions, composed of multiple objectives, are further wrapped by the story to enhance the patient’s immersion. Moreover, a clear guide should be contained in the story.

The art design is grounded game story, shaping scenes and objects to provide patients with direct sensory stimulation. Moreover, the chosen art style plays a crucial role in determining the game atmosphere. Therefore, it is imperative to consider a style suitable for the target patient group during the design process. Additionally, when utilizing HMD, especially wireless HMD, it is advisable to employ preset models to minimize CPU computation and GPU rendering burden, thereby reducing latency.

Music and sound design involve creating background music (BGM) and sound feedback in Function-feedback. In rehabilitation serious games, pleasant sound effects are often utilized to reinforce positive feedback, thereby boosting patient motivation. Meanwhile, music design aims to enrich the game atmosphere to enhance patient immersion in the gaming experience. Therefore, the designed music should align with the game’s narrative to ensure consistency and enhance the overall gaming experience.