//Logo Image
AuthorsPo-Er Hsu (2013-01-28); recommended: Yeh-Liang Hsu (2013-02-10).
Note: This article is the Chapter 5 of Po-Er Hsu’s doctoral thesis “Development of an intelligent robotic wheelchair as the center of mobility, health care, and daily living of older adults.”

Chapter 5. User evaluation of the intelligent robotic wheelchair

This chapter presents the user evaluation of the iRW. As described in previous chapters, functional tests have been conducted to demonstrate the advantage of the functions designed for the iRW. In this chapter, the user evaluation is used to confirm the design concept on the mobility assistance functions and the multiple-DOF seat adjustment mechanism of the iRW. Manual wheelchairs are the most common mobility assistive device adopted by people with impaired mobility; therefore the usability of the iRW is compared against that of a manual wheelchair.

5.1  User evaluation of the mobility assistance functions

The mobility assistance functions of the iRW is based on the omni-directional vehicle which uses four Mecanum wheels to facilitate movement in all directions, including moving sideways, and zero radius of rotation. Three “operators”, the wheelchair user, caregivers, and the iRW itself, are considered in the mobility assistance design for the iRW. Man-machine collaborative control is reflected in the assignment of three “operator priorities” (in descending sequence): the wheelchair user, caregivers, and finally the iRW itself.

Five operation modes, all mutually exclusive, for the three operators are developed. “Joystick mode” is the main operation mode used by the wheelchair user. “Handlebar mode” and “teleoperation mode” are designed for caregivers who intend to push the iRW or to operate the iRW from a remote site. “Obstacle avoidance” and “indoor navigation mode” are the two semi-autonomous modes of the iRW, aiming to reduce the operation load of the wheelchair user or caregivers. In this section, first the operational efficiency of the five operation modes were compared, and then the usability of the iRW against that of a manual wheelchair evaluated.

Operation efficiency of five operation modes

As shown in Figure 5-1, to compare the operation efficiency of operation modes, the operation time from Position 1 to Position 4 using joystick mode, handlebar mode, teleoperation mode, and indoor navigation was measured. The total distance of the testing route was 46.40 m. Five healthy adults (three male and two female) participated in the test. The average age of the testers was 23.0 years old, and none of them had previous experience operating an electric wheelchair. Referring to Figure 5-1, the steps of the test were as follows.

Figure 5-1. Testing route of operation efficiency

(1)           A tester spent five minutes familiarizes himself/herself with the various operation modes.

(2)           With the iRW at Position 1, the tester used indoor navigation mode to transport himself/herself automatically from Position 1 to Position 4.

(3)           After the experimenter repositioned the iRW to Position 1, the tester used joystick mode to maneuver the iRW from Position 1 to Position 4.

(4)           After the experimenter repositioned the iRW to Position 1, the tester used handlebar mode to push the iRW from Position 1 to Position 4.

(5)           After the experimenter repositioned the iRW to Position 1, the tester used teleoperation mode to move the iRW from Position 1 to Position 4.

(6-14)      Steps 3 to 5 were repeated three times.

Each operation mode was set to the same speed as that of indoor navigation mode. The time required for each round was measured. The time required for indoor navigation mode was almost the same for all testers (an average of 194.2 seconds, with standard deviation of 1.3 seconds) and was used as the “standard operation time” in this test when estimating operation efficiency. The operation efficiency of a given operation mode was defined as the ratio of the “standard operation time” to the average operation time of the given operation mode. When the operation efficiency was less than 1, testers took longer to arrive at the desired location using the given operation mode than using indoor navigation mode.

Table 5-1 shows the operation efficiency of each mode in this test. As expected, teleoperation mode has the lowest efficiency. Although all of the testers were familiar with the environment, they paused frequently when operating the iRW from the limited view obtained from the camera of the tablet. The operation efficiency of teleoperation mode increased steadily from Round 1 to Round 3, showing a good learning effect of the operation modes. However, the learning effect from Round 1 to Round 3 was not significant statistically (p = 0.24 > α = 0.05).

The operation efficiency of joystick mode was slightly higher than 1. The testers quickly became proficient with the joystick, even though none of them had any experience operating an electric wheelchair. The standard deviation is the largest among the operation modes. As in the teleoperation mode, the operation efficiency of the joystick mode increased steadily from Round 1 to Round 3, although the learning effect was not significant statistically (p = 0.44 > α = 0.05). In this test, the motor control speed of all operation modes was set to be the same. In actual practice, the speed in joystick mode can be controlled by the wheelchair user, and the maximum speed could be set higher than that of indoor navigation mode, which would increase the operation efficiency of joystick mode.

The operation efficiency of handlebar mode was the highest among the operation modes. The learning effect of handlebar mode from Round 1 to Round 3 was statistically significant (p = 0.027 < α = 0.05), indicating that caregivers should find it easiest to become accustomed to “pushing” the iRW using the handlebar mode. As with joystick mode, the efficiency of handlebar mode is expected to increase when the speed is set higher in practice.

Table 5-1. The operation efficiency of operation modes

 

Indoor navigation

Joystick

mode

Handlebar mode

Teleoperation

Round 1

1

1.09

1.34

0.76

Round 2

1

1.11

1.37

0.80

Round 3

1

1.14

1.39

0.83

Average efficiency

1

1.11

1.37

0.80

Standard deviation

 

0.16

0.13

0.06

Usability comparison between the iRW and manual wheelchair

Six healthy participants (four males and two females, with an average age of 22.8) were recruited to conduct the user evaluation. In order to simulate the reduced ability of older adults, each participant was asked to wear a pair of thick gloves made of cotton to reduce touch, and a pair of knee pads and 1.5 kg sand bags to constrain mobility of the legs.

In this utility evaluation, the participants are requested to do the same tasks by using both the iRW and a manual wheelchair. The tasks include basic movements that a wheelchair user performs in the home environment, such as making turns, crossing a narrow passage, and passing through a door with a threshold following a predefined route. The steps of the evaluation are described below:

(1)           Spending five minutes to become familiar with the operation of the iRW or the manual wheelchair

(2)           Steering the iRW or the manual wheelchair to turn right, turn left, and then pass through a narrow passage (120 cm wide and 300 cm long) on the predefined route

(3)           Steering the iRW or the manual wheelchair to cross the threshold (2.5 cm) and enter the restroom

(4)           Steering the iRW or the manual wheelchair to move as close as possible to the toilet

(5)           Steering the iRW or the manual wheelchair to leave the restroom and return to the initial position.

After finishing the tasks, the participant was asked to fill in a questionnaire to collect the subjective responses to the use of the iRW and the manual wheelchair. In this study, the 5-point scale is used for user satisfaction (1 = very dissatisfied; 5 = very satisfied) regarding the mobility functions of the iRW. The 10-point scale is adopted to measure subjective fatigue (1 = no fatigue; 10 = total fatigue) while using the iRW or the manual wheelchair.

The operation times required for the iRW and the manual wheelchair were measured and compared. Figure 5-2 shows the average operation time for each of the Steps 2-5. In Step 2, the participant took a significantly longer time to pass through the narrow passage in the iRW (p = 0.0002 < α = 0.05). For Steps 2 to 5, the operation times are not significantly different between the iRW and the manual wheelchair. (Step 2: p = 0.06 > α = 0.05; step 3: p = 0.28 > α = 0.05; step 4: p = 0.38 > α = 0.05).

Nevertheless, based on the subjective responses, the participants generally felt less fatigued while operating the iRW than the manual wheelchair. The participants reported very low fatigue (< 3.0) for all body parts while operating the iRW. They felt most fatigue at the right wrist (2.7). For the manual wheelchair, the participants reported a higher fatigue (> 3.0) for all body parts except the abdomen (1.2) and the calves (2.8). The most-fatigued body parts were shoulders, forearms, and wrists (5.8). In addition, while using the iRW, the participants were satisfied with the user interface by using the joystick (average score = 4.5) and the sense of security during the forward/backward movements (average score = 4.0).

Figure 5-2. The average operation time of each step

5.2  User evaluation of the multiple-DOF seat adjustment mechanism

In order to increase the independence of the wheelchair user, the multiple-DOF seat adjustment mechanism of the iRW is achieved by a four-axis Stewart platform to maintain a concise structure, light weight, and intuitive control interface. The multiple-DOF seat adjustment mechanism is capable of the motions of heaving, pitching, and swaying to provide seat elevation, tilt-in-space, and sideways movement functions. Special consideration is paid to arranging the actuators to reduce the control complexity of the parallel mechanism, so that the wheelchair user can make the seat adjustment by simply pressing a button.

As described in the first chapter, it has been shown in previous studies that the adjustability of tilt-in-space angles of a wheelchair can help to relieve seating pressure and improve seating comfort. However, there are few evidences supporting how the elevation and sideways movement may benefit wheelchair users in transfer assistance. Thus in this chapter, the usability of the transfer assistance function provided by the iRW was evaluated. Both operation time and subjective responses for transfer activities of the iRW were compared with those of a manual wheelchair.

Evaluation procedure

Six healthy participants (three males and three females, averaged 22.8 years old) were recruited for user evaluation. Each participant was requested to perform the same transfer activities with the iRW and then with a manual wheelchair, by simulating how one moves from either the iRW or the manual wheelchair to a piece of furniture in the home environment, as well as returning to the iRW or the manual wheelchair. While using the iRW, the participant may use the elevation adjustment and sideways movement functions to facilitate the transfer activities. As for the transfer activities with the manual wheelchair, the participant can use a transfer board. In order to investigate the effectiveness of the seat elevation adjustment and sideways movement functions of the iRW, two conditions of transfer activities were considered. In the first part of the evaluation, the iRW, manual wheelchair, and the target plane were at the same height of 45 cm. In other words, the participant transfers between two planes with the same height, by using the sideways movement function of the iRW or the transfer board with the manual wheelchair. In the second part of the evaluation, the height of the target plane was 50 cm, while the heights of the seat pans of the iRW and the manual wheelchair were both 45 cm. Thus, the participant needs to use the seat elevation adjustment and sideways movement functions of the iRW or the transfer board with the manual wheelchair to move between two planes with different height levels. The steps of the evaluation are described as follows.

(1)           Spending five minutes to become familiar with the operation of the elevation adjustment and sideways movement functions of the iRW and the use of a transfer board with the manual wheelchair

Part I

(2)           Crossing a horizontal gap of 15 cm to transfer from the iRW (with the seat height set at 45 cm) to the target plane with the height of 45 cm by using the sideways movement function to move as close as possible to the target plane

(3)           Transferring back from the target plane to the seat pan of the iRW and using the sideways movement function to return the seat to the default position

(4)           Repeating step 2 and step 3 for five times

(5)           Repeating steps 2 to 4 with the manual wheelchair (with the seat height of 45 cm) by using the transfer board to cover the gap to perform transfer activities

(6)           Filling the questionnaire to collect the subjective responses toward the usability of the iRW and the manual wheelchair for transfer activities

Part II

(7)           Crossing a horizontal gap of 15 cm to transfer from the iRW (with the seat height set at 45 cm) to the target plane with the height of 50 cm by using the elevation adjustment and sideways movement functions to move as close as possible to the target plane

(8)           Transferring back from the target plane to the iRW and using the sideways movement function to return the seat to the default position

(9)           Repeating step 7 and step 8 for five times

(10)        Repeating steps 7 to 9 with the manual wheelchair (with the seat height of 45 cm) by using the transfer board to cover the gap to perform transfer activities

(11)        Filling the questionnaire to collect the subjective responses toward the usability of the iRW and the manual wheelchair for transfer activities

Comparison of operation time

The operation time required for the transfer activities with the iRW and the manual wheelchair were measured and compared. Figure 5-3 shows the average operation time for the different transfer activities with both mobility aids. While transferring to the target plane of 45 cm (Part I), participants took a significantly shorter time to perform transfer activities with the iRW than the manual wheelchair (p = 0.00). Further, there was a significant difference in operation time among participants when transferring from the iRW to the target plane (p = 0.00). This may be due to the preference when determining how close is enough for the individual to move the body. In addition, while transferring from the target plane to the iRW, no significant difference was found in operation time among participants (p = 0.20). It is because that when returning to the default position, the seat of the iRW will move to the destination automatically once the button is pressed. In other words, user adjustment is not required, and hence individual difference will be eliminated. Further, no significant difference was found among the participants in operation time while using the manual wheelchair to perform the transfer activities in both directions (to the target plane: p = 0.06; to the wheelchair: p = 0.36). The results imply that the iRW provided a better efficiency for transfer assistance than the manual wheelchair did. In addition, individual difference in operation efficiency was found while using the sideways movement function of the iRW, but it was not observed when using a transfer board with a manual wheelchair.

Figure 5-3. The average operation time (s) for the transfer activities with the iRW and the manual wheelchair

When the height of the target plane is at 50 cm (Part II), i.e. higher than the seat pan of both mobility aids, the participants also took a significantly shorter time to transfer to the target plane by using the iRW than the manual wheelchair (p = 0.004). Besides, when transferring from the iRW to the target plane, there was also a significant difference in operation time among participants (p = 0.02). In this case, the possible reason is the preference of height and distance differences between the seat pan and the target plane for the safe transfer. However, when transferring back to the iRW, there was no significant difference in operation time among participants (p = 0.06). It is because that when returning to the default position, the seat of the iRW will move to the destination automatically once the button is pressed. Moreover, while using the manual wheelchair to perform the transfer activities, there were significant differences in operation time among participants in both directions (to the target plane: p = 0.001; to the manual wheelchair: p = 0.000). It might be caused by the height difference between the seat pan of the manual wheelchair and the target plane. For people with different skills and experiences, the time required for climbing up or sliding down to another plane with a different height may differ.

Subjective responses

As for the subjective responses, the 5-point scale was first used to evaluate the user satisfaction (1 = very dissatisfied; 5 = very satisfied) toward the iRW. The participants generally felt satisfied with the safety (average score = 3.7) and stability (average score = 3.5) while moving the seat of the iRW up and down and in sideways. In addition, the 5-point scale was adopted to investigate how the participants feel whether the transfer assistance provided by the iRW or the manual wheelchair fits their needs (1 = not at all; 5 = very much). The participants reported that the seat elevation adjustment and sideways movement functions of the iRW (average score = 4.3) fitted their needs for transfer activities better than the combination of the transfer board and the manual wheelchair did (average score = 3.0; p = 0.006), no matter whether there was a height difference between the two planes.

On the other hand, the 5-point scale was further used to evaluate the level of easiness (1 = very difficult; 5 = very easy) and the level of convenience (1 = very inconvenient; 5 = very convenient) toward the transfer assistance provided by the iRW and the manual wheelchair. Besides, the Borg CR-10 scale was adopted to measure the exercise intensity levels (0 = nothing at all; 3 = moderate; 10 = very very hard) while using the iRW and the manual wheelchair for transfer. The statistics in the two parts of evaluation (“to the target plane” and “from the target plane”) are presented and discussed as follows.

In the first part of the evaluation, the participants felt the iRW can make the transfer activities slightly easier (4.0 for the iRW; 3.3 for the manual wheelchair; p = 0.07) and significantly more convenient (4.2 for the iRW; 2.5 for the manual wheelchair; p = 0.01) than the manual wheelchair in both directions, as shown in Table 5-2. Moreover, the participants reported a significantly lower exercise intensity while operating the iRW (average score = 2.5) than the manual wheelchair (average score = 4.5; p = 0.02) in both directions. The results demonstrate that the iRW can help the user to perform transfer activities between two planes at the same height with better convenience and less effort than the manual wheelchair.

Table 5-2. Statistics of the subjective responses toward the transfer activities at the same height (Part I)

Direction

Item

Easiness

Convenience

Exercise intensity

To the target plane

iRW

4.0

4.2*

2.5*

MWC

3.3

2.5*

4.5*

From the target plane

iRW

4.0

4.2*

2.5*

MWC

3.3

2.5*

4.5*

*: significant difference (p < 0.05) between the iRW and the manual wheelchair (MWC)

In the second part of the evaluation, the iRW assisted the participants to transfer to the higher target plane significantly more easily (3.8 for the iRW; 2.6 for the manual wheelchair; p = 0.02) and significantly more conveniently (4.3 for the iRW; 2.6 for the manual wheelchair; p = 0.008) than the manual wheelchair, as shown in Table 5-3. While transferring back from the target plane, there was no difference in easiness (3.5 for the iRW; 3.8 for the manual wheelchair; p = 0.24) and convenience (3.8 for the iRW; 3.3 for the manual wheelchair; p = 0.25) between the two mobility aids. The possible reason is that transferring from the higher target plane to the manual wheelchair was enabled by sliding down through the transfer board, which makes it easier and more convenient than climbing up. Regarding the exercise intensity of the transfer activities, the participants reported a significantly lower intensity with the iRW (average score = 3.0) than the manual wheelchair (average score = 5.2; p = 0.03) while transferring to the target plane. When transferring back from the target plane, the participants perceived a slightly lower intensity with the iRW (average score = 3.0) than the manual wheelchair (average score = 4.7; p = 0.05). The results reveal that the iRW can help the user to perform transfer activities from a plane to another higher one with more ease, better convenience, and less effort than the manual wheelchair.

Table 5-3. Statistics of the subjective responses toward the transfer activities at the different height (Part II)

Direction

Item

Easiness

Convenience

Exercise intensity

To the target plane

iRW

3.8*

4.3*

3.0*

MWC

2.6*

2.6*

5.2*

From the target plane

iRW

3.5

3.8

3.0

MWC

3.8

3.3

4.7

*: significant difference (p < 0.05) between the iRW and the manual wheelchair (MWC)