//Logo Image
Authors: Yi-Shin Chen, Yeh-Liang Hsu, Jun-Ming Lu (2012-04-16); recommended: Yeh-Liang Hsu (2012-09-03).
Note: This paper is presented at the 43rd International Symposium on Robotics (ISR 2012), Taipei, Taiwan, August, 2012.

TRiCmini+ – Telepresence Robot for Interpersonal Communication for Older Adults

Abstract

Aging is associated with an increased risk of isolation. Information and communication technologies have been utilized to assist homecare of older adults. However, in addition to transmitting vital sign data for healthcare purposes, older adults may expect to share their life experiences and feelings by different forms of interactions with their children and family members. Communication tools did facilitate interpersonal communication in terms of real-time verbal communication. Nevertheless, nonverbal communication, such as facial expression and body language, is more powerful and efficient in conveying ideas, thoughts, and emotions.

TRiCmini+”, a telepresence robot for interpersonal communication developed in this research, demonstrates extensive capability to provide different levels of “care delivery” to older adults through vital sign monitoring, interpersonal communication, robotic movements, and social network integration. TRiCmini+ integrates two distinct applications, the “Care Delivery Frame (CDF)” and the telepresence robot. CDF is an App designed for older adults as an information channel on the tablet PC, which is also the “face” of TRiCmini+. CDF is integrated with social network services, and remote caregivers can share messages, photos or video clips with older adults. Moreover, the tablet is also the control center of the robot. In this innovative control structure, the robot control App can be downloaded and maintained easily through the Internet.

The prototype and functional test of TRiCmini+ have been completed. The Internet bandwidth required is at least 150/5kB for smooth real-time communication. Currently it is under usability evaluation, including the interface design and operation efficiency for both remote and local users. Finally, the effectiveness of communication will be evaluated in real application scenarios to confirm the “care delivery” in different forms provided by TRiCmini+ actually meets the expectation of older adults.

Keywords: Interpersonal Communication, Care Delivery, Gerontechnology, Telepresence Robot.

1.    Introduction

Aging is associated with an increased risk of isolation. Information and communication technologies (ICT) have been utilized to assist homecare of older adults. Telehomecare, or the more modern term home telehealth, can be defined as “the use of information and communication technologies to enable effective delivery and management of health services at a patient’s residence” [1]. It has been the primary form of applying ICT in homecare. However, in addition to transmitting vital sign monitoring data for healthcare purposes, the essence of “care” to the older adults should emphasize more on the care from people. The older adults may expect more on communicating with their children and family members, as well as sharing of life experiences and feelings by different forms of interactions.

Communication tools such as mobile phones and video conferencing systems did facilitate remote interpersonal communication in terms of real-time verbal communication. Nevertheless, verbal communication is not the only means of face-to-face communication. In fact, nonverbal communication, such as facial expression and body language, is more powerful and efficient in conveying ideas, thoughts, feelings, and emotions. Mehrabian and Ferris reported that in face-to-face communication, clues from spoken words, voice tone, and facial expression contribute 7%, 38%, and 55% respectively to the total comprehension [2]. Besides, based on the analyses of recorded video tapes, Argyle et al. claimed that non-verbal cues had 4.3 times the effect of verbal cues in communication [3].

The “Telepresence Robot for Interpersonal Communication (TRiC)” has been developed by Gerontechnology Research Center (GRC) in Yuan Ze University. It provides not only verbal communication but also the nonverbal aspects of interpersonal communication [4]. TRiC is used as the avatar or agent of the caregiver in a remote home environment. As shown in Figure 1, with the wireless LAN router located in the home environment, TRiC is connected to the Internet for data and audio/video transmission. The remote user manipulates TRiC through the user interface on a laptop/desktop computer to freely move it around and communicate with the local user, who is staying with the robot in the home environment. However, one problem with TRiC is that, as shown in Figure 2, TRiC becomes a “dummy” if no one logins from remote site.

Figure 1. User scenario while remote user login the TRiCmini.

Figure 2. TRiCmini becomes a “dummy” if no one logins from remote site.

The “Care Delivery Frame (CDF)”, another development from GRC, is a software App designed for older adults who are not familiar with the operation of computers and Internet as a unique information channel on the tablet PC [5]. In addition to health data monitoring, children/caregivers can “deliver care” to their seniors not living together by warm messages and thoughtful reminders on the CDF, as well as sharing their feelings, joy, and life experience by sharing photos and video clips remotely on the CDF (see Figure 3).

Figure 3. Care Delivery Frame and its four main functions.

As shown in Figure 4, “TRiCmini+” presented in this paper is the 3rd generation of TRiC which integrates two distinct applications, CDF and the telepresence robot. TRiCmini+ is tele-operated by the remote user via the Internet, and the three-dimensional face-to-face interaction is duplicated with two-way audio and video communication, as well as robotic movement. By integrating with CDF, TRiCmini+ also demonstrates extensive capability to provide different levels of “care delivery” to the older adults through vital sign monitoring and other forms of interactions, even if no one logins from remote site to control TRiCmini+.

Figure 4. TRiCmini+ and its four main functions.

Currently, TRiCmini+ provides the following 4 main functions:

Ÿ   Vital sign monitoring

Ÿ   Interpersonal communication

Ÿ   Facial expressions and robotic movements

Ÿ   Social network integration

Section II introduces the system structure of TRiCmini+ and the technical details of each module; Section III describes the functions of TRiCmini+ which deliver care in different forms. Finally Section IV concludes this paper.

2.    System structure of Tricmimi+

Figure 5 shows the information structure of TRiCmini+ in the home environment. In this structure, the tablet PC, which contains the CDF software, robot control function, video/audio conferencing and facial expression functions, is the core of the system.

Figure 5. The information structure of TRiCmini+

At the local user site, vital sign data such as blood pressure and blood glucose are transmitted to the tablet via Bluetooth for vital sign monitoring purposes using the CDF software. The remote user can login the tablet via the Internet to browse the vital sign data or to control TRiCmini+ for interpersonal communication.

Basically, TRiCmini+ can be separated into a tablet and a robotic vehicle. As shown in Figure 6, the tablet conducts Internet communication, audio/video conferencing, and is also the face of TRiCmini+ for displaying facial expressions. The robotic vehicle contains a power module and a movement module. The tablet can also be easily removed from the robotic vehicle for personal use. The technical details of the 5 modules of TRiCmini+ are explained below.

Figure 6. The system structure of TRiCmini+.

(1)  Internet communication

TRiCmini+ is connected to the Internet for data/command transmission via 3G mobile communication of the tablet. The remote user relies on the user interface software on a computer or mobile device to login to the tablet to communicate with the local user. Commands from the remote user are transmitted to the tablet to trigger facial expression and audio/video conferencing function within the tablet, or are relayed to the movement module of the robotic vehicle via Bluetooth to control robot movement.

(2)  Facial expression

For more engaged user experiences, TRiCmini+ is given the ability to present facial expressions and whole-body emotions. Facial expressions are built as animations on the tablet which allows the remote user to switch among the 6 universal facial expressions proposed by Ekman [6], “happiness,” “anger,” “disgust,” “sadness,” “fear,” and “surprise.” In addition, the servo motors in the movement module help to create TRiCmini+’s arm gestures. By combining these two features and some special patterns of robotic vehicle movements, TRiCmini+ will be able to produce multiple whole-body emotions. As the remote user chooses one of the 6 whole-body emotions, the movement module is triggered for the associated pattern of arm gestures and movements, while the tablet enables the presentation of specific facial expression.

(3)  Audio/video conferencing

This module enables two-way audio and video communication through TRiCmini+. The voice and images of the local user are captured by the microphone and camera on the tablet. With the neck design of the TRiCmini+, the camera on the tablet can be controlled by the remote user to trace the local user. An additional speaker embedded in TRiCmini+ further amplifies the audio to the local user. Moreover, the local user can choose to display the image of the remote user or the animated facial expression on the tablet.

(4)  Movement module

The core of the movement module in the robotic vehicle is an Arduino microprocessor. The Arduino microprocessor is equipped with Bluetooth shield for data transmission between the tablet and the robotic vehicle. There are 3 sets of motors and omnidirectional wheels controlled by a motor controller, and 3 ultrasonic sensors in this module. Once a command from the remote user is received by the tablet and relayed to the movement module, the controller will run the algorithm to determine how the motors will trigger the omnidirectional wheels. In this way, TRiCmini+ can freely move forwards/backwards or left/right with the speed of about 12 cm/s, as well as turning clockwise/counterclockwise. The ultrasonic sensors will help to detect the objects in the surrounding environment. After the execution of algorithms on the controller, motors will work in varied ways for different purposes.

(5)  Power module

This module includes a 12V LiFePO4 battery and a power management circuit board. If the battery is about to run out, an LED light on the power management circuit board will flash. Then, as the local user plugs into the electric socket for charging, the light keeps shining. Once battery is fully charged, the light goes off.

The appearance of TRiCmini+ is exactly the user interface for local users. For a higher level of anthropomorphism, as shown in Figure 7, its inner structure consists of the head (the tablet), body (containing the movement module and power module), two arms (integrated with servo motors for arm motions), and a round base (equipped with omnidirectional wheels for movements). Figure 8 presents the appearance design and basic spec of TRiCmini+. Considering the convenience of use and the friendliness, TRiCmini+ is in a compact size of 45.5 cm tall and weighing 3.6 kg.

The prototype and functional test of TRiCmini+ have been completed. The Internet bandwidth required is at least 150/5kB for smooth real-time communication. Currently it is under usability evaluation, including the interface design (software interface and robot facial/physical expressions) and operation efficiency for both remote and local users. Finally, the effectiveness of interaction in different forms will be evaluated in real application scenarios (home environment and senior users) to confirm the “care delivery” provided by TRiCmini+ actually meets the expectation of older adults.

Figure 7. The arms and skeleton design of TRiCmini+.

Figure 8. External design and basic spec of TRiCmini+

3.    The Different Forms of Care Delivery

Basically, interaction through TRiCmini+ in either verbal or non-verbal ways will be generated in 4 different forms: vital sign monitoring, interpersonal communication, physical movement and social network integration.

(1)  Vital sign monitoring

The basic care delivery form is the vital sign transmission, storage and analysis. It is also the smallest home telehealth system. All technical functions of a home telehealth system are built on the tablet. The vital sign data are transmitted to the remote users via the Internet. As shown in Figure 9, remote users browse the health monitoring data and do the data analysis on their smart phones.

Figure 9. Vital sign data monitoring and analysis interface.

(2)  Interpersonal communication

Older adults can use the video and audio communication function to do the verbal communication with their families/caregivers through TRiCmini+, similar to using telephones or video conferencing system.

For the local user, the voice will be louder or softer when TRiCmini+ comes closer or farther, which is similar as in the real face-to-face interaction between two people. As the local user responds, the voice message will be received by the microphone and then transmitted to the remote user’s side as well as the video images. Through the speaker or earphone, the remote user will hear exactly what the local user says.

As shown in Figure 10, the user interface for remote users enables the user-friendly and intuitive manipulation of the position of the camera, the omnidirectional movements, facial expressions and whole-body emotions. Towards a universal design for all users, graphic symbols are used for better recognition. Besides, consistency and usability of the user interface are emphasized. Moreover, the content layout follows user experiences and expectations.

Figure 10. Interpersonal communication interface for remote users.

(3)  Facial expressions and robotic movement

Nonverbal communication plays an important role in an interpersonal communication. In addition to spoken words, messages can also be communicated through other symbols. Thus, by integrating facial expressions and robotic movement, simulated physical access are enabled to deliver the nonverbal ways of communication. Normally, the facial expressions display automatically by following the setup rules while no one login from remote. Remote users or caregivers can use the robot control function to achieve not only the movement control but also the nonverbal communication by choosing the facial expressions and robotic movements.

As discussed earlier, TRiCmini+ is given the ability to present facial expressions and even whole-body emotions among the 6 universal facial expressions. The expressions of robotic movements are extended from facial expressions such as waving, hugging, raising hand, covering face (to express surprising). By combining facial and robotic movements and some special patterns of movements, TRiCmini+ will be able to produce multiple whole-body emotions.

(4)  Social Network Integration

Social network services (SNS) such as Facebook are the most popular communication platforms for younger generation. Users register in Facebook, create a personal profile, then they may add other users as friends, and exchange messages, photos, video clips, etc. Facebook is not only a platform for social activities but also create the social contents for its users.

To raise motivation for younger generation to interact with the older adults, TRiCmini+ is integrated with Facebook to provide a wide range of information sharing easily and conveniently. TRiCmini+ (actually the CDF software in the tablet) can be a “friend” to the children/family members on Facebook. Vital sign data monitoring, remote photo sharing and caring messages can all be done from Facebook by the remote user, as shown in Figure 11.

Ÿ   Vital sign data monitoring message: The health monitoring data received by the tablet be posted as a message on Facebook automatically. Thus, children/family members can get this information if they include TRiCmini+ as their “friend” on Facebook.

Ÿ   Caring messages: Children/family members can send warm caring messages to their seniors to display on the tablet via Facebook.

Ÿ   Remote photo sharing: Children/family members uploaded photos and videos clips to Facebook. , as the “friend” on Facebook, TRiCmini+ in the local user site, downloaded those photos, video clips automatically. Management of the display sequence and timing on the tablet can be performed remotely.

Collaborating with information service companies, tablet can also be a platform to display life information such as weather, shopping, as well as music and other entertainment information.

Figure 11. CDF interactive with personal Facebook account

4.    Conclusion

Older adults may expect to share their life experiences and feelings by different forms of interactions with their children and family members. TRiCmini+” has been developed to provide both verbal and nonverbal aspects of interpersonal communications. The three-dimensional face-to-face interaction is duplicated with two-way audio communication. It also demonstrates extensive capability to provide different levels of “care delivery” to older adults through robotic movement, vital sign monitoring, and other forms of communications.

TRiCmini+ delivers an innovative system infrastructure of telepresence robot by using the tablet is also the control center of the telepresence robot, and robot control functions are developed as another App on the tablet. Under this innovative control structure, the robot control App can be downloaded, maintained and updated easily through the Internet. TRiCmini+ also integrates with social network services such as Facebook, in order to raise motivation for younger generation to interact with the older adults.

In summary, TRiCmini+ intends to provide different forms of interaction and care delivery in order to meet the expectation of older adults.

References

[1].    Office of Health and Information Highway, Health Canada, “International Activities in Tele-homecare: Background paper,” September (1998).

[2].    Mehrabian, A. & Ferris, S. R., “Inference of attitudes from nonverbal communication in two channels”, Journal of consulting psychology, 31(3), 248-252, (1967).M. Young, The Technical Writer’s Handbook.,  Mill Valley, CA: University Science, 1989.

[3].    Argyle, M., Salter, V., Nicholson, H., Williams, M. & Burgess, P. The communication of inferior and superior attitudes by verbal and non-verbal signals”, British journal of social and clinical psychology, 9, 222-231 (1970).

[4].    Tsai, T. C., Hsu, Y. L., Hsu, P. E., “Developing a telepresence robot for interpersonal communication with the elderly in a home environment”, 2007 CACS International Automatic Control Conference, 2007/11.

[5].    Chen, Y. S., Hsu, Y. L., Wu, C. C., Chen, Y. W., Wang, J. A., Development of the Care Delivery Frame for senior users, the 9th International Conference on Smart Homes and Health Telematics (ICOST 2011), 2011/06.

[6].    Ekman, P. Universals and cultural differences in facial expressions of emotion (Ed.), Nebraska Symposium on Motivation, Lincoln: University of Nebraska Press (1971).