A talk by an expert “I want to hear what this person has to say now” with Kenta Toshima

Technologies such as metaverse and VR have great potential not only for the entertainment industry, but also for human "care".

What made you decide to combine the fields of welfare and nursing care with VR and the metaverse?

Kenta Toshima (Toshima): Originally, my previous job was in the welfare industry. At the facility, I often worked as a caregiver and a rehabilitation worker, or a so-called “functional training instructor,” working one-on-one with each elderly person or person concerned.

I thought about motivating them to rehabilitate and to keep coming to the facility without a break, and at first I did an activity for about a year in which I used photos and videos to show them places they remembered from their old days.

Many of the participants wanted to go back to their old scenery and places they had experienced in the past. In reality, it is difficult for them to go there due to physical problems, so we came up with a project draft that suggested using the media instead.

As we continued our activities, while they were happy to see us, there was a discrepancy between the photos and videos and their own memories. I thought this was the difference between “memory” and “experience.

That is why we started a project around 2014 called “VR Travel,” in which “memorable places” are filmed from 360° angles, and the viewer can “experience” the images by changing the angle and perspective of the viewer using a head-mounted display while in the facility.

I believe that some elderly people are not comfortable with digital tools.

Toshima: In general, it is often thought that “the elderly are not good with digital tools. However, I have seen many people over the age of 65 who have smartphones and enjoy playing games on them.

In addition, the elderly are the generation that watches a lot of TV, and recently VR is being introduced on TV more and more, so I think there is a background of less resistance due to that influence. Of course, there are those who are resistant. I don't suddenly bring VR goggles and ask everyone to try it either.

I may start by having those who are actively interested experience it, and then mirror the images on a TV so that others can step back and watch from a different position.

In fact, some people who are experiencing a gradual limitation on their physical functions and energy, or who have been in the facility for a long time, have a desire to get out. We want to encourage that feeling.

If we talk about how digital tools are 'tools to make that feeling come true,' you will understand. We will start with such communication.

"VR travel" was born out of caregiving that confronts each person involved.

Is there more than just simple VR, or are you also working on projects related to the metaverse?

Toshima: In our laboratory, we do that as well, and we also do VR travel with multiple people using avatars and creating systems. In the early days, it was a “one-person” experience where you enjoyed a memorable place, but now we are researching experiences where you can share the VR space with someone else.

One of the main features of VR that distinguishes it from photography and video is that it allows people to move their bodies in accordance with their own interests. Data shows that the amount of conversation increases as feedback from active movement. We also found that the point at which people look at the same 360° space differs from person to person.

Some people look up, while others look at their feet. When they discover something, they all look at it in the same way and start talking again. This is completely different from showing something on an iPad or playing a game with others.

Unlike a fixed interface, there is more freedom for the individual to make choices, and it is also helpful for the caregivers to be able to interact with each other. When I travel with people in VR, I sometimes ask them to guide me through things that I didn't notice on the real trip when I was there.

I understand that you are currently affiliated with the University of Tokyo's Research Center for Advanced Science and Technology (RCAST).

Toshima: In my laboratory, we are conducting research on the psychological and physical effects of VR on the elderly. Originally, when we had a “VR trip” at a nursing care facility, the response from everyone and the facility was better than we had expected.

So we did a crowdfunding project to travel across Japan and around the world to capture the memorable places of the people involved using a 360° camera. Eventually, I was asked to give a talk at a domestic academic conference called the “VR Society,” which led me to join the laboratory.

The VR technology itself has existed for more than 60 years, but the field as a whole is now in the phase of how to implement VR technology in society.

I really feel that people are enjoying VR travel as a form of entertainment. However, in the field of welfare, there is a demand for “what is the effect” on the other side of fun. The nursing care field is always busy. In order to be officially adopted, if we can show “effectiveness” and “results,” it will be easier to get into the pace of being used.

Right now, we are researching areas such as “Why does this person feel happy?” and “What made this person's mind and body move in response?

Currently, the functions of VR goggles already allow us to obtain quantitative and constant data such as eye tracking and 'where did the person look? In the paper, it was proven that using VR goggles “improves the ability to move the neck” and “improves the ability to look around carefully.

This is very important to me as a rehabilitation worker in the field. There is training and rehabilitation that says, “Let's raise your legs and arms,” but there are not many menus that move the neck.

I think it is interesting and useful to be able to induce this naturally and link it to rehabilitation.

So you are saying that VR is useful not only for simply enjoying content, but also for rehabilitation and data collection.

Toshima: The ability to collect data means that we can provide content that is more tailored to the needs of the individual and has a higher resolution to meet those needs.

Currently, we combine content based on our own experiences and provide it locally, but I feel that if we can collect detailed data, we will be able to provide more effective content for people in the future, in terms of social implementation.

It can also serve as a rehabilitation tool as well as a monitoring tool to inform people of changes in their caregiving status, which is necessary.

What other projects are you working on?

Toshima: We are working on a “VR blowgun” as part of the “rehabilitation that incorporates VR with activities” that we are doing at nursing care facilities. When it comes to breathing rehabilitation, there are only specialized tools for “inhaling and exhaling,” but we focused on the popular blowgun, which is a popular activity at nursing care facilities.

The user wears VR goggles, uses a microphone to sense the strength of his/her breathing, and uses a combination of neck angles to hit a target in VR. The purpose of this project is to make rehabilitation fun with VR.

In a current project that Meta and the government are working on together, we are asking elderly people who are enjoying Pokemon Go to go out and shoot video material for VR. This will be a good opportunity for the elderly to go out into the city and contribute to society.

And when they use care services, it will be content they can use themselves. In the future, we are thinking of matching “people who can shoot VR” and “people who want to see the shot VR” according to their needs.

Although the focus in VR and the metaverse tends to be on technology, the way in which people are connected to each other will also become important.

Related Other Project 1 “VR Blowgun

This project is one of the “rehabilitation projects that incorporate activities into VR. This project is a VR version of the “blowgun” activity, which is popular at nursing homes and also serves as respiratory training. The strength of the breath is sensed with a microphone, and the combination of the microphone and the angle of the neck is used to hit a target on the VR screen. While “VR travel” involves superficial muscle training, this is a fun way to rehabilitate internal organs in VR.

As the need for “home care” increases in the future, the metaverse space will accelerate as a new “place to be” for the elderly and those in need of care.

We often hear that monetization is difficult in the VR/metaverse industry. What are your thoughts on this, including future prospects?

Toshima: In the field of nursing care and welfare, we have not yet entered the monetization phase. The industry itself is basically based on long-term care insurance, so when activities are brought in from the outside, most of them are at the facility's actual expense.

So, either the facility pays for the activity or the elderly pay for it themselves. On the other hand, the term “senior citizen facilities” covers a wide range of facilities, including special nursing homes for the elderly and serviced residences for the elderly.

In order to survive in the future, facilities will need to be unique. In the future, the elderly who seek a variety of needs will be the group that requires nursing care. They are looking for ways to use digital technology not only to improve work efficiency, but also to enhance their quality of life.

In such a situation, my project has been invited to various places as a model case. The current focus is on having people come to nursing homes, but from now on we expect to see a gradual shift toward increasing the number of elderly people who can continue to live at home.

In such an environment, VR is portable and can provide a variety of experiences and rehabilitation programs at home.

In addition, in the metaverse area, I expect that as home care advances, it will become more difficult for the elderly to interact with others, and they will become more isolated. I believe that areas such as gathering places with digital tools will surely accelerate in the future.

I believe that the fields of VR and metaverse are generally associated with gaming and entertainment. What do you think are the similarities and differences with the welfare and nursing care fields?

Toshima: I am not very familiar with the entertainment field, but I think there are many basic similarities. On the other hand, in the case of welfare, it is not often the case that one simply heals one's body with surgery or medication and that is the end of it, but rather the direction is to support the person concerned over a long span of time.

The older a person gets, the more devices are attached to his/her body. For example, glasses and canes, and in the case of pacemakers, they are placed inside the body. I think that welfare and nursing care are compatible with VR technology in the form of a “digital cane” that is connected to the outside world as one of these devices.

Currently, people wear optical glasses, but I think there will be cases in the future where people will be able to see better through the camera on the glasses. Digital technology also makes it possible to add or subtract information. If we can create gradations in reality, VR, and AR, I think this will naturally become common, not only in the entertainment and welfare fields.

This is also true in the context of the metaverse mentioned earlier, but I believe that the VR domain will develop as a new place for people whose activities in real space are becoming restricted or narrowed. In a real space, elderly people can only socialize with people of the same generation or with people they meet regularly at senior citizens' associations or parks.

In a metaverse space, however, people of all ages and nationalities can gather together. I think it would be great if the space could be developed into a platform with a high degree of freedom. Of course, what kind of “comfortable” culture will be created by the people and systems within the gathering will be a different story.

▶Related Other Project 2: “Future welfare project using VR

This project is being conducted in collaboration with Meta, which operates Facebook, and the government. We will hold a workshop for shooting VR contents to be used for “VR travel,” and ask seniors who are still in good health to take pictures. This matching will provide an opportunity for energetic seniors to get out on the town and contribute to society. In the future, when the filmmakers themselves need nursing care services, they will be able to look back on the footage they filmed as a “VR trip.

Profile

Kenta Toshima

Kenta Toshima is an academic specialist at the Inami/Kadouchi Laboratory, Research Center for Advanced Science and Technology, the University of Tokyo, where he has worked since 2014 as an employee at a nursing home and started providing VR reminiscence tours to caregivers who have difficulty going out. The company is currently working on a new project to provide VR reminiscence tours to those in need.



Related Article.

Expert Talk: “I Want to Hear What This Person Has to Say Now”--Motoyuki Oi

A Talk with an Expert: “Now, I Want to Hear What This Person Has to Say” -- Shoichi Furukawa