What is "body sharing" technology that lets you experience the world from the comfort of your own home?

Game graphics are becoming more and more realistic. The PlayStation 5 Pro, which was announced the other day, has machine power that far surpasses previous models, taking the home gaming experience to the next level. In fact, the PlayStation 5 has a function called haptic feedback built into the controller.

When the character you control in the game runs on sand, you feel the sensation of touching the sand with your hands, and when you grip the grip of a gun, you feel the weight of really pulling the trigger. The impact of combining these systems with a head-mounted display is tremendous.

What would happen if you could feel this excitement as if you were one with your avatar in the metaverse? Or what if you could experience the atmosphere of participating in a distant relatives' gathering that you couldn't attend?

Body sharing expands your online world experience to the fullest.

You've probably heard that the human body is powered by electrical signals. Humans have five senses: vision, hearing, smell, taste, and touch. Many people understand that when these organs receive external stimuli, they switch on...

In fact, recent research has shown that humans have more than just a few senses. Stimuli received by the sensory organs are converted into electrical signals and transmitted to the brain through the nerves.

Therefore, if electrical stimuli similar to these electrical signals are given to the human body, in theory, the human will have the illusion that it does not exist in that place, and this sense should be able to expand endlessly.

H2L Inc. is a company that is researching and developing a system to expand personal experiences in this way. H2L has succeeded in digitizing human movements and sensations using its unique muscle displacement sensor.

They are then trying to transmit this data to virtual avatars, robots, or third parties. They call this technology "body sharing."

▶Body sharing technology allows you to recreate and feel the impact and weight of a bird that has landed on your hand in a virtual space.

Actually, I heard that body sharing has already been realized as a variety of services without our knowledge. So I interviewed H2L to find out about the actual development status and what kind of usage they envision.

What kind of usage scenarios did the research on “body sharing” envision in the first place? “It started from the personal desire of our company’s representative, Tamaki,” said Sato, who corresponded with us in the interview, and gave us the details. H2L’s representative director is Tamaki Emi, who also serves as a professor at the Faculty of Engineering at the University of the Ryukyus.

When Tamaki was a teenager, she had been forced to repeatedly go in and out of the hospital due to a chronic illness. At that time, she realized that she could do most things in her room, and she thought that hospital life was quite good.

So Tamaki wondered if she could experience the outdoors while in her hospital room. However, not only could she not find such a service, but there was almost no basic research on it. Tamaki thought that if it didn't exist, he would make it himself, and started researching how to experience traveling indoors.

Tamaki's focus was on "proprioception." Proprioception is the senses received inside the body, such as muscles, tendons, and joints, deeper than the skin that senses antennae, and is also called "deep sense."

Specifically, it refers to senses related to the position and movement of the body, and the amount of force applied, such as the sensation of the weight of an apple when holding it in your hand (= weight sense) and the sensation that the presence of an apple inhibits your grip (= resistance sense).

If this sensation can be transmitted electrically, you can experience the sensation of holding an apple even if there is no apple there. If this is combined with a head-mounted display, you can experience the sensation of experiencing outdoors while indoors.

This research into controlling proprioception has borne fruit, and the system developed by H2L has already begun to be put into practical use.

RaraaS allows people who can no longer move their bodies as they wish due to a disability or illness to share their body with a robotic arm in a remote location and harvest crops in a realistic manner by transmitting the movements of their hands and arms.

The head-mounted display shows the scenery of Okinawa and the participant can hear the sound of the water, so adding the resistance of the oars to that creates a very realistic kayaking experience in the middle of a city or in a hospital room. This technology is not only useful for tourism, but is also expected to be useful in the welfare field, as it allows people who have difficulty working outside to experience going outside.

A "remote-controlled kayak robot" that can remotely control a kayak 700 km away by rowing from a bed while feeling the resistance of the water.

Systems that seem unbelievable, such as a system that applies electrical stimulation to the fingers of patients with paralysis due to spinal cord injuries and other nerve disorders, for use in rehabilitation, are already close to being commercialized.

The biggest challenge is. "When sharing experiences in real time, the amount of communication is enormous. Since we cannot solve this problem on our own, we are collaborating with NTT Docomo to develop technology for 5G and 6G.

Currently, we can share our senses without any discomfort, but with 6G, data communication will be possible with a delay of 1/1000th of a second, which exceeds the human reaction speed, so we will be able to share our senses over long distances more comfortably," says Sato. This means that high-resolution display devices will be required next.

"That's right. We are currently developing using existing head-mounted displays, but I think that in the future, more high-performance devices will be required. We are working on improving proprioception input/output devices so that we can keep up with that.

The muscle displacement sensor device "FirstVR" currently on sale is a wristwatch type. Since it can only input data from the part that is wrapped around the device, we are exploring the shape of the device that can obtain data from all parts of the body."

It is true that if we can accurately share our senses not only from the fingertips and arms but also from large muscles such as the waist and back muscles, we can have a more immersive experience. By the way, H2L's body sharing technology is already in the verification stage, but how long will it be before we can actually see it and experience it normally?

"We are promoting research and development and industrial introduction with the goal of achieving 'triple the amount of experiences in 2020 by 2029.

From 2023, we have been commissioned to carry out the national project 'Strategic Innovation Promotion Program (SIP) 3rd Phase / Development of basic technologies and rules for expanding the virtual economy' implemented by the Cabinet Office, and are conducting research and development to utilize body sharing in the 'Interverse' as a place connecting physical space and cyberspace as 'research and application of proprioceptive data input/output technology for the purpose of experience sharing.

We aim to build an industrial structure that promotes the practical application of experience sharing between cyberspace and physical space by sharing proprioception.

It seems that sharing of sensations through virtual worlds and the Internet will be realized much sooner than we thought.

Column

"Remotely controlled kayak robot"

Remote tourism driving a kayak in Okinawa from 700km away

Wearing a head-mounted display and rowing from a bench may seem ridiculous at first glance, but the participant's arms share the same water pressure as if they were actually rowing. The head-mounted display shows the scenery of Okinawa and the participant can hear the sound of the water, so by adding the resistance of the oars to this, a very realistic kayaking experience can be experienced in the middle of a city or in a hospital room. This technology is not only useful for tourism, but is also expected to be useful in the welfare field, as it allows people who have difficulty working outside to experience going outside.

FirstVR

You can also share and experience professional fingering

Using body sharing technology, it will become possible to experience the physical movements of professional musicians and athletes that cannot be acquired overnight. It is an interesting use in the sense that it will allow the body to memorize professional techniques, but it is also expected to be useful in rehabilitation settings, such as helping people who have lost the ability to move their fingers due to paralysis to remember how to move them. There have been cases where paralysis has been cured by a slight stimulus, so body sharing, which can generate artificial stimuli, has attracted attention.

Related articles

The oft-discussed topic of "AI employees" - will they replace humans?

From national project to private business: The era of space travel is finally upon us