A Haptic Future: Sensing the Virtual

What if you could touch and feel the Metaverse? Could smell also be replicated in virtual spaces? Haptics are the technologies we use to connect our bodies to our digital devices. The vibration mode on mobile phones is an example of haptic technology. Or the 4D experiences cinemas offer. Some advanced gaming peripherals allow participants to feel bullet hits on their bodies. And the adult entertainment industry has been experimenting with haptics for decades.

Various technologies have been developed to create haptic feedback for VR users. Gloves, through to full bodysuits, have been created with varying degrees of success. The technologies are designed to replicate touch and feel in a Metaverse space. Also, telepresence is a factor where remote action (such as surgery) can occur at a distance.

Alex Book, Co-Founder and Chief Strategy Officer of ARCADE explained: “The principle of ‘embodiment’ is well understood in VR to mean the genuine sense of being physically present within the virtual environment. Haptics play a fundamental role in this, providing additional sensory input that helps build a picture for the mind and body, convincing us that we really are interacting with the digital space around us.”

Book concluded: “When we switch to mixed reality (MR) or augmented reality (AR), we need less persuasion that we are present – we can see clearly that we are – but multi-sensory input is still critical when engaging with spatial content, making it entirely believable that we really are picking up that block, or painting with that brush, or pressing that button. With believability comes comfort and familiarity, both essential for the mass adoption of spatial experience.”

Touchy, feely

To gain an insight into how haptics has developed and the business case for their use, Silicon UK spoke with Donald Butts, Senior Director of Technology, InterDigital.

What are the current capabilities of haptic VR technology, and how do you envision them evolving in the near future?

“Haptic VR technology is currently primarily centred around vibrotactile feedback, predominantly provided by handheld controllers. This feedback gives users a touch experience when interacting with virtual environments. Yet, the experience is relatively coarse, often limited to vibrations, and lacks the depth to replicate the nuances of texture, temperature, and other elements that make up real-world tactile experiences.

“That being said, we are starting to see haptic technology integrated into a wider array of devices, such as haptic seats, which create more immersive experiences, particularly within gaming and entertainment environments. These devices can deliver a spectrum of physical sensations synchronized with VR content, such as vibrations and movements, substantially enhancing the user’s sense of presence in a virtual world.

“Looking to the future, I envision several key advancements shaping the evolution of haptic VR technology. Innovations in materials science could lead to the development of sophisticated haptic devices capable of conveying more nuanced tactile feedback. Emerging technologies like mid-air haptics, which utilize ultrasonic waves to create touch sensations in the air, could offer enhanced immersive experiences. This could be particularly beneficial for VR systems without traditional handheld controllers, such as the Apple VisionPro VR headset.

“Furthermore, there are promising concepts around self-haptics. By using the user’s own body for physical feedback, it removes the need for extra hardware and paves the way for a more natural, intuitive haptic experience. These developments will push the boundaries of what’s possible in haptic VR technology, drawing us ever closer to truly immersive and realistic virtual experiences.”

Haptic feedback has shown promise in enhancing VR experiences. What key advancements are expected to take haptic feedback to the next level?

“At the moment, haptic feedback systems are mainly limited to vibrotactile sensations. However, key advancements driven by industry standards and best practices can lead to more immersive and realistic haptic experiences.

“Advancements in material science and wearable technology will allow the development of more responsive haptic devices like full-body suits or advanced gloves. For these innovations to truly enhance VR experiences across various platforms, adherence to established standards and protocols will be paramount.

“One key advancement will be in the field of self-haptics. Progress in this area will require the development of sophisticated algorithms for real-time manipulation of users’ hand positions and poses, incorporating AI and machine learning to make the haptic feedback more personalized and realistic.”

How will the development of more sophisticated haptic devices impact the overall realism and immersion in virtual environments?

“The emergence of more sophisticated haptic devices certainly has the potential to significantly enhance realism and immersion in virtual environments. Standardizing these advanced haptic devices is critical to ensure their broad adaptability, making immersive VR experiences universally accessible across different platforms.

“However, it’s essential to recognize that the Quality of Personal Experience (QoPE) isn’t solely determined by the haptic devices themselves, but by an end-to-end (E2E) solution encompassing robust wireless network performance, sophisticated sensing technology, and a human-in-the-loop system.

“As we venture into more immersive, mobile VR environments, the requirements for high-speed, low-latency communication become more stringent. The tactile internet, aiming to transmit haptic feedback in real-time, requires a latency not exceeding one millisecond to align with the human neurological system’s capacity to relay touch sensations to the brain. Any deviation from this can lead to undesirable effects such as cyber-sickness or nausea in users.

“Standards in wireless networking, such as those promised by emerging technologies like 5G and beyond, are paramount in ensuring the reliable and real-time transmission of haptic and sensory data. Alongside this, future networks will need to incorporate a human-in-the-loop system for QoPE. Currently, sensors alone can’t quantify a user’s experience, but as we move towards more immersive applications, the human user will need to be integrated into the system. This approach allows the system to manage the user’s experience effectively, offering a personalized quality of experience and facilitating graceful degradation when necessary.”

Are there any specific challenges or limitations that haptic VR technology currently faces, and how are researchers and developers working to overcome them?

“Haptic VR technology currently faces several challenges. One major limitation is the difficulty in accurately simulating a wide range of tactile sensations. However, through industry collaboration and standard-setting, this challenge can be tackled by working towards universally accepted definitions and parameters for haptic feedback.

“Another challenge is the physical constraints of the hardware. Many haptic devices are bulky and restrict users’ movements. This not only hinders immersion but also interoperability between different devices. Lightweight and mobile-friendly VR experiences are preferred, but they often offer no means for haptic feedback.”

The addition of tactile feedback in a virtual space clearly has applications in healthcare, for example. Gaming will continue to develop this technology to improve the realism that gamers experience. This industry will likely create the first affordable haptic technologies that all businesses could deploy. As with all technologies, having a clearly defined business case for these devices is vital. The Metaverse won’t be just a visual experience for long.