By Manu Sainsily, Sr. UI/UX Designer Specialized in XR <br/>
Of the five main senses, touch is the most proficient – ‘it’s the only one capable of simultaneous input and output – which means that it is the best sense for communication.
One of my colleagues at Immersion, Felipe Almeida, recently tackled the question, “What is haptics?” (You can read his article here.) He takes a thoughtful scientific approach to define haptics as the “science of mediated touch.”
Going beyond the definition, we know that touch has a broad role in our everyday lives.
Humans are social animals. We are driven by our emotions, and our emotions are connected to our feelings. Touch has always played an essential role in how we communicate. It is a language that allows us to interact with the world from the minute we are born.
While the world continues to become digitized with new technology that allows us to connect with others across vast distances, the absence of touch and presence have become ever more apparent. In this current environment of COVID 19, we are keeping each other safe by being apart from one another; and yet, at the same time we know that this behavior is unsustainable because we need each other’s presence and touch to feel connected.
If we take a look back, our ancestors used painting to save and share their stories visually. While this form of visual communication was asynchronous, it was effective. Indeed, years later, when a member from another tribe would visit the cavern and witness the art left behind, it would still tell the stories of its originator; then and even now, thousands of years in the future.
As technology progresses, asynchronous communication continues to factor in the preservation of our stories and how we share experiences. We see this in the form of vinyl, radio, the telephone, and podcasts for audio; the internet, movies, YouTube for both audio and visual. Even today new tools are being invented to connect and share asynchronously: communications channels such as text messaging, Slack, Teams, or Duos where you can leave a message for someone else to read, regardless of time or place.
Haptics, the science of mediated touch, enables touch to be a part of the digital technology experience that is pervasive in today’s world – often making the asynchronous feel like real-time. The recent implementations of more advanced haptics in gaming will inspire developers to explore how to use touch as a part of their stories. This, hopefully, is a precursor to enabling everyone to connect using touch to communicate with each other (as we have always done since we were babies) – but digitally and often asynchronously. This is important because all these mediums of communication create a wholly different experience when put together.
In itself, haptics is a meaningful form of communication. In the tech industry, and especially in gaming, haptics allows humans to interact with computers initially through analog proxies such as joysticks, buttons, mice, and keyboards, and later on through skeuomorphic digital interfaces, using gestures like swipe, scroll, or double-tap. It is also known that combining multiple types of feedback (auditory, visual, haptics) increases the sense of immersion. Understanding this layering is key to unlocking the future of XR technologies in augmented, mixed, and virtual reality. In these applications, multi-sensorial experiences come into play. As a result, mixing different feedback to create an immersive experience is less costly than amplifying a single sense (audio or visual) to make up for a missing sense. No matter how much you spend on increasing the quality of the graphics, you won’t be able to make up for the missing audio. Increasing audio quality alone does not replace haptics, and so on. However, adding some haptics to a standard audiovisual experience moves the user much closer to the feeling of synchronicity and immersion.
We must even go beyond the current implementations of haptic technology and allow for a multimodal touch experience that not only uses vibrations but also temperature, pressure, and other types of tactile modalities. This is the promise of haptics.
In the past, people have failed to recognize that haptics is a valuable and almost indispensable technology. The new haptic-enabled mainstream accessories such as the PlayStation 5 DualSense controller will usher in a new understanding of the role that haptics play in the user experience. The broad haptic capabilities in the PlayStation 5 will enable developers, designers, and creatives to use a new medium in which they can create immersive experiences and take the time to design and implement meaningful haptic experiences.
We call the people that play with this new design medium “hapticians,” a combination of haptics and magicians. According to Oliver Schneider, Assistant Professor at the University of Waterloo, a haptician is one who is skilled at making haptic sensations, technology, or experiences. They engage in haptic experience design (HaXD), the process of designing for touch – a creative capability that is extremely challenging and poorly understood. HaXD is the design-planning, development, and evaluation of user experiences, deliberately connecting interactive technology to one or more perceived senses of touch, possibly as part of a multi-sensory experience.
It is our duty as hapticians to make gaming and interaction with digital devices accessible, fun, and meaningful again. As a fellow haptician, I feel like the world finally has their eyes on us, and we can make a true difference. Creators, advertisers, and even enterprise platforms will start to focus on the sense of immersion. From mobile to gaming to automotive, all industries will be affected and will need to continue to adapt to this digital transformation.
“The last best experience that anyone has anywhere becomes the minimum expectation for the experience they want everywhere,” said IBM’s GBS leader Bridget van Kranlingen a few years back. Gaming engines are already being used to develop the future of user experiences in many fields such as film making, enterprise training, work field, and maintenance, etc. We will naturally continue to see a merge between the Metaverse and our reality; for example, as we observe social media adopting more and more AR/MR features (Snapchat and Instagram lenses, etc.), and video games and smart devices integrating XR-capable hardware (LiDAR camera on latest iPhones, Qualcomm new Snapdragon XR2 chip, etc.).
To this day, there are still many challenges to address in haptics (ethics, accessibility, etc.), and we have a unique window of opportunity to add our stone to the edifice while it is being defined, built, and adopted by the mainstream. Companies responsible for these tools and engines are finally listening and ready to put in the work to allow creative storytellers to express themselves through these new mediums. The community is alive and bubbling, inviting both fresh and seasoned minds to unite and bring their unique flavors to the table.
While well-known tech companies have benefited from the value of haptics, only a few have put it at the forefront of their ecosystem. This is changing; soon haptics will be everywhere. Not only in your smartphones, computers, and gaming consoles, but also inside your car, IoT devices, and wearables – in a way that is very different from the standard environment. Combining haptic experiences with the power of technologies such as XR, 5G, Cloud, and AI will unlock new possibilities and “superpowers”: spatial awareness augmented world and passive haptics, 3D audio in sync with vibrations, etc.
We have the fate of future human-computer interactions at our fingertips, and there has never been a better time to be a haptician!