Intensifying emotional reactions via tactile gestures in immersive films

Georgios Karafotias, Akiko Teranishi, Georgios Korres, Friederike Eyssel, Scandar Cobti, Mohamad Eid

Research output: Contribution to journalArticle

Abstract

The film industry continuously strives to make visitors' movie experience more immersive and thus, more captivating. This is realized through larger screens, sophisticated speaker systems, and high quality 2D and 3D content. Moreover, a recent trend in the film industry is to incorporate multiple interaction modalities, such as 4D film, to simulate rain, wind, vibration, and heat, in order to intensify viewers' emotional reactions. In this context, humans' sense of touch possesses significant potential for intensifying emotional reactions for the film experience beyond audio-visual sensory modalities. This article presents a framework for authoring tactile cues (tactile gestures as used in this article) and enabling automatic rendering of said gestures to intensify emotional reactions in an immersive film experience. To validate the proposed framework, we conducted an experimental study where tactile gestures are designed and evaluated for the ability to intensify four emotional reactions: high valence-high arousal, high valence-low arousal, low valence-high arousal, and low valence-low arousal. Using a haptic jacket, participants felt tactile gestures that are synchronized with the audio-visual contents of a film. Results demonstrated that (1) any tactile feedback generated a positive user experience; (2) the tactile feedback intensifies emotional reactions when the audio-visual stimuli elicit clear emotional responses, except for low arousal emotional response since tactile gestures seem to always generate excitement; (3) purposed tactile gestures do not seem to significantly outperform randomized tactile gesture for intensifying specific emotional reactions; and (4) using a haptic jacket is not distracting for the users.

Original languageEnglish (US)
Article number29
JournalACM Transactions on Multimedia Computing, Communications and Applications
Volume13
Issue number3
DOIs
StatePublished - Jun 1 2017

Fingerprint

Feedback
Rain
Industry
Hot Temperature

Keywords

  • Affective haptics
  • Immersive virtual reality
  • Multimodal interaction
  • Tactile gestures

ASJC Scopus subject areas

  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

Intensifying emotional reactions via tactile gestures in immersive films. / Karafotias, Georgios; Teranishi, Akiko; Korres, Georgios; Eyssel, Friederike; Cobti, Scandar; Eid, Mohamad.

In: ACM Transactions on Multimedia Computing, Communications and Applications, Vol. 13, No. 3, 29, 01.06.2017.

Research output: Contribution to journalArticle

@article{3b7447f5f7eb4a618bd99580a40df00a,
title = "Intensifying emotional reactions via tactile gestures in immersive films",
abstract = "The film industry continuously strives to make visitors' movie experience more immersive and thus, more captivating. This is realized through larger screens, sophisticated speaker systems, and high quality 2D and 3D content. Moreover, a recent trend in the film industry is to incorporate multiple interaction modalities, such as 4D film, to simulate rain, wind, vibration, and heat, in order to intensify viewers' emotional reactions. In this context, humans' sense of touch possesses significant potential for intensifying emotional reactions for the film experience beyond audio-visual sensory modalities. This article presents a framework for authoring tactile cues (tactile gestures as used in this article) and enabling automatic rendering of said gestures to intensify emotional reactions in an immersive film experience. To validate the proposed framework, we conducted an experimental study where tactile gestures are designed and evaluated for the ability to intensify four emotional reactions: high valence-high arousal, high valence-low arousal, low valence-high arousal, and low valence-low arousal. Using a haptic jacket, participants felt tactile gestures that are synchronized with the audio-visual contents of a film. Results demonstrated that (1) any tactile feedback generated a positive user experience; (2) the tactile feedback intensifies emotional reactions when the audio-visual stimuli elicit clear emotional responses, except for low arousal emotional response since tactile gestures seem to always generate excitement; (3) purposed tactile gestures do not seem to significantly outperform randomized tactile gesture for intensifying specific emotional reactions; and (4) using a haptic jacket is not distracting for the users.",
keywords = "Affective haptics, Immersive virtual reality, Multimodal interaction, Tactile gestures",
author = "Georgios Karafotias and Akiko Teranishi and Georgios Korres and Friederike Eyssel and Scandar Cobti and Mohamad Eid",
year = "2017",
month = "6",
day = "1",
doi = "10.1145/3092840",
language = "English (US)",
volume = "13",
journal = "ACM Transactions on Multimedia Computing, Communications and Applications",
issn = "1551-6857",
publisher = "Association for Computing Machinery (ACM)",
number = "3",

}

TY - JOUR

T1 - Intensifying emotional reactions via tactile gestures in immersive films

AU - Karafotias, Georgios

AU - Teranishi, Akiko

AU - Korres, Georgios

AU - Eyssel, Friederike

AU - Cobti, Scandar

AU - Eid, Mohamad

PY - 2017/6/1

Y1 - 2017/6/1

N2 - The film industry continuously strives to make visitors' movie experience more immersive and thus, more captivating. This is realized through larger screens, sophisticated speaker systems, and high quality 2D and 3D content. Moreover, a recent trend in the film industry is to incorporate multiple interaction modalities, such as 4D film, to simulate rain, wind, vibration, and heat, in order to intensify viewers' emotional reactions. In this context, humans' sense of touch possesses significant potential for intensifying emotional reactions for the film experience beyond audio-visual sensory modalities. This article presents a framework for authoring tactile cues (tactile gestures as used in this article) and enabling automatic rendering of said gestures to intensify emotional reactions in an immersive film experience. To validate the proposed framework, we conducted an experimental study where tactile gestures are designed and evaluated for the ability to intensify four emotional reactions: high valence-high arousal, high valence-low arousal, low valence-high arousal, and low valence-low arousal. Using a haptic jacket, participants felt tactile gestures that are synchronized with the audio-visual contents of a film. Results demonstrated that (1) any tactile feedback generated a positive user experience; (2) the tactile feedback intensifies emotional reactions when the audio-visual stimuli elicit clear emotional responses, except for low arousal emotional response since tactile gestures seem to always generate excitement; (3) purposed tactile gestures do not seem to significantly outperform randomized tactile gesture for intensifying specific emotional reactions; and (4) using a haptic jacket is not distracting for the users.

AB - The film industry continuously strives to make visitors' movie experience more immersive and thus, more captivating. This is realized through larger screens, sophisticated speaker systems, and high quality 2D and 3D content. Moreover, a recent trend in the film industry is to incorporate multiple interaction modalities, such as 4D film, to simulate rain, wind, vibration, and heat, in order to intensify viewers' emotional reactions. In this context, humans' sense of touch possesses significant potential for intensifying emotional reactions for the film experience beyond audio-visual sensory modalities. This article presents a framework for authoring tactile cues (tactile gestures as used in this article) and enabling automatic rendering of said gestures to intensify emotional reactions in an immersive film experience. To validate the proposed framework, we conducted an experimental study where tactile gestures are designed and evaluated for the ability to intensify four emotional reactions: high valence-high arousal, high valence-low arousal, low valence-high arousal, and low valence-low arousal. Using a haptic jacket, participants felt tactile gestures that are synchronized with the audio-visual contents of a film. Results demonstrated that (1) any tactile feedback generated a positive user experience; (2) the tactile feedback intensifies emotional reactions when the audio-visual stimuli elicit clear emotional responses, except for low arousal emotional response since tactile gestures seem to always generate excitement; (3) purposed tactile gestures do not seem to significantly outperform randomized tactile gesture for intensifying specific emotional reactions; and (4) using a haptic jacket is not distracting for the users.

KW - Affective haptics

KW - Immersive virtual reality

KW - Multimodal interaction

KW - Tactile gestures

UR - http://www.scopus.com/inward/record.url?scp=85022320965&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85022320965&partnerID=8YFLogxK

U2 - 10.1145/3092840

DO - 10.1145/3092840

M3 - Article

AN - SCOPUS:85022320965

VL - 13

JO - ACM Transactions on Multimedia Computing, Communications and Applications

JF - ACM Transactions on Multimedia Computing, Communications and Applications

SN - 1551-6857

IS - 3

M1 - 29

ER -