A gesture recogintion architecture for Arabic sign language communication system

Tamás Aujeszky, Mohamad Eid

Research output: Contribution to journalArticle

Abstract

Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 %. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.

Original languageEnglish (US)
Pages (from-to)8493-8511
Number of pages19
JournalMultimedia Tools and Applications
Volume75
Issue number14
DOIs
StatePublished - Jul 1 2016

Fingerprint

Communication systems
Audition
Multimedia systems
Communication
Animation
Feedback
Costs

Keywords

  • Avatar animation
  • Gesture recognition
  • Multimedia systems and tools
  • Sign language communication systems
  • Usability study

ASJC Scopus subject areas

  • Software
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

A gesture recogintion architecture for Arabic sign language communication system. / Aujeszky, Tamás; Eid, Mohamad.

In: Multimedia Tools and Applications, Vol. 75, No. 14, 01.07.2016, p. 8493-8511.

Research output: Contribution to journalArticle

@article{579f9fc7a24a4aa6903480a42beca7e3,
title = "A gesture recogintion architecture for Arabic sign language communication system",
abstract = "Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 {\%}. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.",
keywords = "Avatar animation, Gesture recognition, Multimedia systems and tools, Sign language communication systems, Usability study",
author = "Tam{\'a}s Aujeszky and Mohamad Eid",
year = "2016",
month = "7",
day = "1",
doi = "10.1007/s11042-015-2767-2",
language = "English (US)",
volume = "75",
pages = "8493--8511",
journal = "Multimedia Tools and Applications",
issn = "1380-7501",
publisher = "Springer Netherlands",
number = "14",

}

TY - JOUR

T1 - A gesture recogintion architecture for Arabic sign language communication system

AU - Aujeszky, Tamás

AU - Eid, Mohamad

PY - 2016/7/1

Y1 - 2016/7/1

N2 - Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 %. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.

AB - Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 %. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.

KW - Avatar animation

KW - Gesture recognition

KW - Multimedia systems and tools

KW - Sign language communication systems

KW - Usability study

UR - http://www.scopus.com/inward/record.url?scp=84985031067&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84985031067&partnerID=8YFLogxK

U2 - 10.1007/s11042-015-2767-2

DO - 10.1007/s11042-015-2767-2

M3 - Article

AN - SCOPUS:84985031067

VL - 75

SP - 8493

EP - 8511

JO - Multimedia Tools and Applications

JF - Multimedia Tools and Applications

SN - 1380-7501

IS - 14

ER -