A gesture recogintion architecture for Arabic sign language communication system

Tamás Aujeszky, Mohamad Eid

    Research output: Contribution to journalArticle

    Abstract

    Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 %. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.

    Original languageEnglish (US)
    Pages (from-to)8493-8511
    Number of pages19
    JournalMultimedia Tools and Applications
    Volume75
    Issue number14
    DOIs
    StatePublished - Jul 1 2016

    Fingerprint

    Communication systems
    Audition
    Multimedia systems
    Communication
    Animation
    Feedback
    Costs

    Keywords

    • Avatar animation
    • Gesture recognition
    • Multimedia systems and tools
    • Sign language communication systems
    • Usability study

    ASJC Scopus subject areas

    • Software
    • Media Technology
    • Hardware and Architecture
    • Computer Networks and Communications

    Cite this

    A gesture recogintion architecture for Arabic sign language communication system. / Aujeszky, Tamás; Eid, Mohamad.

    In: Multimedia Tools and Applications, Vol. 75, No. 14, 01.07.2016, p. 8493-8511.

    Research output: Contribution to journalArticle

    Aujeszky, Tamás ; Eid, Mohamad. / A gesture recogintion architecture for Arabic sign language communication system. In: Multimedia Tools and Applications. 2016 ; Vol. 75, No. 14. pp. 8493-8511.
    @article{579f9fc7a24a4aa6903480a42beca7e3,
    title = "A gesture recogintion architecture for Arabic sign language communication system",
    abstract = "Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 {\%}. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.",
    keywords = "Avatar animation, Gesture recognition, Multimedia systems and tools, Sign language communication systems, Usability study",
    author = "Tam{\'a}s Aujeszky and Mohamad Eid",
    year = "2016",
    month = "7",
    day = "1",
    doi = "10.1007/s11042-015-2767-2",
    language = "English (US)",
    volume = "75",
    pages = "8493--8511",
    journal = "Multimedia Tools and Applications",
    issn = "1380-7501",
    publisher = "Springer Netherlands",
    number = "14",

    }

    TY - JOUR

    T1 - A gesture recogintion architecture for Arabic sign language communication system

    AU - Aujeszky, Tamás

    AU - Eid, Mohamad

    PY - 2016/7/1

    Y1 - 2016/7/1

    N2 - Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 %. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.

    AB - Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 %. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.

    KW - Avatar animation

    KW - Gesture recognition

    KW - Multimedia systems and tools

    KW - Sign language communication systems

    KW - Usability study

    UR - http://www.scopus.com/inward/record.url?scp=84985031067&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84985031067&partnerID=8YFLogxK

    U2 - 10.1007/s11042-015-2767-2

    DO - 10.1007/s11042-015-2767-2

    M3 - Article

    VL - 75

    SP - 8493

    EP - 8511

    JO - Multimedia Tools and Applications

    JF - Multimedia Tools and Applications

    SN - 1380-7501

    IS - 14

    ER -