Search:

TolmacsKesztyu

Full name: InterpreterGlove
Department: Department of Distributed Systems
Start date: 2012. 09. 01.
End date: 2014. 08. 31.

Participants

Description

The goal of the TolmácsKesztyű (InterpreterGlove) project is to create such an innovative intelligent tool that can be used by speech- and hearing impaired people to get in touch with their nondisabled fellows along their quotidian life in a way that is intuitive for both parties. The tool - the glove - that operates as a simultaneous translator helps the disabled to communicate with the nondisabled using their native language - the sign language. As the interpreterglove reads out the signed text loud, the hearing-abled can fluently communicate with the disabled without any sign language knowledge.

Deaf-mute people's communication potential will be highly expanded by this device. Members of this disability group currently either need to go through cumbersome explanations to describe their intent or they need to be assisted by a sign language translator. Using the translatorglove they are enabled to express themselves to non sign language speakers, that is to say this device will significantly advance the social integration of the disabled. Users of the device are enabled to handle situations that formerly were prevented by their disability. The translatorglove opens up possibilities like shopping, personal banking, taking part in classical education and beyond.

In the course of the project we develop a complex integrated hardware and software system. Using the one handed prototype glove users will be able to show Dactil characters and custom gestures that are mapped to user specified expressions. The glove will be made of leather and contain all the sensors, electronic parts and communication module in a seamless, ergonomic way. The glove connects to the user's cell phone that is responsible to perform complex tasks and computation like transforming the gesture stream to text by natural language processing and reading out the result. The software will be available on all major mobile platforms, namely iOS, Android and Windows Phone.