Linguistic study of sign languages
is quite new compared to spoken languages, having begun only
in the 1960s. Linguists are very interested in sign languages
because of what they can reveal about the possibilities of
human language that don't rely at all on sound. One of the
problems is that studying sign languages involves analysing
video footage - and because sign languages lack any standard
writing or transcription system, this is extremely
labour-intensive. This project will develop computer vision
tools to assist with video analysis. This will in turn help
linguists increase their knowledge of the language with a long
term ambition of creating the world's first machine readable
dataset of a sign language, a goal that was achieved for large
amounts of text of spoken language in the 1970s.
The ultimate goal of this project is to take the annotated
data and understanding from linguistic study and to use this
to build a system that is capable of watching a human signing
and turning this into written English. This will be a world
first and an important landmark for deaf-hearing
communication. To achieve this the computer must be able to
recognise not only hand motion and shape but the facial
expression and body posture of the signer. It must also
understanding how these aspects are put together into phrases
and how these can be translated into written/spoken language.
Although there have been some recent advances in sign language
recognition via data gloves and motion capture systems like
Kinect, part of the problem is that most computer scientists
in this research area do not have the required in-depth
knowledge of sign language. This project is therefore a
strategic collaboration between leading experts in British
Sign Language linguistics and software engineers who
specialise in computer vision and machine learning, with the
aim of building the world's first British Sign Language to
English Translation system and the first practically
functional machine translation system for any sign language.
Robert
Adam (British Deaf Association)
Onno Crasborn (Radboud University Nijmegen)
Sarah Ebling (Interkantonale Hochschule für
Heilpadago)
Thomas Hanke (University of Hamburg)
Andrew McParland (BBC)
Mark Wheatley (European Union of the Deaf),