Review Paper on Sign Language Recognition Using Deep Learning - IT Assignment Help

Assignment Task:

Task:

Review Paper on Sign language recognition using deep learning
1 Miss. Maithily Marathe, 2 Prof. Madhuri Bhalekar1 PG Scholar, 2 Assistant Professor
1 Computer engineering, 2Computer engineering
1 Dr.vishwanath karad MIT world Peace university,Pune, India

Abstract:
Generally human beings communicate with each other to transmit their thoughts to each other. But sometimes we often seen in society that some of the people can’t hear or speak, these types of peoples are known as deaf or dumb peoples. These dumb peoples also have their thoughts; they also want to communicate with the society. So dumb people use their own language of communication this language is known as “Sign Language”.
The sign language is just a collection of human gesture and the gestures show a movement of body parts that convey some meaningful message. As sign language is not understandable to everyone a translator is required to make communication between dumb people and normal people. Eventually these translators are not available every time, so what type of gestures is made by dumb people is not recognized by common people. Gestures are most suitable and natural to have interaction or communication with the system (Computer). So it is suitable to build platform between dumb people and normal human beings. In this research work, we initiate to discover a platform for gesture or hand movement recognition by using convolution neural network algorithm, by using this technique we can identify human gesture in image. This technique captures images and sensors works for converting the signs into the text. The dataset are set using the meaning of signs at that run time it gets converted by using algorithms with the help of neural network and network model. This converted text is readable for everyone.
Keywords: - American Sign Language, classification, convex hull, convolutional neural network, deep learning, real-time.
I. INTRODUCTION
Mostly people with disability or differently able are often isolated from accessing proper health, education and other social interaction. Durable technology can change the life of peoples who are suffering for disability to a great extent in all means. Deafness and vocal impairment are one of the major disabilities faced by human beings from centuries. This problem hinders a person to communicate in verbal languages to the outside world leading to isolation from the rest of the major verbally communicating society. They use sign language to communicate with people but it is limited within them or near relatives. To communicate with society they require a manual translator. A sign language converts natural language into hand gestures along with facial expression and eyebrow movements. Sign language is not universal; it changes from region to region. As sign language is not understandable to everyone a translator is required to make communication between dumb people and normal people. Eventually these translators are not available every time. To address these we use to models to recognize gesture in sign language. We use the custom sign language dataset for video data for training the model for recognize gesture. The dataset has multiple gestures that have been performed multiple times giving us variation in context and video condition. For clarity, videos are getting recorded at common frame rate. We propose to use Convolutional Neural Network model to extract structural features from the video stream for Sign Language Recognition. Then by implementing Long-Short-Term-Memory the Recurrent Neural Network we can extract temporal features from video sequences via two methods: using the output from Softmax and pool layer of the CNN respectively.
In this video action recognition, every hand movement has different meaningful textual form. This method is slightly different from still images, because human action continuously ever changing motions or collection of moving elements. So that is important to find which gesture is appropriate for which meaningful text. In this way, it’s crucial to investigate differing spatial-temporal highlights for activity acknowledgment. Using deep learning we make use of motion information in various ways. Neural networks can identify automatically learning features from large amounts of dataset. Using different algorithms of CNN can handle each frame’s information is correct and appropriately result. The main objective of this research is to make a system for deaf or dumb people’s hand gesture recognizer, which recognizes the gesture and converted into proper meaningful text. A complete recognition system must be able to identify alphabets, numerals, static and dynamic words, context, emotions, facial expressions, eyebrow movement, body posture and numerous other situations. Successful recognition can only be achieved by exploring computer vision, machine learning, human computer combination and deaf society. The recent trend is increased use of machine learning where a machine is trained with a number of data sets.
II. HISTORY
Up to now lot of research work has been dedicated to recognition of hand gestures. These can be broadly divided into glove-based and sensor-based methods and appearance-based or vision-based methods. Glove or sensor based approaches provide good accuracy as these specific devices collect data or features directly from signer but, it will have an overhead of carrying and signing with the external device. Vision-based methods, on the other hand, do the recognition task from images or videos based on the features calculated using various image or video processing techniques. This can be again categorized into 2-D Vision based techniques and 3-D Vision based techniques.
Shweta S. Shinde, Vitthal K. Bhosale and Rajesh M. Autee introduced a system for two-way interaction technique, which is used for dumb and deaf person. In that system, they used artificial intelligence (AI), data mining, and image processing for taking input as a hand gesture and produce output as a voice.
In recent years the CNN (convolutional neural network) had great success in computer vision tasks including image recognition. Starting with Alex Net which popularized deep convolutional neural networks by winning the Image Net challenge, also made significant contributions and improved performance. This work aims to utilizes the advance in deep learning techniques and apply them to create a robust and real time finger spelled sign language classifier
In more recent work of new mode architecture Dense Convolutional Network (Dense Net) was proposed. Dense Net presented a way to solve the vanishing gradient problem of deep networks by connecting each layer to every other layer in a feed-forword fashion. Dense net exploits the potential of the network through feature reuse which results to less number of parameters. With these advantages network model was based on Dense Net.
Sole et al. used Extreme Learning Machine (ELM) to learn to classify static hand gestures on the letters of the Auslan dictionary. Although the research shows good preliminary results, the data used to test network was restricted to image collected from same day, thus lacking generalization.

Fig.1. the hand poses used as a basis in constructing the dataset for each letter.
III. LITERATURE REVIEW
Paper Title: American Sign Language Recognition using Deep Learning and Computer Vision
Author Name: Kshitij Bantupalli, Ying XieIn this paper, they address the problem of people who are affected by an individual’s ability to communicate using speech and hearing. People who are affected by this use other media of communication such as sign language. Although sign language is ubiquitous in recent times, there remains a challenge for non-sign language speakers to communicate with sign language speakers or signers. With recent advances in deep learning and computer vision there has been promising progress in the fields of motion and gesture recognition using deep learning and computer vision based techniques. The focus of this work is to create a vision based application which offers sign language translation to text thus aiding communication between signers and non-signers. The proposed model takes video sequences and extracts temporal and spatial features from them. We then use Inception, a CNN (Convolutional Neural Network) for recognizing spatial features. We then use a RNN (Recurrent Neural Network) to train on temporal features. The dataset used is the American Sign Language Dataset. [1]
Paper Title: Human Action Recognition Using Deep Neural Networks
Author Name: Rashmi R. Koli, Tanveer N. BagbanIn this paper, they present the recognition of human activities such as gestures this is the most difficult and challenging thing in neural network. They introduce to develop a platform for hand movement recognition, which recognizes hand movement (gestures), by using the CNN, we can identify human gestures in the image. The purpose of this gadget that broadens the popularity of gestures, which can recognize gestures and then convert gesture images into text accordingly. The system pays special attention to the CNN training component using the CNN algorithm. The concept includes designing a gadget that uses in-depth mastery standards to treat input as a gesture and then provide recognizable output as text. This system is introduced for Deaf and Dumb People for removing the communication gap between Normal People. Through, deaf and mute people can use gestures as their main language, which will be converted to text or audio. [2]
Paper Title: Alphabet Sign Language Image Classification Using Deep Learning
Author Name: Rangel Daroya, Daryl Peralta, and Prospero Naval
In this paper, they presented a work of a method to classify RGB images of static letter hand poses in Sign Language using a Convolutional Neural Network (CNN) inspired by Densely Connected Convolutional Neural Networks (Dense Net). It was further implemented to classify sign languages in real time using a web camera. Dense Net has been widely used for classification tasks due to the advantages it introduces such as alleviating the vanishing gradient - a common problem encountered with deep networks. Since a deep network is proposed to be used for our sign language classification task, this characteristic is useful. Our proposed network was able to achieve an accuracy of 90.3 % which is comparable to other works including those that used depth images in addition to RGB images. Our network was also able to achieve prediction rates of 50 to 100 Hz which makes it capable of real-time prediction. They presented an approach to real-time alphabet sign language image classification using deep learning. Our proposed deep learning model based on Dense Net was able to perform with 90.3 % accuracy on our test dataset. Our work also included data gathering using a web camera to increase the dataset size to more than 100,000 RGB images making our prediction more robust. Our network was also capable of real-time prediction using image frames from a web camera with rates of 50 to 100 Hz. [3]
Paper Title: Signet: A Deep Learning based Indian Sign Language Recognition System
Author Name: Sruthi C. J and Lijiya a Member, IEEE
In this paper, they insist the need for sign language interpreters in all Government organizations and public sector undertakings in order to abide RPwD Act 2016. This can avoid their isolation from the rest of society to a great extent. In this work, we propose a signer independent deep learning based methodology for building an Indian Sign Language (ISL) static alphabet recognition system. Here, we review various existing methods in sign language recognition and implement Convolutional Neural Network (CNN) architecture for ISL static alphabet recognition from the binary silhouette of signer hand region. They also discuss in detail, the dataset used along with the training phase and testing phase of CNN. The proposed method was successfully implemented with an accuracy of 98.64% which is better than most of the currently existing methods. The paper presented a vision based deep learning architecture for signer independent Indian sign language recognition system. The system was successfully trained on all 24 ISL static alphabets with a training accuracy of 99.93% and with testing and validation accuracy of 98.64%. The recognition accuracy obtained is better than most of the current state of art methods. [4]
Paper Title: A Brief Review of the Recent Trends in Sign Language Recognition
Author Name: Nimisha K P and Agnes Jacob
In this paper, since the deaf and dumb constitute nearly 90 million of the world population. Therefore sign language recognition (SLR) and translation is a fertile area of research. The two main approaches for SLR are (i) image based and (ii) sensor based. Image based approach involves one or more cameras to capture an image sequence of the signer performing the sign, and then uses image processing to recognize the sign. The sensor based method uses instrumental gloves assembled with sensors to track the hand articulates. This paper mainly describes various image or vision based SLR systems comprising feature extraction and classification. Translation of SL to speech is also described briefly. Overall, this paper is expected to be a complete introduction to automatic hand gesture recognition and sign language interpretation (SLI). From all the proposed methods and experiments that have been analysed, a nearly perfect system for sign translation is still far away from reality. This review paper is intended to provide a lay person with an overview of this topic and briefs the reader on popular techniques in SLR. [5]
Paper Title: Indian Sign Language Gesture Recognition using Image Processing and Deep Learning
Author Name: Neel Kamal Bhagat, Vishnusai Y, Rathna G N
In this paper, they proposed a real time hand gesture recognition system based on the data captured by the Microsoft Kinect RGBD camera. Given that there is no one to one mapping between the pixels of the depth and the RGB camera, we used computer vision techniques like 3D construction and affine transformation. After achieving one to one mapping, segmentation of the hand gestures was done from the background noise. Convolutional Neural Networks (CNNs) were utilised for training 36 static gestures relating to Indian Sign Language (ISL) alphabets and numbers. The model achieved an accuracy of 98.81% on training using 45,000 RGB images and 45,000 depth images. Further Convolutional LSTMs were used for training 10 ISL dynamic word gestures and an accuracy of 99.08% was obtained by training 1080 videos. The model showed accurate real time performance on prediction of ISL static gestures, leaving a scope for further research on sentence formation through gestures. This paper proposes a real-time model for ISL gesture recognition, based on the incoming image data from the Kinect. Effective real time background subtraction was done using depth perception techniques. Computer vision techniques were used to achieve one-to-one mapping between the depth and the RGB pixels. The model trained on the dynamic dataset showed high variance leaving scope of further research in this area for achieving real-time performance. Also further research can be focused on real-time prediction of more words related to ISL and also on sentence formation. [6]
Paper Title: A Real-Time System For Recognition Of American Sign Language By Using Deep Learning
Author Name: Murat Taskiran, Mehmet Killioglu, and Nihan KahramanIn this paper, they have developed a real-time sign language recognition system for people who do not know sign language to communicate easily with hearing-impaired people. The sign language used in this paper is American Sign Language. In this study, the convolutional neural network was trained by using dataset collected in 2011 by Massey University, Institute of Information and Mathematical Sciences, and 100% test accuracy was obtained. After network training is completed, the network model and network weights are recorded for the real-time system. In the real-time system, the skin colour is determined for a certain frame for hand use, and the hand gesture is determined using the convex hull algorithm, and the hand gesture is defined in real-time using the registered neural network model and network weights. The accuracy of the real-time system is 98.05%. Proposed system has advantage in terms of test accuracy according to these similar studies. Also, the applicability as real-time system is validated. This system achieved high accuracy even with the letters that have similar gestures. [7]
IV. CONCLUSION
This is a review paper which is focused on collection of different research papers that introduced for deaf or dumb people for bring out the communication gap between Normal peoples. Through this project, deaf and dumb people can use gestures as their main language which will be converted to the text or audio format. Therefore the connection between them can easily occupy the area. This is research within the function extraction and lighting to make the system more reliable. The system converts gestures in video into frames and then meaningful text or audio so ordinary people know what the deaf and mute will say. Hence these deaf and dumb people can establish and maintain contact with their society. Gestures are essential factor of human relation, each interpersonally and in the surrounding of human machine interfaces. There are many ways to recognize human gestures, so it needs spot valuable key element in action. The CNN algorithms interpret the gesture and build a statement from video. This statement or textual information is the meaning of those gestures.
 

ACKNOWLEDGEMENT
First and foremost, I would like to express my sincere gratitude to my guide Prof. Madhuri Bhalekar who has in the literal sense, guided and supervised me. I am indebted with a deep sense of gratitude for the constant inspiration and valuable guidance throughout the work.
 

The above IT Assignment has been solved by our  IT Assignment  Experts at My Uni Paper. Our Assignment Writing Experts are efficient to provide a fresh solution to this question. We are serving more than 10000+ Students in Australia, UK & US by helping them to score HD in their academics. Our experts are well trained to follow all marking rubrics & referencing style.

Be it a used or new solution, the quality of the work submitted by our assignment experts remains unhampered. You may continue to expect the same or even better quality with the used and new assignment solution files respectively. There’s one thing to be noticed that you could choose one between the two and acquire considered worthy of the highest distinction.