Tongue For Dumb (hand gesture based communication interpreter system)

Idea behind the development of this module is to speak for the voiceless people in order to fulfill their basic needs thereby to make them self reliable and independent. It aims to facilitate vocally disabled people by means of a hand gesture based communication interpreter system.  A system that can efficiently translate gestures made by  hand to both text and auditory voice. 

BLOCK DIAGRAM

The whole module has been divided into 2 major parts:

1) Hand gesture reading and text display
Accelerometer is used to sense the gestures of the finger. Depending upon the changed axis of the accelerometer attached on finger, creates readings. These readings are fed at the analog pin A0, A1 and A2 of the atmega328 (microcontroller). These analog I/Ps are then converted into digital signals by the inbuilt ADC in the microcontroller. According to the readings received by the accelerometer, microcontroller forwards the corresponding text to the LCD for the display of text. Atmega 32 microcontroller had been programmed to convert the reading received from the accelerometer into text and speech signals.

2) Fetching of audio files and providing voice signal to the speaker
Whenever any gesture is sensed, master micro-controller display text on the LCD and gives instruction to the slave micro-controller. Slave microcontroller carry out the speech signal instruction and fed signal to the audio amplifier. These analog signals received from by the audio amplifier circuit comprising of  LM386 increase the gain of the audio. A speaker is used to output these voice signals.

HARDWARE COMPONENTS     ARDUINO WITH BUILT IN ATMEGA 328,  ACCELEROMETER,  LM386,  SPEAKER,  LCD 16X2

SOFTWARE USED  ARDUINO 1.6.3,  MATLAB R2010a,  AUDACITY 2.1.0

CIRCUIT DESCRIPTION

The circuit consist of 2 section:

First section consist of power supply, Atmega 328(1), LCD display and accelerometer. 12v battery provides the initial power to the voltage regulator 7805 which further provides the 5 v supply to the whole circuit. Pin D4, D5, D6 and D7 is connected with pins 5, 4, 3, 2 of atmega 328 respectively. Pin X, Y, Z of accelerometer are connected with pin A0, A1, A2 of atmega 328 respectively. 

Second section consist of Atmega 328(2), audio amplifier and speaker. The atmega 328 (2) is connected with the atmega 328(1) through pin Tx, Rx. Pin 3 of audio amplifier is connected with the pin 11 of Atmega 328(2). The audio amplifier comprises of power amplifier LM328 IC. An external capacitor of 10uf connected between pin 1 and 8 of the IC provides gain of 200 (46 dB). The speaker is connected to pin 8 through the 250uf capacitor. The Pin 3 is connected to the potentiometer and is connected to the pin D6 of the microcontroller. Speaker is connected with pin 5 of audio amplifier. 

Speech Synthesis

Speech Synthesis is performed using the MATLAB and audacity software. MATLAB software is used for the output scaling of the compressed audio files and audacity software is used to achieve compressed file. Following are the steps performed for speech synthesis:

Step 1: Compression of audio file and converting them into the .wav format using audacity software.The original audio is in the mp3 format which is compressed using audacity software at a 8kHz sample rate and 16 bits per sample. The need to compress the file is beacuse samples do not fit in the micro-controller uncompressed. The size of the audio file after the compression ranges between 1KB - 20 KB. This compressed file is then converted to .wav format. as the c code.The need to change the format from mp3 to .wav is that the c code requires the array of the audio file and only .wav file can provide this array.
Step 2: Output scaling using Matlab software

The samples of the audio file in the .wav format contains value of the samples in the double format. This needs to be converted into native format. This is called 
Output Scaling.Output scaling is based on typical bit-widths found in a .wav file. 'Native' format uses data type unit8 (unsigned integer) and data range ranges from 0 <= Y <= 255.
The need to perform output scaling is that the c code can only read the array existing in native format.Result is the array ranging between values 0 <= Y <= 255. This array is copied into the c code which is read by the microcontroller.

CIRCUIT SNAPSHOT

 

GESTURES USED FOR THE PROJECT

                      

Fig 4. HOW U DOING        Fig.5 WE ARE FROM AMITY

RESULTS

          

 Fig.6 When the accelerometer is oriented Z axis: values= 400+           

 

 Fig.7 When accelerometer is oriented in Y axis: values = 400 + at the middle column 

Fig.8 When the accelerometer is oriented X axis : values : at the fisrt column = 400+

By connecting more accelerometer to the figures and making use of flex sensor more gestures can be read hence more alphabets and sentences can be formed.

LINK TO PROGRAM CODE

Code 1 (For LCD) click here

Code 2 (For Text To Speech) click here

 

 

Category: 

Share

Who's new

  • ravirajpatil871...
  • shubhambajoria
  • yassir
  • demiholyman890954
  • scottgillum51169040

Get Notified

 

Share

We are Social

Syndicate

Subscribe to Syndicate