Friday, August 28, 2015

Final Proposed Vocabulary and Grammar

       A detailed survey was developed and administered with the purpose of gathering the feedback from the programming community regarding alternative options for the proposed vocabulary and grammar.  The questions selected for the survey represent an assortment of programming commands that fall under each of the programming control flow structures: Sequential Structures; Decision or Conditional Structures, and Looping or Repetitive Structures.  The participants were students of Computer Science and Computer Engineering  from the University of Puerto Rico and the Polytechnic University of Puerto Rico. 
       After performing a qualitative and quantitative analysis of the results obtained from the survey, which contains 10 questions (each one is a multiple choice featuring two or more possible verbalizations of the code and one write-in answer slot for alternative suggestions), a final version of the grammar and vocabulary was proposed, and is presented in Table 1. It is expected to be embraced by the programming community.
       The proposed voice command is written in bold, arial, and red, the corresponding C++ code, as a consequence of the voice command, in italics, bold, and black (this is the output produced by the voice-recognition software in an editor), and variables and literals presented in italics, and green.  

Table 1: Final Proposed Vocabulary and Grammar      
Programming structure
Voice commands
Variable definition
defining integer variable
            int ;
total
            int total;
Standard output: literals
printing literal
            cout << “ ”;
hello world
           cout << “ hello world”;
Standard output: variables
printing variable
            cout << ;
total
            cout << total;
Mathematical operations
printing
            cout << ;
grouping
            cout << () ;
total adding 2
            cout << ( total + 2 ) ;
moving right, symbol frontslash 4
            cout << ( total + 2 ) / 4 ;
Standard input
getting input into variable total
cin >> total;
Modulus operations
total modulus 3
            total % 3
Relational operations
total symbol greater than seven
total > 7
Logical operations
(statementA) operating and
            (statement) &&
negating (statementB)
            (statement) && !(statement)
Loops
inserting for loop
            for ( ; ; )
            {

            }
inserting initiation (initializeExpression)
inserting condition (conditionalExpression
inserting modification (modifyExpression)
            for ( initializeExpression; conditionalExpression; modificationExpression)
            {

            }
entering loop (statement(s))
            for ( initializeExpression; conditionalExpression; modificationExpression)
            {
                        statement(s);
            }
Conditionals
inserting conditional
if()
{
}
inserting condition total symbol greater than seven
if( total > 7)
{
}
printing literal eight
if( total > 7)
{
cout << “8”;
}
else printing literal six
if( total > 7)
{
cout << “8”;
}else
{
cout << “6”;
}

Tuesday, April 14, 2015

Prototype using for loop

Today, Jean K. and I finished implementing the vocabulary showed in an earlier post. We also made changes to the vocabulary by adding and eliminating some words, commands, sentences in Simon's Grammar module. During the implementation we create the commands necessary to create a program that would print the numbers from 0 to 9. Here is a video that we made showing the creation of this program.


Note: This video does not has sound because there was some technical difficulties when it was recorded.

Saturday, April 4, 2015

Categorized vocabulary

In the last few weeks, Jean Karlo and I were working with the vocabulary that was developed last semester. We made a few tables to categorized the vocabulary. The categories were also established last semester in the proposal of Andrea and Natalia.

Here is the first table (draft) that I made 

action
object
operator
text
command
launch
key
printing
variable
adding
literal
symbol
entering
else
grouping

frontslash

operating

for
getting

modulus

initiation


defining

greater than

condition


inserting

and

modification




negating





Later, Jean Karlo made a few changes expanding the vocabulary and adding new categories

Action
Object
Operator
String
Command
inserting
variable
adding
literal
entering
loop
frontslash
operating
defining
modulus
initiation
greater than
condition
and
modification
negating
less than
equal
grouping
()


Launch
Key
Text
Type
Number
opening
else
accum
character (char)
1 to 9
closing
for
total
integer (int)
starting
printing
a - z
float
ending
while
double
conditional (if)
string
function
class

Friday, April 3, 2015

Our First Prototype of the Semester.

Last week in our weekly meeting Gustavo Gratacos, an undergraduate student who worked in this project in previous years, came by to help us with the problem that we were having with Simon. The problem was that Simon recognized the commands that we created but didn't executed them. After showing Gustavo the problems that we were having and walking him through the steps that we were following, he managed to solve the problem. When we were adding the plugins in the ‘Commands’ Module we needed to click on the plugin that was added and delete what was in the Trigger space, which in this case was ‘Computer’ as shown in the picture below. After doing this, all of the commands that we created worked.




In this meeting we also created a Simple "Hello World" program in a text editor called ‘Kate’, which we made a video of .


Thursday, March 5, 2015

Implementation of the Vocabulary

   During the last few weeks, Valerie and I have been working on implementing the vocabulary and grammar that was created last semester into Simon. I have a windows computer and it hasn’t been responding to the voice commands from the scenario that I downloaded which was “EN/H4W] Mouse”. We created a scenario of my own and my goal was to open an executable file, which in this case was a Python command line. We were able to create a command using the "Program" type and activate it manually by clicking on the Trigger button that is in the Grammar module, but we were not able to activate it by voice. Valerie and I tried to do the same thing in her Mac computer but had the same results as before.

   Another thing that happened these past few weeks is that Simon has been giving me the following error every time I activate it:

The recognition reported the following error:
Failed to setup recognition: Julius did not initialize correctly


After asking the KDE community what might be causing this error and how can I solve it through an email, they were able to point out that I was mixing a speech model ([EN/VF/JHTK] Voxforge) and a scenario (EN/H4W] Mouse) that had different phonemes. According to Peter, the person who respond my question, “the EN/H4W mouse scenario uses the H4W phoneme set, while the EN/VF base model uses the VF phoneme set”. Now I need to correct this mistake and use a scenario that matches the phoneme of the speech model. Valerie and I need to figure out how to activate the commands that we create and I need to do the necessary adjustments to correct the mistake I made by selecting a scenario with a matching  EN/VF phoneme. 

Friday, February 20, 2015

Hello Kavita!

Hello! I am the newest member of this research.  I’m so excited to join this project and I’m willing to keep up with all of the work that all of you have done already. It is amazing what technology can do through developing different tools to help and improve assistive technologies. Still there are countless problems that we can assist and make it possible for people with disabilities. It is important to mention that having a disability does not mean that you can’t achieve a task.


            Last week, Professor Ordóñez presented me Kavita through facebook. It was a pleasure to know about the inspiration of the project and it will be an honor to have the opportunity of meeting her in person some day. She will now be like a role model to me because of all of the achievements that she done in her life. I will be happy to work in this research with all of you and Kavita.

Thursday, February 5, 2015

Development of a Vocabulary for Speech Recognition Software


  Last semester Andrea, Natalia, Xiomara and I developed a vocabulary to use while programming  using speech recognition software such as Simon. The vocabulary uses gerunds to identify the key words that are commonly used in many programming languages.  We then created a survey that consisted of 10 statements with three options on how the person would verbalize it. The statements used basic programing structures such as loops, conditional, creating variables, among others to determine the best way to verbalize them.  
   A total of 33 students from the University of Puerto Rico, Rio Piedras Campus and Polytechnic University of Puerto Rico responded to the survey.  Based on the results of the survey, we developed a final version of the vocabulary that we expect would be accepted by the programming community.