Friday, November 6, 2015

Demo of Simon Codes at Assets 2015

Congratulations to everyone that had participated at the Kavita Lab with Simon Codes. Like Andrea Claudio said in her post, we went to Lisbon, Portugal to present a demo of Simon Codes and received the second Best Demo Award. I'm so happy for everyone and want to thanks our mentor Dra. Patricia Ordóñez.

Lisbon, Portugal! Oh, such a beautiful city! I am in love with Lisbon. But the conference was truly inspiring. This was my first participation at a conference at all. For the demo presentation, at first I was nervous but when I started to communicate with everybody it was an amazing feeling and a very satisfying experience. 

Thank you for everything Andrea, Patti, Assets 2015, Lisbon and everybody!






Wednesday, November 4, 2015

ACM ASSETS 2015 Conference: Lisbon, Portugal

                               

         We went to Lisbon, Portugal to present the demo. We called it SIMON CODES. Having the opportunity to meet professionals in the industry and academia is unique and bolstering. Each time I explained the scope of SIMON CODES I felt a personal satisfaction that I was able to transmit, getting the people interested in the project. The papers presented were astonishing, and the projects were of high quality and mainly oriented to the computing and accessibility for people with disabilities.
ASSETS 2015 Conference logo


        Lisbon is a neat place with well-educated citizens and beautiful places to visit and enjoy. The morning run was our time to see the country, and we certainly took advantage of it.

                     
              
Here is a link to the list of accepted demos for ASSETS 2015:

Friday, September 18, 2015

My First Week with the Kavita Project



First of all, I thank my predecessors and contemporaries in the project for all their work to date; and of course, Dr. Ordoñez for inviting me to contribute. Beyond a worthwhile endeavor in developing accessibility tools for programming, working with spoken interface software presents many opportunities for insight into how much more we can integrate computing into daily life.

Building on the progress of the past few years, I, along with Valerie and Andrea, have been configuring Simon to write code on behalf of us, instructed (mostly) by our voice. As we implement the recommendations of the previously administered survey, we're taking note of limitations as well as how to work around them in our effort to create something truly useful.

As well as using the previously set up Simon, I've been trying to get the program running in my computer - technical post to follow. I have run into some hurdles, but as these are worked around I'll learn more about Simon and a little about my computer as well.

This has been my first post, stay tuned for more.

Friday, August 28, 2015

Final Proposed Vocabulary and Grammar

       A detailed survey was developed and administered with the purpose of gathering the feedback from the programming community regarding alternative options for the proposed vocabulary and grammar.  The questions selected for the survey represent an assortment of programming commands that fall under each of the programming control flow structures: Sequential Structures; Decision or Conditional Structures, and Looping or Repetitive Structures.  The participants were students of Computer Science and Computer Engineering  from the University of Puerto Rico and the Polytechnic University of Puerto Rico. 
       After performing a qualitative and quantitative analysis of the results obtained from the survey, which contains 10 questions (each one is a multiple choice featuring two or more possible verbalizations of the code and one write-in answer slot for alternative suggestions), a final version of the grammar and vocabulary was proposed, and is presented in Table 1. It is expected to be embraced by the programming community.
       The proposed voice command is written in bold, arial, and red, the corresponding C++ code, as a consequence of the voice command, in italics, bold, and black (this is the output produced by the voice-recognition software in an editor), and variables and literals presented in italics, and green.  

Table 1: Final Proposed Vocabulary and Grammar      
Programming structure
Voice commands
Variable definition
defining integer variable
            int ;
total
            int total;
Standard output: literals
printing literal
            cout << “ ”;
hello world
           cout << “ hello world”;
Standard output: variables
printing variable
            cout << ;
total
            cout << total;
Mathematical operations
printing
            cout << ;
grouping
            cout << () ;
total adding 2
            cout << ( total + 2 ) ;
moving right, symbol frontslash 4
            cout << ( total + 2 ) / 4 ;
Standard input
getting input into variable total
cin >> total;
Modulus operations
total modulus 3
            total % 3
Relational operations
total symbol greater than seven
total > 7
Logical operations
(statementA) operating and
            (statement) &&
negating (statementB)
            (statement) && !(statement)
Loops
inserting for loop
            for ( ; ; )
            {

            }
inserting initiation (initializeExpression)
inserting condition (conditionalExpression
inserting modification (modifyExpression)
            for ( initializeExpression; conditionalExpression; modificationExpression)
            {

            }
entering loop (statement(s))
            for ( initializeExpression; conditionalExpression; modificationExpression)
            {
                        statement(s);
            }
Conditionals
inserting conditional
if()
{
}
inserting condition total symbol greater than seven
if( total > 7)
{
}
printing literal eight
if( total > 7)
{
cout << “8”;
}
else printing literal six
if( total > 7)
{
cout << “8”;
}else
{
cout << “6”;
}

Tuesday, April 14, 2015

Prototype using for loop

Today, Jean K. and I finished implementing the vocabulary showed in an earlier post. We also made changes to the vocabulary by adding and eliminating some words, commands, sentences in Simon's Grammar module. During the implementation we create the commands necessary to create a program that would print the numbers from 0 to 9. Here is a video that we made showing the creation of this program.


Note: This video does not has sound because there was some technical difficulties when it was recorded.

Saturday, April 4, 2015

Categorized vocabulary

In the last few weeks, Jean Karlo and I were working with the vocabulary that was developed last semester. We made a few tables to categorized the vocabulary. The categories were also established last semester in the proposal of Andrea and Natalia.

Here is the first table (draft) that I made 

action
object
operator
text
command
launch
key
printing
variable
adding
literal
symbol
entering
else
grouping

frontslash

operating

for
getting

modulus

initiation


defining

greater than

condition


inserting

and

modification




negating





Later, Jean Karlo made a few changes expanding the vocabulary and adding new categories

Action
Object
Operator
String
Command
inserting
variable
adding
literal
entering
loop
frontslash
operating
defining
modulus
initiation
greater than
condition
and
modification
negating
less than
equal
grouping
()


Launch
Key
Text
Type
Number
opening
else
accum
character (char)
1 to 9
closing
for
total
integer (int)
starting
printing
a - z
float
ending
while
double
conditional (if)
string
function
class

Friday, April 3, 2015

Our First Prototype of the Semester.

Last week in our weekly meeting Gustavo Gratacos, an undergraduate student who worked in this project in previous years, came by to help us with the problem that we were having with Simon. The problem was that Simon recognized the commands that we created but didn't executed them. After showing Gustavo the problems that we were having and walking him through the steps that we were following, he managed to solve the problem. When we were adding the plugins in the ‘Commands’ Module we needed to click on the plugin that was added and delete what was in the Trigger space, which in this case was ‘Computer’ as shown in the picture below. After doing this, all of the commands that we created worked.




In this meeting we also created a Simple "Hello World" program in a text editor called ‘Kate’, which we made a video of .


Thursday, March 5, 2015

Implementation of the Vocabulary

   During the last few weeks, Valerie and I have been working on implementing the vocabulary and grammar that was created last semester into Simon. I have a windows computer and it hasn’t been responding to the voice commands from the scenario that I downloaded which was “EN/H4W] Mouse”. We created a scenario of my own and my goal was to open an executable file, which in this case was a Python command line. We were able to create a command using the "Program" type and activate it manually by clicking on the Trigger button that is in the Grammar module, but we were not able to activate it by voice. Valerie and I tried to do the same thing in her Mac computer but had the same results as before.

   Another thing that happened these past few weeks is that Simon has been giving me the following error every time I activate it:

The recognition reported the following error:
Failed to setup recognition: Julius did not initialize correctly


After asking the KDE community what might be causing this error and how can I solve it through an email, they were able to point out that I was mixing a speech model ([EN/VF/JHTK] Voxforge) and a scenario (EN/H4W] Mouse) that had different phonemes. According to Peter, the person who respond my question, “the EN/H4W mouse scenario uses the H4W phoneme set, while the EN/VF base model uses the VF phoneme set”. Now I need to correct this mistake and use a scenario that matches the phoneme of the speech model. Valerie and I need to figure out how to activate the commands that we create and I need to do the necessary adjustments to correct the mistake I made by selecting a scenario with a matching  EN/VF phoneme. 

Friday, February 20, 2015

Hello Kavita!

Hello! I am the newest member of this research.  I’m so excited to join this project and I’m willing to keep up with all of the work that all of you have done already. It is amazing what technology can do through developing different tools to help and improve assistive technologies. Still there are countless problems that we can assist and make it possible for people with disabilities. It is important to mention that having a disability does not mean that you can’t achieve a task.


            Last week, Professor Ordóñez presented me Kavita through facebook. It was a pleasure to know about the inspiration of the project and it will be an honor to have the opportunity of meeting her in person some day. She will now be like a role model to me because of all of the achievements that she done in her life. I will be happy to work in this research with all of you and Kavita.