Implementation of the Vocabulary

   During the last few weeks, Valerie and I have been working on implementing the vocabulary and grammar that was created last semester into Simon. I have a windows computer and it hasn’t been responding to the voice commands from the scenario that I downloaded which was “EN/H4W] Mouse”. We created a scenario of my own and my goal was to open an executable file, which in this case was a Python command line. We were able to create a command using the "Program" type and activate it manually by clicking on the Trigger button that is in the Grammar module, but we were not able to activate it by voice. Valerie and I tried to do the same thing in her Mac computer but had the same results as before.

   Another thing that happened these past few weeks is that Simon has been giving me the following error every time I activate it:

The recognition reported the following error:

Failed to setup recognition: Julius did not initialize correctly

After asking the KDE community what might be causing this error and how can I solve it through an email, they were able to point out that I was mixing a speech model ([EN/VF/JHTK] Voxforge) and a scenario (EN/H4W] Mouse) that had different phonemes. According to Peter, the person who respond my question, “the EN/H4W mouse scenario uses the H4W phoneme set, while the EN/VF base model uses the VF phoneme set”. Now I need to correct this mistake and use a scenario that matches the phoneme of the speech model. Valerie and I need to figure out how to activate the commands that we create and I need to do the necessary adjustments to correct the mistake I made by selecting a scenario with a matching  EN/VF phoneme. 

Hello Kavita!

Hello! I am the newest member of this research.  I’m so excited to join this project and I’m willing to keep up with all of the work that all of you have done already. It is amazing what technology can do through developing different tools to help and improve assistive technologies. Still there are countless problems that we can assist and make it possible for people with disabilities. It is important to mention that having a disability does not mean that you can’t achieve a task.

            Last week, Professor Ordóñez presented me Kavita through facebook. It was a pleasure to know about the inspiration of the project and it will be an honor to have the opportunity of meeting her in person some day. She will now be like a role model to me because of all of the achievements that she done in her life. I will be happy to work in this research with all of you and Kavita.

Development of a Vocabulary for Speech Recognition Software

  Last semester Andrea, Natalia, Xiomara and I developed a vocabulary to use while programming  using speech recognition software such as Simon. The vocabulary uses gerunds to identify the key words that are commonly used in many programming languages.  We then created a survey that consisted of 10 statements with three options on how the person would verbalize it. The statements used basic programing structures such as loops, conditional, creating variables, among others to determine the best way to verbalize them.  

   A total of 33 students from the University of Puerto Rico, Rio Piedras Campus and Polytechnic University of Puerto Rico responded to the survey.  Based on the results of the survey, we developed a final version of the vocabulary that we expect would be accepted by the programming community.

Our vocabulary in the works

Xiomara, Andrea, Natalia and I have been working on developing a vocabulary that can be used to program by voice. Our goal is that this vocabulary can be used in any programming language such as Python and C++. First Xiomara selected a piece of code from the internet that was written in Python. We used this piece of code to write down how we would say it if we were to speak the code out loud. Having an idea of how we would command by voice, we each wrote down a list of words and phrases that we thought were essential to program in any programming language. For example we each wrote a phrase that would be used to create a Function. Now that we each created our own list, we are in the process of selecting the best words from our lists to create a vocabulary that will be used to program by voice. 

Considering Andrew Begel's lectures

Some forms of assistive technology challenge our conceptions of technology enhanced performance. Voice recognition software for programming is very uncommon, yet necessary. One way to reduce the amount of typing while programming is to use speech recognition. Programming itself has a structured format that can be incorporated into an IDE that enhances speed of code production with fewer keystrokes and navigation through the display.

Its needs to be easy to learn, preferably using spoke constructs that the programmer already knows. But, would all programmers naturally speak the same language when they verbalize their program? This question was answered by Andrew Begel and his team in their paper: An Assessment of a Speech-Based Programming Environment. They conducted an experiment in which participants read a one-page pre-existing Java program out loud. They found that there does exist a common vernacular among programmers for speaking programs despite the diversity of their educational training. This enables us to create a verbalized programming language definition which will work for most programmers. 

There's a very specific issue to address, and it should be planned in the way of a whole new environment, taking advantage of the existing tools for programming but having in mind that it has to satisfy specific needs. Because the target is to program by voice recognition, the future work would be in the way of a whole new kind of interaction, a new kind of input. 

Creating awareness

https://www.youtube.com/watch?v=dTwXeZ4GkzI

https://www.youtube.com/watch?v=8YZkubsKRPw

Like many others I didn't understand why people with disabilities got insulted when they were told that they were disabled. Now that I have seen the video with Amiee Mullins I understand why they get insulted for being called disabled. Saying that a person is disabled is like telling them that they’re useless and can’t do anything, which is not the case for people with disabilities. In many occasions they have to be creative in order to overcome a physical obstacle that they must face. This is something that is very important in life and in our society. Having said that, this doesn't make them more special than any other person because each and every one of us faces many obstacles in life and we all find different ways to face these obstacles. Therefore, people with disabilities should be treated as equals and we should all be conscious about how important it is to say or write it correctly because it can make a great impact.

Entry about the paper: "An Assessment of a Speech-Based Programming Environment"

Considering:

Begel, A. and Graham, S. L., "An Assessment of a Speech-Based
Programming Environment", In Visual Languages and Human-Centric
Computing (VL/HCC'06), pp. 116-120, 2006.

This study illustrates the viability of a speech-recognition program for daily work in a programmer's life. It closes itself in a period of the user's lifetime, meaning that he will use it in case of being unable to type for any reason or because of suffering from RSI. The results show the pros and cons, being the machine's errors in its job to recognize the programmer's voice vs. the time it takes the user to type. This, exhibits the need of the machine's voice recognition accuracy in order to interpret the commands rapidly to make it faster and more effective than typing. When computer science reach the required level of precision around this program, the user's mistrust in the machine's VR will not obstruct their normal pace programming; thus, it'll surpass the effectiveness of using the keyboard.