Our vocabulary in the works

Xiomara, Andrea, Natalia and I have been working on developing a vocabulary that can be used to program by voice. Our goal is that this vocabulary can be used in any programming language such as Python and C++. First Xiomara selected a piece of code from the internet that was written in Python. We used this piece of code to write down how we would say it if we were to speak the code out loud. Having an idea of how we would command by voice, we each wrote down a list of words and phrases that we thought were essential to program in any programming language. For example we each wrote a phrase that would be used to create a Function. Now that we each created our own list, we are in the process of selecting the best words from our lists to create a vocabulary that will be used to program by voice. 

Considering Andrew Begel's lectures

Some forms of assistive technology challenge our conceptions of technology enhanced performance. Voice recognition software for programming is very uncommon, yet necessary. One way to reduce the amount of typing while programming is to use speech recognition. Programming itself has a structured format that can be incorporated into an IDE that enhances speed of code production with fewer keystrokes and navigation through the display.

Its needs to be easy to learn, preferably using spoke constructs that the programmer already knows. But, would all programmers naturally speak the same language when they verbalize their program? This question was answered by Andrew Begel and his team in their paper: An Assessment of a Speech-Based Programming Environment. They conducted an experiment in which participants read a one-page pre-existing Java program out loud. They found that there does exist a common vernacular among programmers for speaking programs despite the diversity of their educational training. This enables us to create a verbalized programming language definition which will work for most programmers. 

There's a very specific issue to address, and it should be planned in the way of a whole new environment, taking advantage of the existing tools for programming but having in mind that it has to satisfy specific needs. Because the target is to program by voice recognition, the future work would be in the way of a whole new kind of interaction, a new kind of input. 

Creating awareness

https://www.youtube.com/watch?v=dTwXeZ4GkzI

https://www.youtube.com/watch?v=8YZkubsKRPw

Like many others I didn't understand why people with disabilities got insulted when they were told that they were disabled. Now that I have seen the video with Amiee Mullins I understand why they get insulted for being called disabled. Saying that a person is disabled is like telling them that they’re useless and can’t do anything, which is not the case for people with disabilities. In many occasions they have to be creative in order to overcome a physical obstacle that they must face. This is something that is very important in life and in our society. Having said that, this doesn't make them more special than any other person because each and every one of us faces many obstacles in life and we all find different ways to face these obstacles. Therefore, people with disabilities should be treated as equals and we should all be conscious about how important it is to say or write it correctly because it can make a great impact.

Entry about the paper: "An Assessment of a Speech-Based Programming Environment"

Considering:

Begel, A. and Graham, S. L., "An Assessment of a Speech-Based
Programming Environment", In Visual Languages and Human-Centric
Computing (VL/HCC'06), pp. 116-120, 2006.

This study illustrates the viability of a speech-recognition program for daily work in a programmer's life. It closes itself in a period of the user's lifetime, meaning that he will use it in case of being unable to type for any reason or because of suffering from RSI. The results show the pros and cons, being the machine's errors in its job to recognize the programmer's voice vs. the time it takes the user to type. This, exhibits the need of the machine's voice recognition accuracy in order to interpret the commands rapidly to make it faster and more effective than typing. When computer science reach the required level of precision around this program, the user's mistrust in the machine's VR will not obstruct their normal pace programming; thus, it'll surpass the effectiveness of using the keyboard.

An Empowering Perspective

http://www.ted.com/talks/stella_young_i_m_not_your_inspiration_thank_you_very_much 

http://www.ted.com/talks/aimee_mullins_the_opportunity_of_adversity

http://www.ted.com/talks/ron_mccallum_how_technology_allowed_me_to_read

Stella Young's "I'm not your inspiration, thank you very much", as well as Amee Mullins's "The opportunity of adversity", illustrate the need of a collective change of perspective from society toward people with disabilities; the need to change to an empowering perspective rather than a condoling and belittling one. Furthermore, in his speech "How technology allowed me to read", John McCallum shows the speakers everything that the development of assistive technologies aided him to achieve, and the indispensable support that said technologies provided in his journey to becoming an outstanding man among people with and without medical disabilities alike. These thoughts lead to the understanding that the development of tools within the field of assistive technologies is part of this necessary change of attitude toward people that society has come to wrongly regard as "disabled". Assistive technologies are an empowering tool to help people with disabilities rise above their medical prognosis in a world designed for people without their own singularities.

What can you do?

This video teaches that the human will is stronger than any other force.
What can you do?
 http://www.youtube.com/watch?v=J9aHT-syZxs

Getting involved in the environment

https://www.youtube.com/watch?v=dTwXeZ4GkzI

While watching this video, a ship was installed in my brain. I mean that I, suddenly, understood our world and society from a new perspective ,the language. Also, I learned to what extent our point of view can carry us.

The way we construct our values through our language creates our point of view. We need to know the words we use to create an adequate thesis statement, a thought. Since our language hasn't evolved at the same rate as the technology, we need to be precise and careful with how we refer to others. If not, we can hurt others and make them think they aren't capable human beings. On the other hand, today's technology proves the human capability, that everyone, no matter of his physical conditions,  is capable to achieve his goals. For this, we need to prepare everybody to adapt(our greatest human skill) to the language and thus, to the society.This way, we will open ourselves to diversity and reach the maximum potential of the human will.

Project Usability

In order to measure the usability of our research work, we prepared a virtual machine with the requirements needed to run our scenarios. We are looking for testing: Completion Rates, Usability Problems, Task Time, Task Level Satisfaction, Errors and Expectation. We have not thoroughly researched what the best way to test the usability of our work, so this time, it will work as a pilot test. For more information about the usability method, visit: this blog

We will ask users to do two exercises. Both exercises consist in write a program that calculates the Fibonacci sequence. The difference between them is that the first one will be typing and the other one will be speaking. In both we will measure the time it takes to perform each of them.

C++ scenario and the Composite plugin

I managed to make a scenario in Simon that implements basic C++ keywords and structures via voice recognition. Finding out about the Simon plugin "Composite" was a big help in order to make programming easier. It allows us to execute various commands from different plugins in a specific order. This lets us create an entire block of code and make the cursor go back to where we would normally want to continue writing the code. For instance, by saying "while", we could create this block of code:

while (  ) {
 
}

and make the cursor go back to the parenthesis in order to put the logical condition in. This opens up many other possibilities in terms of navigating and writing code.

By calling the switch text macro, going up two times, activating the "end" key, and going left four times, the cursor will end up inside the switch statement's parenthesis.

Here's the "WHILE" command using the composite plugin.

Since creating your own identifiers and strings is still not supported, variables are limited to being things like: var, index, temp and accum as of right now. With the current available keywords, I was able to open an empty c++ file, program something basic in it, and save it:

It currently doesn't return or do anything, but the simple steps of accumulating the sum of numbers is still there. 

There are still some issues though. Additional spaces and indents will obviously not be recognized by the scenario, and removing them by backspacing time and again is a drag. Additionally, the "undo" command does not work to remove blocks of code you type since Simon types things key by key. A quick solution for this is to make a shift command that holds shift to select the text you want and then backspace it, but it's still not very efficient.

I hope I could explain myself well enough... I'll be making a video explaining everything better and hopefully making a more impressive program with voice recognition and netbeans/eclipse this weekend.

Ongoing

So far we have created Simon’s scenarios that allow us to control different IDE’s such as Netbeans, Sublime, Gedit and others. We have also created scenarios for programming in different programming languages ​​such as C++ and Java. Create these scenarios has been quite straightforward thanks to Simon, but we have identified that we wanted to give some improvement to this software in order to respond to our needs. Because programming itself has a format layout and development structure, implementing innovative features into the keyboard is an integral part of the project and the intended user experience. So, We are now working on some of the Simon's plugins such the Keyboard, the Text Macro and the Shortcuts commands, in order to find a more efficient  way to create an interface that generally works on any IDE. 

Prototype

A rough prototype of a C++ scenario ... hope you like it.

PS : Sorry for my bad english :(

Install Simon successfully on Linux

We are now three students in the team. Each of us has installed Simon successfully on a virtual machine with Kubuntu 13.10. One of the member of the team, Rafael, suggested it as an environment for Simon, because this operative system is based on free software from many communities of developers including KDE software, and Simon needs some requirements from KDE. This makes Kubuntu a better environment to install Simon.

You will need to install these requirements (you can copy and paste the following command):

• sudo apt-get install git build-essential cmake gettext gettext-kde kkdeartwork libqwt-dev kdelibs5-dev libxtst-dev libqt4-sql-sqlite libphonon-dev libattica0.4 libattica-dev zlib1g-dev libasound2-dev julius

And then, you can do:

• sudo apt-get install simon simon-data

Playing with Simon on Linux

Install linux Simon was a bit challenging. After passing over high number of errors, I run the program and was able to use several downloaded models that use the HTK dictionary in English. After playing with the windows manager, keyboard and other models I could see the great potecial of SIMON