AT-SPI news

In previous posts I have explained how you can install and see Simon work perfectly on Windows only. I also expressed my concern about haven't available the AT-SPI plug-in on windows, but it was available in Fedora. It happens that the AT-SPI is an exclusive tool of Linux operating systems. Which leaves us with only two options to continue our project. Install Simon successfully on a Linux operating system that can use the AT-SPI plug-in or extended the Simon's keyboard in Qt, in order to allow that this keyboard can respond to the voice recognition instead of the mouse control.

Dictation on Simon

Over the course of the summer, Simon's developers has been working on bringing dictation capabilities to Simon. Now, They're trying to build up a network of developers and researchers that work together to build high accuracy, large vocabulary speech recognition systems for a variety of domains (desktop dictation being just one of them). Even before this project started, Simon already had a "Dictation" command plugin. Basically, this plugin would just write out everything that Simon recognizes. But that's far from everything there is to dictation from a software perspective.

First of all, is necessary to take care of replacing the special words used for punctuation, like ".period", with their associated signs. To do that, Simon's developers implemented a configurable list of string replaces in the dictation plugin. Dictation will improve our project with better recognition of the vocabulary that we'll have between Caribou and Simon. 

Simon Installation 3: Windows: It Works

I'm just tried to install the compiled version of Simon on windows and it's working pretty well. Finally, Simon does what I tell him to do. Also, I made this video: Testing Simon. In this video I was just trying it and, surprisingly, Simon wrote in the browser the commands that I gave. "show places" and "programs" commands did not work, because they are commands of scenarios that I'm not installed yet.

The thing here is that the Simon's scenarios doesn't have the AT-SPI plug-in that allow us to connect Simon with another application that have AT-SPI as a plug-in too. In order to get the connection between these application we have to, in some way, enable this plug-in in the windows version of Simon.

Also, to turning Simon into a real dictation system, first of all, the Simon's developers needed to hack Simond a bit to accept and use an n-gram based language model instead of the scenarios grammar when the first was available. With this little bit of trickery, Simon was already able to use the models they built in the last weeks. More info here.

Simon Installation 2: Fedora 18 & 19: Almost Failed

In Fedora 18 I tried only to get it through yum search simon and it didn't found it. So I tried with Fedora 19 and it's work.

Steps to install the compiled application:

1. get it through yum search simon 
2. when it found the packages, do: yum install simon simon-doc simon-l10 

After the installation, I downloaded some scenarios to Simon and I trained it to se them. Also, the scenarios have the AT-SPI plug-in that we need to connect Simon with Caribou. But, when I tried to compile the trained scenarios and tried to connect Simon with it server I got an error: //screenshot

Simon Installation 1: Ubuntu 13.04: Failed

Steps to install it from source files: 

1. download the source file: git clone git://anont.kde.org/simon 
2. run this command 

          ./build_ubuntu.sh

And I got an error with the Qt installation that will break plugins and linking of some applications.

Steps to install the compiled application: 

1. get it through apt-cache search search simon 
2. when it found the packages, do: apt-get install simon simon-doc simon-l10n

And I got an error with the Simon's scenarios:

I realized that it can’t be installed in Ubuntu and also, I found it in this blog that say: maybe the current package is still broken

meet Simon

The speech recognition that we are using for the research is Simon. It is an open source project and also very complete in the sense that it have plug-ins like AT-SPI, dictation (in process) and a keyboard, among other things. See Simon in action. When I was trying to install Simon, I tried four different OS. Ubuntu 13.04, Fedora 18, Fedora 19 and W indows. I tried the two ways that I know to do it: downloading the source files to build it and downloading the application compiled. I’ll put it here as individual posts, with the steps that I followed and the errors that I has.

Presenting at Carnegie Mellon University

As part of a workshop, I was at CMU this October. I had the opportunity to present our idea to the faculty, graduate students and the student of the OurCS 2013 program. The poster consists of the conceptual idea without being developed and without final results. Which is fine to be presented at a workshop. This was the poster that I presented:

Also, I was working in a research during those three days on speech recognition, with Alan Black. We were working on automated methods for building new voices for speech synthesis. I synthesized my own sentences with Spanish accent. This helped me a lot to understand how Simon works. Besides, Simon used as a dependency a software that was created at CMU, Sphinx. Although Simon users do not have to worry about this process that happens internally when you train it.

We are currently developing the proof of concept between Simon and Caribou through AT-SPI. Because we are working with open source projects and we are also in mid-semester, we have taken time to prepare the environment to install both projects. By the end of the semester, at least, we'll have a fully functional Simon with applications that are compatible through the AT-SPI plugin as gedit.

Technical Caribou installation steps in Fedora and Ubuntu

Fedora 19
1. Create virtual machines using VMware using Fedora 19
2. Tried to install AT-SPI2 in Fedora and it affected the yum functionality.  Finally reinstalled Fedora. We did a yum update command for this.
3. Created a directory in home folder for the project and proceed to download a clone of the Caribou git repository. It downloaded all the source files.
4. Proceed to install gnome-common package with all it's dependencies (yum install gnome-common)


Caribou git

Ubuntu 12.0.4
1. Used same virtual environment to create another virtual machine with Ubuntu 12.0.4
2. run apt-get update command
3. Created a directory in home folder for the project and proceed to download a clone of the Caribou git repository. It downloaded all the source files.
4. Proceed to install gnome-common package with all it's dependencies (apt-get install gnome-common)



Next Steps:
1. Run the ./autogen.sh command to generate a pre-compiled version of Caribou*
2. Install any dependencies required to complete this task successfully
3. Compile the Caribou source code
4.Test the compiled version

*Read the config file and proceed to install all dependency packages before running the ./autogen.sh command on each system respectively.

Free vs Open Source