Andrew Begel

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2006-8

January 26, 2006

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-8.pdf

Programmers who suffer from repetitive stress injuries find it difficult to program by typing. Speech interfaces can reduce the amount of typing, but existing programming-by-voice techniques make it awkward for programmers to enter and edit program text. We used a human-centric approach to address these problems. We first studied how programmers verbalize code, and found that spoken programs contain lexical, syntactic and semantic ambiguities that do not appear in written programs. Using the results from this study, we designed Spoken Java, a semantically identical variant of Java that is easier to speak. Inspired by a study of how voice recognition users navigate through documents, we developed a novel program navigation technique that can quickly take a software developer to a desired program position.

Spoken Java is analyzed by extending a conventional Java programming language analysis engine written in our Harmonia program analysis framework. Our new XGLR parsing framework extends GLR parsing to process the input stream ambiguities that arise from spoken programs (and from embedded languages). XGLR parses Spoken Java utterances into their many possible interpretations. To semantically analyze these interpretations and discover which ones are legal, we implemented and extended the Inheritance Graph, a semantic analysis formalism which supports constant-time access to type and use-definition information for all names defined in a program. The legal interpretations are the ones most likely to be correct, and can be presented to the programmer for confirmation.

We built an Eclipse IDE plugin called SPEED (for SPEech EDitor) to support the combination of Spoken Java, an associated command language, and a structure-based editing model called Shorthand. Our evaluation of this software with expert Java developers showed that most developers had little trouble learning to use the system, but found it slower than typing.

Although programming-by-voice is still in its infancy, it has already proved to be a viable alternative to typing for those who rely on voice recognition to use a computer. In addition, by providing an alternative means of programming a computer, we can learn more about how programmers communicate about code.

Advisors: Susan L. Graham

---

BibTeX citation:

@phdthesis{Begel:EECS-2006-8,
    Author= {Begel, Andrew},
    Title= {Spoken Language Support for Software Development},
    School= {EECS Department, University of California, Berkeley},
    Year= {2006},
    Month= {Jan},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-8.html},
    Number= {UCB/EECS-2006-8},
    Abstract= {Programmers who suffer from repetitive stress injuries find it  difficult to program by typing. Speech interfaces can reduce the amount of typing, but existing programming-by-voice techniques make  it awkward for programmers to enter and edit program text. We used a human-centric approach to address these problems. We first studied  how programmers verbalize code, and found that spoken programs contain lexical, syntactic and semantic ambiguities that do not appear in  written programs. Using the results from this study, we designed Spoken Java, a semantically identical variant of Java that is easier  to speak. Inspired by a study of how voice recognition users navigate through documents, we developed a novel program navigation technique  that can quickly take a software developer to a desired program position.

Spoken Java is analyzed by extending a conventional Java programming  language analysis engine written in our Harmonia program analysis framework. Our new XGLR parsing framework extends GLR parsing to  process the input stream ambiguities that arise from spoken programs (and from embedded languages). XGLR parses Spoken Java utterances  into their many possible interpretations. To semantically analyze these interpretations and discover which ones are legal, we implemented and  extended the Inheritance Graph, a semantic analysis formalism which supports constant-time access to type and use-definition information  for all names defined in a program. The legal interpretations are the ones most likely to be correct, and can be presented to the  programmer for confirmation.

We built an Eclipse IDE plugin called SPEED (for SPEech EDitor) to  support the combination of Spoken Java, an associated command language, and a structure-based editing model called Shorthand. Our  evaluation of this software with expert Java developers showed that most developers had little trouble learning to use the system, but  found it slower than typing.

Although programming-by-voice is still in its infancy, it has already  proved to be a viable alternative to typing for those who rely on voice recognition to use a computer. In addition, by providing an  alternative means of programming a computer, we can learn more about how programmers communicate about code.},
}

EndNote citation:

%0 Thesis
%A Begel, Andrew 
%T Spoken Language Support for Software Development
%I EECS Department, University of California, Berkeley
%D 2006
%8 January 26
%@ UCB/EECS-2006-8
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-8.html
%F Begel:EECS-2006-8