Study of Program Synthesizers & Novice Programmers

Dhanya Jayagopal, Justin Lubin and Sarah Chasins

EECS Department
University of California, Berkeley
Technical Report No. UCB/EECS-2022-59
May 11, 2022

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-59.pdf

Novice programmers face many challenges in introductory classes, which leads to high dropout rates. Meanwhile, modern program synthesizers have progressed a lot—they now deliver on their promise of lightening the burden of programming by automatically generating code. This has led many researchers to hypothesize synthesis may have a role to play in introductory classes. However, there is little research on how to make these systems learnable not only by seasoned programmers, but also by novices. To explore the possibility of using program synthesizers in educational contexts, we must understand what makes program synthesizers learnable for novices in the first place. Furthermore, we must also understand how synthesizers affect student learning outcomes. Both chapters in this thesis focus on the application of program synthesizers in educational contexts. In Chapter 2, we present an in-depth analysis of the learnability of different publicly available program synthesizers by novice programmers with the goal of understanding how different design dimensions affect the learnability of such tools. In Chapter 3, we provide an overview of existing literature on the pedagogical impact of program synthesizers—or, rather, lack thereof. Also in Chapter 3, we contribute a user study done to evaluate the impact of our own custom program synthesizer on student learning outcomes and do not find any evidence of improved learning outcomes. This project is joint work with Justin Lubin and Sarah Chasins, and each chapter lists my specific contributions at the start.

Advisor: Sarah Chasins


BibTeX citation:

@mastersthesis{Jayagopal:EECS-2022-59,
    Author = {Jayagopal, Dhanya and Lubin, Justin and Chasins, Sarah},
    Title = {Study of Program Synthesizers & Novice Programmers},
    School = {EECS Department, University of California, Berkeley},
    Year = {2022},
    Month = {May},
    URL = {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-59.html},
    Number = {UCB/EECS-2022-59},
    Abstract = {Novice programmers face many challenges in introductory classes, which leads to  high dropout rates. Meanwhile, modern program synthesizers have progressed a lot—they now  deliver on their promise of lightening the burden of programming by automatically generating  code. This has led many researchers to hypothesize synthesis may have a role to play in  introductory classes. However, there is little research on how to make these systems learnable  not only by seasoned programmers, but also by novices. To explore the possibility of using  program synthesizers in educational contexts, we must understand what makes program  synthesizers learnable for novices in the first place. Furthermore, we must also understand how  synthesizers affect student learning outcomes. Both chapters in this thesis focus on the  application of program synthesizers in educational contexts. In Chapter 2, we present an in-depth  analysis of the learnability of different publicly available program synthesizers by novice programmers with the goal of understanding how different design dimensions affect the  learnability of such tools. In Chapter 3, we provide an overview of existing literature on the  pedagogical impact of program synthesizers—or, rather, lack thereof. Also in Chapter 3, we  contribute a user study done to evaluate the impact of our own custom program synthesizer on  student learning outcomes and do not find any evidence of improved learning outcomes. This  project is joint work with Justin Lubin and Sarah Chasins, and each chapter lists my specific  contributions at the start.}
}

EndNote citation:

%0 Thesis
%A Jayagopal, Dhanya
%A Lubin, Justin
%A Chasins, Sarah
%T Study of Program Synthesizers & Novice Programmers
%I EECS Department, University of California, Berkeley
%D 2022
%8 May 11
%@ UCB/EECS-2022-59
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-59.html
%F Jayagopal:EECS-2022-59