Jeremy Ferguson

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2024-83

May 10, 2024

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-83.pdf

While Programming By Example (PBE) holds the promise of putting programming tasks in reach for non-programmer domain experts, PBE for complex programs often requires a large number of examples. We aim to bring the benefits of PBE to domains in which providing sufficient examples is prohibitively tedious or time-consuming. We build our approach around the insight that users often have useful domain knowledge that input-output examples can express only indirectly. We introduce a synthesis framework that accepts as input both: (i)~traditional labeled examples and (ii)~decisions about whether particular domain concepts are relevant to the task at hand. This novel synthesis framework allows us to explore the tradeoff between providing examples vs. domain knowledge. For a sample domain, we find that a difference of 8 additional domain concept decisions improves synthesized programs as much as 80 additional input-output examples. To our knowledge, this is the first synthesis approach that brings together examples and explicitly elicited and checked domain knowledge; we hope it opens the door to additional approaches for this new flavor of multi-modal program synthesis.

Advisors: Sarah Chasins

---

BibTeX citation:

@mastersthesis{Ferguson:EECS-2024-83,
    Author= {Ferguson, Jeremy},
    Title= {Eliciting Domain Expertise Reduces Examples Needed for Program Synthesis},
    School= {EECS Department, University of California, Berkeley},
    Year= {2024},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-83.html},
    Number= {UCB/EECS-2024-83},
    Abstract= {While Programming By Example (PBE) holds the promise of putting programming tasks in reach for non-programmer domain experts, PBE for complex programs often requires a large number of examples. We aim to bring the benefits of PBE to domains in which providing sufficient examples is prohibitively tedious or time-consuming. We build our approach around the insight that users often have useful domain knowledge that input-output examples can express only indirectly. We introduce a synthesis framework that accepts as input both: (i)~traditional labeled examples and (ii)~decisions about whether particular domain concepts are relevant to the task at hand. This novel synthesis framework allows us to explore the tradeoff between providing examples vs. domain knowledge. For a sample domain, we find that a difference of 8 additional domain concept decisions improves synthesized programs as much as 80 additional input-output examples. To our knowledge, this is the first synthesis approach that brings together examples and explicitly elicited and checked domain knowledge; we hope it opens the door to additional approaches for this new flavor of multi-modal program synthesis.},
}

EndNote citation:

%0 Thesis
%A Ferguson, Jeremy 
%T Eliciting Domain Expertise Reduces Examples Needed for Program Synthesis
%I EECS Department, University of California, Berkeley
%D 2024
%8 May 10
%@ UCB/EECS-2024-83
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-83.html
%F Ferguson:EECS-2024-83