Daniel Fremont and Xiangyu Yue and Tommaso Dreossi and Shromona Ghosh and Alberto L. Sangiovanni-Vincentelli and Sanjit A. Seshia

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2018-8

April 18, 2018

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-8.pdf

Synthetic data has proved increasingly useful in both training and testing machine learning models such as neural networks. The major problem in synthetic data generation is producing meaningful data that is not simply random but reflects properties of real-world data or covers particular cases of interest. In this paper, we show how a probabilistic programming language can be used to guide data synthesis by encoding domain knowledge about what data is useful. Specifically, we focus on data sets arising from scenes, configurations of physical objects: for example, images of cars on a road. We design a domain-specific language, Scenic, for describing scenarios that are distributions over scenes. The syntax of Scenic makes it easy to specify complex relationships between the positions and orientations of objects. As a probabilistic programming language, Scenic allows assigning distributions to features of the scene, as well as declaratively imposing hard and soft constraints over the scene. A Scenic scenario thereby implicitly defines a distribution over scenes, and we formulate the problem of sampling from this distribution as scene improvisation.We implement an improviser for Scenic scenarios and apply it in a case study generating synthetic data sets for a convolutional neural network designed to detect cars in road images. Our experiments demonstrate the usefulness of our approach by using Scenic to analyze and improve the performance of the network in various scenarios.

---

BibTeX citation:

@techreport{Fremont:EECS-2018-8,
    Author= {Fremont, Daniel and Yue, Xiangyu and Dreossi, Tommaso and Ghosh, Shromona and Sangiovanni-Vincentelli, Alberto L. and Seshia, Sanjit A.},
    Title= {Scenic: Language-Based Scene Generation},
    Year= {2018},
    Month= {Apr},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-8.html},
    Number= {UCB/EECS-2018-8},
    Abstract= {Synthetic data has proved increasingly useful in both training and
testing machine learning models such as neural networks. The major
problem in synthetic data generation is producing meaningful
data that is not simply random but reflects properties of real-world
data or covers particular cases of interest. In this paper, we show
how a probabilistic programming language can be used to guide data
synthesis by encoding domain knowledge about what data is useful.
Specifically, we focus on data sets arising from scenes, configurations
of physical objects: for example, images of cars on a road. We
design a domain-specific language, Scenic, for describing scenarios
that are distributions over scenes. The syntax of Scenic makes it
easy to specify complex relationships between the positions and
orientations of objects. As a probabilistic programming language,
Scenic allows assigning distributions to features of the scene, as
well as declaratively imposing hard and soft constraints over the
scene. A Scenic scenario thereby implicitly defines a distribution
over scenes, and we formulate the problem of sampling from this
distribution as scene improvisation.We implement an improviser for
Scenic scenarios and apply it in a case study generating synthetic
data sets for a convolutional neural network designed to detect cars
in road images. Our experiments demonstrate the usefulness of our
approach by using Scenic to analyze and improve the performance
of the network in various scenarios.},
}

EndNote citation:

%0 Report
%A Fremont, Daniel 
%A Yue, Xiangyu 
%A Dreossi, Tommaso 
%A Ghosh, Shromona 
%A Sangiovanni-Vincentelli, Alberto L. 
%A Seshia, Sanjit A. 
%T Scenic: Language-Based Scene Generation
%I EECS Department, University of California, Berkeley
%D 2018
%8 April 18
%@ UCB/EECS-2018-8
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-8.html
%F Fremont:EECS-2018-8