Zifeng Huang

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2020-13

January 10, 2020

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-13.pdf

Sketching is an effective and natural method of visual communication among engineers, artists, and designers. This thesis explores several deep-learning-driven techniques for recognizing and generating sketches. We introduce two novel systems: 1) Swire, a system for querying large repositories of design examples with sketches; and 2) Sketchforme, a system that automatically composes sketched scenes from user-specified natural language descriptions. Through the development of these systems, we introduce multiple state-of-the-art techniques to perform novel sketch understanding and generation tasks supported by these systems. We also evaluate the performance of these systems using established metrics and user studies of interactive use-cases. Our evaluations show that these systems can effectively support interactive applications and open up new avenues of human-computer interaction in the domains of art, education, design, and beyond.

Advisors: John F. Canny

---

BibTeX citation:

@mastersthesis{Huang:EECS-2020-13,
    Author= {Huang, Zifeng},
    Title= {Deep-learning-based Machine Understanding of Sketches: Recognizing and Generating Sketches with Deep Neural Networks},
    School= {EECS Department, University of California, Berkeley},
    Year= {2020},
    Month= {Jan},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-13.html},
    Number= {UCB/EECS-2020-13},
    Abstract= {Sketching is an effective and natural method of visual communication among engineers, artists, and designers. This thesis explores several deep-learning-driven techniques for recognizing and generating sketches. We introduce two novel systems: 1) Swire, a system for querying large repositories of design examples with sketches; and 2) Sketchforme, a system that automatically composes sketched scenes from user-specified natural language descriptions. Through the development of these systems, we introduce multiple state-of-the-art techniques to perform novel sketch understanding and generation tasks supported by these systems. We also evaluate the performance of these systems using established metrics and user studies of interactive use-cases. Our evaluations show that these systems can effectively support interactive applications and open up new avenues of human-computer interaction in the domains of art, education, design, and beyond.},
}

EndNote citation:

%0 Thesis
%A Huang, Zifeng 
%T Deep-learning-based Machine Understanding of Sketches: Recognizing and Generating Sketches with Deep Neural Networks
%I EECS Department, University of California, Berkeley
%D 2020
%8 January 10
%@ UCB/EECS-2020-13
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-13.html
%F Huang:EECS-2020-13