Forrest Iandola

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2016-207

December 16, 2016

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-207.pdf

In recent years, the research community has discovered that deep neural networks (DNNs) and convolutional neural networks (CNNs) can yield higher accuracy than all previous solutions to a broad array of machine learning problems. To our knowledge, there is no single CNN/DNN architecture that solves all problems optimally. Instead, the “right” CNN/DNN architecture varies depending on the application at hand. CNN/DNNs comprise an enormous design space. Quantitatively, we find that a small region of the CNN design space contains 30 billion different CNN architectures.

In this dissertation, we develop a methodology that enables systematic exploration of the design space of CNNs. Our methodology is comprised of the following four themes.

1. Judiciously choosing benchmarks and metrics. 2. Rapidly training CNN models. 3. Defining and describing the CNN design space. 4. Exploring the design space of CNN architectures.

Taken together, these four themes comprise an effective methodology for discovering the “right” CNN architectures to meet the needs of practical applications.

Advisors: Kurt Keutzer

---

BibTeX citation:

@phdthesis{Iandola:EECS-2016-207,
    Author= {Iandola, Forrest},
    Title= {Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale},
    School= {EECS Department, University of California, Berkeley},
    Year= {2016},
    Month= {Dec},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-207.html},
    Number= {UCB/EECS-2016-207},
    Abstract= {In recent years, the research community has discovered that deep neural networks (DNNs) and convolutional neural networks (CNNs) can yield higher accuracy than all previous solutions to a broad array of machine learning problems. To our knowledge, there is no single CNN/DNN architecture that solves all problems optimally. Instead, the “right” CNN/DNN architecture varies depending on the application at hand. CNN/DNNs comprise an enormous design space. Quantitatively, we find that a small region of the CNN design space contains 30 billion different CNN architectures.

In this dissertation, we develop a methodology that enables systematic exploration of the design space of CNNs. Our methodology is comprised of the following four themes.

1. Judiciously choosing benchmarks and metrics.
2. Rapidly training CNN models.
3. Defining and describing the CNN design space. 
4. Exploring the design space of CNN architectures.

Taken together, these four themes comprise an effective methodology for discovering the “right” CNN architectures to meet the needs of practical applications.},
}

EndNote citation:

%0 Thesis
%A Iandola, Forrest 
%T Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale
%I EECS Department, University of California, Berkeley
%D 2016
%8 December 16
%@ UCB/EECS-2016-207
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-207.html
%F Iandola:EECS-2016-207