Adith Balamurugan and Avideh Zakhor

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2019-97

May 24, 2019

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-97.pdf

Building floor plans with locations of safety, security and energy assets such as IoT sensors, thermostats, fire sprinklers, EXIT signs, fire alarms, smoke detectors, routers etc. are vital for climate control, emergency response, security, safety, and maintenance of building infrastructure. Existing approaches to building survey are tedious, error prone, and usually involve an operator with a clipboard and pen, or a tablet enumerating and localizing assets in each room. We propose an interactive method for a human operator to use an app on a smart phone to expedite such a task. One major component of this semi-automated building survey method is to accurately detect and classify assets of interest. Our approach is to use deep learning methods to train a neural network to recognize assets of interest, and uses human-in-the-loop interactive methods to correct erroneous recognition. These corrections serve to improve the accuracy of the model over time as more assets are recorded. A major hurdle faced when classifying via this method is that the appearance of a single type of asset, e.g. power outlet, can vary greatly from building to building or even from room to room, so a high rate of human correction is required to accurately recognize every asset of interest, since the model is unable to adapt in real time. In this thesis, we propose an online "one-shot learning" approach which combines aspects of long-term and short-term memory to minimize the amount of human correction required for accurate asset detection, even in situations involving never-before-seen asset appearances. We use a Neural Turing Machine (NTM) architecture, a type of Memory Augmented Neural Network (MANN), with augmented memory capacity which allows us to rapidly incorporate new data into our model to improve prediction accuracy after only a few examples, all without compromising the ability to remember previously learned data. This approach greatly improves the training time needed to update the model between building survey sessions. Experiments show that our proposed method matches and sometimes outperforms the prediction accuracy attained using more traditional batch processing deep learning methods where new data is used in conjunction with all old data to train the model. The advantage is especially pronounced for assets in new buildings that the model has never seen prior.

Advisors: Avideh Zakhor

---

BibTeX citation:

@mastersthesis{Balamurugan:EECS-2019-97,
    Author= {Balamurugan, Adith and Zakhor, Avideh},
    Title= {Online One-Shot Learning for Indoor Asset Detection},
    School= {EECS Department, University of California, Berkeley},
    Year= {2019},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-97.html},
    Number= {UCB/EECS-2019-97},
    Abstract= {Building floor plans with locations of safety, security and energy assets such as IoT sensors, thermostats, fire sprinklers, EXIT signs, fire alarms, smoke detectors, routers etc. are vital for climate control, emergency response, security, safety, and maintenance of building infrastructure. Existing approaches to building survey are tedious, error prone, and usually involve an operator with a clipboard and pen, or a tablet enumerating and localizing assets in each room. We propose an interactive method for a human operator to use an app on a smart phone to expedite such a task. One major component of this semi-automated building survey method is to accurately detect and classify assets of interest. Our approach is to use deep learning methods to train a neural network to recognize assets of interest, and uses human-in-the-loop interactive methods to correct erroneous recognition. These corrections serve to improve the accuracy of the model over time as more assets are recorded. A major hurdle faced when classifying via this method is that the appearance of a single type of asset, e.g. power outlet, can vary greatly from building to building or even from room to room, so a high rate of human correction is required to accurately recognize every asset of interest, since the model is unable to adapt in real time. In this thesis, we propose an online "one-shot learning" approach which combines aspects of long-term and short-term memory to minimize the amount of human correction required for accurate asset detection, even in situations involving never-before-seen asset appearances. We use a Neural Turing Machine (NTM) architecture, a type of Memory Augmented Neural Network (MANN), with augmented memory capacity which allows us to rapidly incorporate new data into our model to improve prediction accuracy after only a few examples, all without compromising the ability to remember previously learned data. This approach greatly improves the training time needed to update the model between building survey sessions. Experiments show that our proposed method matches and sometimes outperforms the prediction accuracy attained using more traditional batch processing deep learning methods where new data is used in conjunction with all old data to train the model. The advantage is especially pronounced for assets in new buildings that the model has never seen prior.},
}

EndNote citation:

%0 Thesis
%A Balamurugan, Adith 
%A Zakhor, Avideh 
%T Online One-Shot Learning for Indoor Asset Detection
%I EECS Department, University of California, Berkeley
%D 2019
%8 May 24
%@ UCB/EECS-2019-97
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-97.html
%F Balamurugan:EECS-2019-97