Robust Naive Bayes

Aditya Mishra

EECS Department
University of California, Berkeley
Technical Report No. UCB/EECS-2021-66
May 13, 2021

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-66.pdf

Robustness of deep learning methods remains an open issue in a variety of NLP tasks due to the inherent complexity of neural networks. In this paper, we focus on a simple, yet effective model for large-scale text classification: Multinomial Naive Bayes (MNB). In this work, we derive the robust counterpart to MNB, Robust Naive Bayes (RNB), in different adversarial settings that are relevant to text. We compare the robustness of our model against SVM, logistic regression and neural networks in a variety of settings. Our results show that RNB is comparable to other models under random perturbations but vastly outperforms them against targeted attacks. We describe an algorithm for training our model which is orders of magnitude faster than the training time of more complex models.

Advisor: Laurent El Ghaoui


BibTeX citation:

@mastersthesis{Mishra:EECS-2021-66,
    Author = {Mishra, Aditya},
    Title = {Robust Naive Bayes},
    School = {EECS Department, University of California, Berkeley},
    Year = {2021},
    Month = {May},
    URL = {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-66.html},
    Number = {UCB/EECS-2021-66},
    Abstract = {Robustness of deep learning methods remains an open issue in a variety of NLP tasks due to the inherent complexity of neural networks. In this paper, we focus on a simple, yet effective model for large-scale text classification: Multinomial Naive Bayes (MNB). In this work, we derive the robust counterpart to MNB, <i>Robust Naive Bayes</i> (RNB), in different adversarial settings that are relevant to text. We compare the robustness of our model against SVM, logistic regression and neural networks in a variety of settings. Our results show that RNB is comparable to other models under random perturbations but vastly outperforms them against targeted attacks. We describe an algorithm for training our model which is orders of magnitude faster than the training time of more complex models.}
}

EndNote citation:

%0 Thesis
%A Mishra, Aditya
%T Robust Naive Bayes
%I EECS Department, University of California, Berkeley
%D 2021
%8 May 13
%@ UCB/EECS-2021-66
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-66.html
%F Mishra:EECS-2021-66