Naive Bayes Classifier

Naive Bayes Classifier is a classification algorithm that uses the Bayes theorem for classification. It gives very good results when it comes to NLP tasks such as sentimental analysis. It is a fast and uncomplicated classification algorithm. This algorithm is quite popular to be used in NLP.

To understand the Naive Bayes Classifier, we need to understand the Bayes theorem.

Bayes Theorem is a principled way of calculating a conditional probability. Conditional probability is the probability that something will happen, given that something else has already occurred. By using conditional probability, we can find out the probability of an event will occur given the knowledge of the previous event.

  • P(A|B) — is the probability of A given that B has already happened.
  • P(B|A) — is the probability of B given that A has already happened.
  • P(A) — is the unconditional probability of A occurring.
  • P(B) — is the unconditional probability of B occurring.

Naive bayes is a supervised learning algorithm for classification so the task is to find the class of observation (data point) given the values of features. Naive bayes classifier calculates the probability of a class given a set of feature values (i.e. p(yi | x1, x2 , … , xn)). Input this into Bayes’ theorem:

Naive Bayes classifier calculates the probability of an event in the following steps:

  • Calculate the prior probability for given class labels
  • Find Likelihood probability with each attribute for each class
  • Put these value in Bayes Formula and calculate posterior probability.
  • See which class has a higher probability, given the input belongs to the higher probability class.

Types of Naive Bayes Classifier:

This is mostly used for document classification problem, i.e whether a document belongs to the category of sports, politics, technology etc. The features/predictors used by the classifier are the frequency of the words present in the document.

This is similar to the multinomial naive bayes but the predictors are boolean variables. The parameters that we use to predict the class variable take up only values yes or no, for example if a word occurs in the text or not.

Pros and Cons of Naive Bayes

  • It is easy and fast to predict a class of test data set.
  • Naive Bayes classifier performs better compare to other models assuming independence.
  • It performs well in case of categorical data as compared to numeric data.
  • It requires less training data.
  • If an instance in test data set has a category that was not present during training then it will assign it “Zero” probability and won’t be able to make prediction. This is known as Zero frequency problem.
  • It is also known as a bad estimator.
  • It solely relies on the independence predictor assumption.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store