Why is naive Bayesian classification called “naive ?

Why is naive Bayesian classification called “naive”? Briefly outline the major ideas of naive Bayesian classification.

Naive Bayesian classification is called naive because it assumes class conditional independence. That is, the effect of an attribute value on a given class is independent of the values of the other attributes.
This assumption is made to reduce computational costs, and hence is considered “na¨ıve”. The major idea behind na¨ıve Bayesian classification is to try and classify data by maximizing P(X|Ci)P(Ci) (where i is an index of the class) using the Bayes’ theorem of posterior probability. 
In general: 
  • We are given a set of unknown data tuples, where each tuple is represented by an n-dimensional vector, X = (x1, x2. . . xn) depicting n measurements made on the tuple from n attributes, respectively A1,A2, ..,An. We are also given a set of m classes, C1,C2, . . .Cm. 
  • Using Bayes theorem, the na¨ıve Bayesian classifier calculates the posterior probability of each class conditioned on X. X is assigned the class label of the class with the maximum posterior probability conditioned on X. Therefore, we try to maximize P(Ci|X) = P(X|Ci)P(Ci)/P(X). However, since P(X) is constant for all classes, only P(X|Ci)P(Ci) need be maximized. If the class prior probabilities are not known, then it is commonly assumed that the classes are equally likely, i.e. P(C1) = P(C2) = · · · = P(Cm), and we would therefore maximize P(X|Ci). Otherwise, we maximize P(X|Ci)P(Ci). The class prior probabilities may be estimated by P(Ci) = si s , where si is the number of training tuples of class Ci, and s is the total number of training tuples. 
  • In order to reduce computation in evaluating P(X|Ci), the na¨ıve assumption of class conditional independence is made. This presumes that the values of the attributes are conditionally independent of one another, given the class label of the tuple, i.e., that there are no dependence relationships among the attributes. – If Ak is a categorical attribute then P(xk|Ci) is equal to the number of training tuples in Ci that have xk as the value for that attribute, divided by the total number of training tuples in Ci. – If Ak is a continuous attribute then P(xk|Ci) can be calculated using a Gaussian density function.

Comments

Post a Comment

Popular posts from this blog

Tree pruning useful in decision tree induction

Why is tree pruning useful in decision tree induction? What is a drawback of using a separate set of tuples to evaluate pruning? The decision tree built may overfit the training data. There could be too many branches, some of which may reflect anomalies in the training data due to noise or outliers. Tree pruning addresses this issue of overfitting the data by removing the least reliable branches (using statistical measures). This generally results in a more compact and reliable decision tree that is faster and more accurate in its classification of data. The drawback of using a separate set of tuples to evaluate pruning is that it may not be representative of the training tuples used to create the original decision tree. If the separate set of tuples are skewed, then using them to evaluate the pruned tree would not be a good indicator of the pruned tree’s classification accuracy. Furthermore, using a separate set of tuples to evaluate pruning means there are less tuples to use for ...
Read more

Machine Learning Syllabus

MACHINE LEARNING UNIT -I: The ingredients of machine learning, Tasks: the problems that can be solved with machine learning, Models: the output of machine learning, Features, the workhorses of machine learning. Binary classification and related tasks: Classification, Scoring and ranking, Class probability estimation UNIT-II: Beyond binary classification:  Handling more than two classes, Regression, Unsupervised and descriptive learning. Concept learning : The hypothesis space, Paths through the hypothesis space, Beyond conjunctive concepts UNIT- III: Tree models: Decision trees, Ranking and probability estimation trees, Tree learning as variance reduction. Rule models: Learning ordered rule lists, Learning unordered rule sets, Descriptive rule learning, First-order rule learning UNIT-IV: Linear models: The least-squares method, The perceptron: a heuristic learning algorithm for linear classifiers, Support vector machines, obtaining probabilities from linear cla...
Read more