Estimating Continuous Distributions in Bayesian Classifiers

Abstract

Naive-Bayes classifier is a popular technique of classification in machine learning. Improving the accuracy of naive-Bayes classifier will be significant as it has great importance in classification using numerical attributes. For numeric attributes, the conditional probabilities are either modeled by some continuous probability distribution over the range of that attribute's values or by conversion of numeric attribute to discrete one using discretization. The limitation of the classifier using discretization is that it does not classify those instances for which conditional probabilities of any of the attribute value for every class is zero. The proposed method resolves this limitation of estimating probabilities in the naive-Bayes classifier and improve the classification accuracy for noisy data. The proposed method is efficient and robust in estimating probabilities in the naive-Bayes classifier. The proposed method has been tested over a number of databases of UCI machine learning repository and the comparative results of existing naive-Bayes classifier and proposed method has also been illustrated.

Keywords

• Conditional Probability
• Estimate Probability
• Noisy Data
• Test Instance
• Numeric Attribute

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

1. Aha, D., Kibler, D.: Instance-based learning algorithms. Machine Learning 6, 37–66 (1991)

   Google Scholar

2. Catlett, J.: On changing continuous attributes into ordered discrete attributes. In: Proceedings of the European Working Session on Learning, pp. 164–178 (1991)

   Google Scholar

3. Cestnik, B.: Estimating probabilities: A crucial task in machine learning. In: Proceedings of the 9th European Conference on Artificial Intelligence, pp. 147–149 (1990)

   Google Scholar

4. Domingos, P., Pazzani, M.: On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning. 29, 103–130 (1997)

   CrossRef  MATH  Google Scholar

5. Dougherty, J., Kohavi, R., Sahami, M.: Supervised and unsupervised discretization of continuous features. In: Proceedings of the Twelfth International Conference on Machine Learning, pp. 194–202 (1995)

   Google Scholar

6. Duda, R.O., Hart, P.E.: Pattern classification and scene analysis. John Wiley and Sons, New York (1973)

   MATH  Google Scholar

7. Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Machine Learning 29, 131–163 (1997)

   CrossRef  MATH  Google Scholar

8. John, G.H., Langley, P.: Estimating continuous distributions in Bayesian classifiers. In: Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, pp. 338–345 (1995)

   Google Scholar

9. Kerber, R.: Chimerge: Discretization for numeric attributes. In: National Conference on Artificial Intelligence AAAI Press, pp. 123–128 (1992)

   Google Scholar

10. Langley, P., Iba, W., Thompson, K.: An analysis of Bayesian classifiers. In: Proceedings of the Tenth National Conference on Artificial Intelligence, pp. 223–228 (1992)

    Google Scholar

11. Lu, J., Yang, Y., Webb, G.I.: Incremental Discretization for Nave-Bayes Classifier. In: Proceedings of the Second International Conference on Advanced Data Mining and Applications, pp. 223–238 (2006)

    Google Scholar

12. Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases. University of California, Irvine, CA, Department of Information and Computer Science. (1998), http://www.ics.uci.edu/mlearn/MLRepository.html

13. Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Mateo, CA (1993)

    Google Scholar

14. Yang, Y., Webb, G.: A Comparative Study of Discretization Methods for Naive-Bayes Classifiers. In: Proceedings of the Pacific Rim Knowledge Acquisition Workshop, Tokyo, Japan, pp. 159–173 (2002)

    Google Scholar

15. Yang, Y., Webb, G.: On why discretization works for naive-Bayes classifiers. In: Proceedings of the 16th Australian Joint Conference on Artificial Intelligence (AI) (2003)

    Google Scholar

Download references

Author information

Authors and Affiliations

1. Indian Institute of Technology, Delhi, Hauz Khas, New Delhi, 110 016, India

   B. Chandra & M. P. Gupta

2. Institute for Systems Studies and Analyses, Metcalfe House, Delhi, 110 054, India

   Manish Gupta

Editor information

Editors and Affiliations

Rights and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Chandra, B., Gupta, M., Gupta, M.P. (2007). Robust Approach for Estimating Probabilities in Naive-Bayes Classifier. In: Ghosh, A., De, R.K., Pal, S.K. (eds) Pattern Recognition and Machine Intelligence. PReMI 2007. Lecture Notes in Computer Science, vol 4815. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77046-6_2

Download citation

• .RIS
• .ENW
• .BIB

• DOI : https://doi.org/10.1007/978-3-540-77046-6_2

• Publisher Name: Springer, Berlin, Heidelberg

• Print ISBN: 978-3-540-77045-9

• Online ISBN: 978-3-540-77046-6

• eBook Packages: Computer Science Computer Science (R0)

• 

connlopery.blogspot.com

Source: https://link.springer.com/chapter/10.1007/978-3-540-77046-6_2

Share this post