In this work two studies are done and they are referred as first study which is named ¿A Hybrid Classification Method Using Bayesian, K Nearest Neighbor Methods and Genetic Algorithm¿ and second study which is named ¿Utilization of K Nearest Neighbor Method for Expectation Maximization Based Classification Method¿. A hybrid method is formed by using k nearest neighbor (KNN), Bayesian methods and genetic algorithm (GA) together at first study. The aim is to achieve successful results on classifying by eliminating data that make difficult to learn. In second study a data elimination approach is proposed to improve data clustering. Main idea is to reduce the number of data with KNN method and to guess a class with most similar training data. KNN method considered as the preprocessor for Bayesian classifier and then the results over the data sets are investigated. Test processes are done with five of well-known University of California Irvine (UCI) machine learning data sets. These are Iris, Breast Cancer, Glass, Yeast and Wine data sets.