Tissue classification and feature selection have been increasing studied during the last two decades, however the available methods are still limited and need improvement. In this manuscript, we develop tissue classification and feature selection methods based on Dynamic Adaboost with logistic regression as its weak learner and a new VariationalBayesian (VB) logistic regression with regularization. Furthermore we investigate the statistical properties of these methods and extend VB logistic regression to handle large scale data.In chapter 1, we will introduce some key concepts like Ultrasound Tissue Classification, Level Set Segmentation method, Bayesian version of Lasso and Elastic Net and Variational Bayesian approximation. In chapter 2, we will introduce a framework of tumorsegmentation and feature extraction for ultrasound B-mode images, as well as a semi-parametricmodel for the texture features. In chapter 3, we apply the Adaboost method with logistic regression as weak learner for tumor classification. Genetic Algorithm (GA) is used for stochastic search based feature selection and the algorithm is parallelized toaccelerate the computation. In chapter 4, we propose a new variational Bayesian logisticregression incorporating the Lasso and Elastic Net type regularization for feature selection. In chapter 5, we extend the above VB logistic regression to large scale data by map/reduce cloud computing.We will illustrate the experimental results in each chapter using simulation data andultrasound image data from our research.