High Dimensional Markov Chain Monte Carlo with Multiple GPUs

Bayesian Neural Network;Hamiltonian Monte Carlo;Image Classification;Bridge Sampling;GPU;Parallelization;519.5

자연과학대학 통계학과 ;

University:서울대학교 대학원

关键词: Bayesian Neural Network;    Hamiltonian Monte Carlo;    Image Classification;    Bridge Sampling;    GPU;    Parallelization;    519.5;   

Others  :  http://s-space.snu.ac.kr/bitstream/10371/142470/1/000000149694.pdf

美国|英语

来源: Seoul National University Open Repository

【 摘 要 】

Allocating computation over multiple threads to reduce running time has become a key to training big models such as deep neural networks because a Graphics Processing Unit (GPU), which is parallel in nature, can speed up intensive matrix operations. We present a new MCMC algorithm that can be distributed over multiple GPUs by combining bridge sampling with Hamiltonian Monte Carlo on partitioned sample spaces. We empirically show that this approach can expedite MCMC sampling for any unnormalized target distribution such as Bayesian Neural Network in a high dimensional setting. Furthermore, in the presence of multimodality, this algorithm is expected to be more efficient in mixing MCMC chains when proper partitions are chosen. Finally, by comparing the parameter distributions of different learning method, we suggest that further studies could be conducted on the effect of a constrained sample space on the generalization error.