0
Research Papers

# Bayesian Annealed Sequential Importance Sampling: An Unbiased Version of Transitional Markov Chain Monte Carlo

[+] Author and Article Information
Stephen Wu

CSELab,
ETH-Zurich,
Zurich CH-8092, Switzerland
e-mail: stewu@ism.ac.jp

Panagiotis Angelikopoulos

CSELab,
ETH-Zurich,
Zurich CH-8092, Switzerland

Professor
Department of Mechanical Engineering,
University of Thessaly,
Volos 38334, Greece
e-mail: costasp@mie.uth.gr

Petros Koumoutsakos

Professor
CSELab,
ETH-Zurich,
Zurich CH-8092, Switzerland;
Department of Mechanical Engineering,
Massachusetts Institute of Technology,
Cambridge, MA 02138
e-mail: petros@ethz.ch

1Present address: The Institute of Statistical Mathematics, 10–3 Midori-cho, Tachikawa, Tokyo 190-8562, Japan.

2Corresponding author.

Manuscript received July 7, 2016; final manuscript received March 27, 2017; published online September 7, 2017. Assoc. Editor: Yan Wang.

ASME J. Risk Uncertainty Part B 4(1), 011008 (Sep 07, 2017) (13 pages) Paper No: RISK-16-1101; doi: 10.1115/1.4037450 History: Received July 07, 2016; Revised March 27, 2017

## Abstract

The transitional Markov chain Monte Carlo (TMCMC) is one of the efficient algorithms for performing Markov chain Monte Carlo (MCMC) in the context of Bayesian uncertainty quantification in parallel computing architectures. However, the features that are associated with its efficient sampling are also responsible for its introducing of bias in the sampling. We demonstrate that the Markov chains of each subsample in TMCMC may result in uneven chain lengths that distort the intermediate target distributions and introduce bias accumulation in each stage of the TMCMC algorithm. We remedy this drawback of TMCMC by proposing uniform chain lengths, with or without burn-in, so that the algorithm emphasizes sequential importance sampling (SIS) over MCMC. The proposed Bayesian annealed sequential importance sampling (BASIS) removes the bias of the original TMCMC and at the same time increases its parallel efficiency. We demonstrate the advantages and drawbacks of BASIS in modeling of bridge dynamics using finite elements and a disk-wall collision using discrete element methods.

<>

## Figures

Fig. 7

Results of running different TMCMC with burn-in period for every MCMC step. Dashed lines represent lmax = 1, and solid lines represent lmax = , i.e., the original TMCMC algorithm.

Fig. 6

Results of running TMCMC with varying lmax and β2

Fig. 1

Summary of the TMCMC algorithm

Fig. 9

Results of running different TMCMC with bimodal Gaussian likelihoods. Dashed lines represent lmax = 1 and solid lines represent lmax = , i.e., the original TMCMC algorithm.

Fig. 14

Mean estimates of running different TMCMC with approximately unidentifiable Gaussian likelihoods (six dimensions: D-1, …, D-6). Dashed lines represent lmax = 1, and solid lines represent lmax = , i.e., the original TMCMC algorithm.

Fig. 15

Standard deviation estimates of running different TMCMC with approximately unidentifiable Gaussian likelihoods (six dimensions: D-1, …, D-6). Dashed lines represent lmax = 1 and solid lines represent lmax = , i.e., the original TMCMC algorithm.

Fig. 16

Testing stagewise evidence convergence of TMCMC using 3D Gaussian likelihood with uniform prior. No burn-in in all stages.

Fig. 17

Testing stagewise evidence convergence of TMCMC using 3D Gaussian likelihood with uniform prior. Twenty steps of burn-in for each Markov chain in the first two stages only.

Fig. 18

Testing stagewise evidence convergence of TMCMC using 3D Gaussian likelihood with uniform prior. Fifty steps of burn-in for each Markov chain in the first two stages only.

Fig. 19

Testing stagewise evidence convergence of TMCMC using 3D Gaussian likelihood with uniform prior. Twenty steps of burn-in for each Markov chain in all stages.

Fig. 20

Simulation of a disk impacting at an inclined surface [17]

Fig. 21

Posterior samples for one case of the disk-wall collision example. Upper diagonal: projection of the posterior samples in all pairs of 2D parameter space (color scale corresponds to log-likelihood values of the samples). Diagonal: marginal distributions of the model parameters estimated using kernel histograms. Box-plots denote the mean and 95% percentiles. Lower diagonal: projected densities in 2D parameter space constructed via a kernel estimate (color scale corresponds to log-posterior values).

Fig. 22

Subdivided structural components of the Metsovo bridge [17]

Fig. 8

Results of running different TMCMC with various dimension Gaussian likelihoods. Dashed lines represent lmax = 1, and solid lines represent lmax = , i.e., the original TMCMC algorithm.

Fig. 5

Acceptance rate of all MCMC runs when running a single stage TMCMC on a 3D standard Gaussian likelihood with varying maximum chain length from 1 to

Fig. 4

Correlation estimates of running a single stage TMCMC on a 3D standard Gaussian likelihood with varying maximum chain length from 1 to . The black lines represent results directly after resampling and before MCMC (solid line—mean value; dashed lines—one standard deviation bound). Only the correlation between the second and the third dimension (D-2 and D-3) is shown. The other two pairs have the exact same behavior.

Fig. 3

Standard deviation estimates of running a single stage TMCMC on a 3D standard Gaussian likelihood with varying maximum chain length from 1 to . The black lines represent results directly after resampling and before MCMC (solid line—mean value; dashed lines—one standard deviation bound). Only the third dimension (D-3) is shown. The other two dimensions (D-2) have the exact same behavior.

Fig. 2

Mean estimates of running a single stage TMCMC on a 3D standard Gaussian likelihood with varying maximum chain length from 1 to . The black lines represent results directly after resampling and before MCMC (solid line—mean value; dashed lines—one standard deviation bound). Only the third dimension (D-3) is shown. The other two dimensions (D-2) have the exact same behavior.

Fig. 10

Mean estimates of running different TMCMC with bimodal Gaussian likelihoods (two dimensions: D-1 and D-2). Dashed lines represent lmax = 1, and solid lines represent lmax = , i.e., the original TMCMC algorithm.

Fig. 11

Standard deviation estimates of running different TMCMC with bimodal Gaussian likelihoods (two dimensions: D-1 and D-2). Dashed lines represent lmax = 1, and solid lines represent lmax = , i.e., the original TMCMC algorithm.

Fig. 12

Estimates for correlation and the weights’ ratio between the two Gaussian distributions when running different TMCMC with bimodal Gaussian likelihoods. Dashed lines represent lmax = 1, and solid lines represent lmax = , i.e., the original TMCMC algorithm.

Fig. 13

Results of running different TMCMC with approximately unidentifiable Gaussian likelihoods. Dashed lines represent lmax = 1, and solid lines represent lmax = , i.e., the original TMCMC algorithm.

## Discussions

Some tools below are only available to our subscribers or users with an online account.

### Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Articles from Part A: Civil Engineering
Related Proceedings Articles
Related eBook Content
Topic Collections