Research Papers

Hierarchical Stochastic Model in Bayesian Inference for Engineering Applications: Theoretical Implications and Efficient Approximation

[+] Author and Article Information
Stephen Wu

CSELab ETH-Zurich,
CH-8092, Switzerland
e-mail: stewu@ism.ac.jp

Panagiotis Angelikopoulos

CSELab ETH-Zurich,
CH-8092, Switzerland

James L. Beck

Department of Mechanical and
Civil Engineering,
California Institute of Technology,
Pasadena, CA 91125
e-mail: jimbeck@caltech.edu

Petros Koumoutsakos

CSELab ETH-Zurich,
CH-8092, Switzerland
e-mail: petros@ethz.ch

1Pressent address: The Institute of Statistical Mathematics, Tokyo 190-8562, Japan.

2Pressent address: D.E. Shaw Research LLC, NY, NY 10036.

Manuscript received December 21, 2017; final manuscript received June 6, 2018; published online August 14, 2018. Assoc. Editor: Siu-Kui Au.

ASME J. Risk Uncertainty Part B 5(1), 011006 (Aug 14, 2018) (12 pages) Paper No: RISK-17-1104; doi: 10.1115/1.4040571 History: Received December 21, 2017; Revised June 06, 2018

Hierarchical Bayesian models (HBMs) have been increasingly used for various engineering applications. We classify two types of HBM found in the literature as hierarchical prior model (HPM) and hierarchical stochastic model (HSM). Then, we focus on studying the theoretical implications of the HSM. Using examples of polynomial functions, we show that the HSM is capable of separating different types of uncertainties in a system and quantifying uncertainty of reduced order models under the Bayesian model class selection framework. To tackle the huge computational cost for analyzing HSM, we propose an efficient approximation scheme based on importance sampling (IS) and empirical interpolation method (EIM). We illustrate our method using two engineering examples—a molecular dynamics simulation for Krypton and a pharmacokinetic/pharmacodynamics (PKPD) model for cancer drug.

Copyright © 2019 by ASME
Your Session has timed out. Please sign back in to continue.


Woolrich, M. W. , Behrens, T. E. J. , Beckmann, C. F. , Jenkinson, M. , and Smith, S. M. , 2004, “ Multilevel Linear Modelling for FMRI Group Analysis Using Bayesian Inference,” Neuroimage, 21(4), pp. 1732–1747. [CrossRef] [PubMed]
Fei-Fei, L. , and Perona, P. , 2005, “ A Bayesian Hierarchical Model for Learning Natural Scene Categories,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, June 20–25, pp. 524–531.
Ballesteros, G. , Angelikopoulos, P. , Papadimitriou, C. , and Koumoutsakos, P. , 2014, “ Bayesian Hierarchical Models for Uncertainty Quantification in Structural Dynamics,” Vulnerability, Uncertainty, and Risk: Quantification, Mitigation, and Management, Vol. 162, M. Beer , S. Au , and J. W. Hall , eds., American Society of Civil Engineers (ASCE), Reston, VA, pp. 1615–1624.
Hahn, P. R. , Goswami, I. , and Mela, C. F. , 2015, “ A Bayesian Hierarchical Model for Inferring Player Strategy Types in a Number Guessing Game,” Ann. Appl. Stat, 9(3), pp. 1459–1483. [CrossRef]
Sato, M. , Yoshioka, T. , Kajihara, S. , Toyama, K. , Goda, N. , Doya, K. , and Kawato, M. , 2004, “ Hierarchical Bayesian Estimation for Meg Inverse Problem,” Neuroimage, 23(3), pp. 806–826. [CrossRef] [PubMed]
Calvetti, D. , and Somersalo, E. , 2008, “ Hypermodels in the Bayesian Imaging Framework,” Inverse Probl, 24(3), p. 034013. [CrossRef]
Bae, K. , and Mallick, B. K. , 2004, “ Gene Selection Using a Two-Level Hierarchical Bayesian Model,” Bioinformatics, 20(18), pp. 3423–3430. [CrossRef] [PubMed]
Ji, S. , Xue, Y. , and Carin, L. , 2008, “ Bayesian Compressive Sensing,” IEEE Trans. Signal Process, 56(6), pp. 2346–2356. [CrossRef]
Huang, Y. , Beck, J. , Wu, S. , and Li, H. , 2014, “ Robust Bayesian Compressive Sensing for Signals in Structural Health Monitoring,” Comput.-Aided Civ. Infrastruct. Eng., 29(3), pp. 160–179. [CrossRef]
Gelman, A. , and Hill, J. , 2006, Data Analysis Using Regression and Multilevel/Hierarchical Models, Cambridge University Press, Cambridge, UK.
Tiao, G. , and Tan, W. , 1965, “ Bayesian Analysis of Random-Effect Models in Analysis of Variance. I. posterior Distribution of Variance-Components,” Biometrika, 52(1/2), pp. 37–53. [CrossRef]
Hill, B. , 1965, “ Inference About Variance Components in the One-Way Model,” J. Am. Stat. Assoc, 60(311), pp. 806–825. [CrossRef]
Congdon, P. , 2010, Applied Bayesian Hierarchical Methods, CRC Press, Boca Raton, FL.
Guha, N. , Wu, X. , Efendiev, Y. , Jin, B. , and Mallick, B. K. , 2015, “ A Variational Bayesian Approach for Inverse Problems With Skew-t Error Distributions,” J. Comput. Phys., 301, pp. 377–393. [CrossRef]
Gelman, A. , Carlin, J. , Stern, H. , and Rubin, D. , 2004, Bayesian Data Analysis, 2 ed., Chapman & Hall/CRC, Boca Raton, FL.
Barrault, M. , Maday, Y. , Nguyen, N. C. , and Patera, A. T. , 2004, “ An ‘Empirical Interpolation’ Method: Application to Efficient Reduced-Basis Discretization of Partial Differential Equations,” C. R. Math, 339(9), pp. 667–672. [CrossRef]
Beck, J. , and Yuen, K. , 2004, “ Model Selection Using Response Measurements: Bayesian Probabilistic Approach,” J. Eng. Mech.-ASCE, 130(2), pp. 192–203. [CrossRef]
Beck, J. , 2010, “ Bayesian System Identification Based on Probability Logic,” Struct. Control Health Monit., 17(7), pp. 825–847. [CrossRef]
Koller, D. , and Friedman, N. , 2009, Probabilistic Graphical Models: Principles and Techniques, MIT Press, Cambridge, MA.
Mackay, D. , 1994, “ Bayesian Non-Linear Modelling for the Prediction Competition,” ASHRAE Trans., 100(2), pp. 1053–1062.
Tipping, M. , 2004, “ Bayesian Inference: An Introduction to Principles and Practice in Machine Learning,” Advanced Lectures on Machine Learning, Vol. 3176, Springer, Berlin, pp. 41–62.
Tipping, M. , 2001, “ Sparse Bayesian Learning and the Relevance Vector Machine,” J. Mach. Learn. Res., 1(3), pp. 211–244. http://www.jmlr.org/papers/v1/tipping01a.html
Tipping, M. , and Faul, A. , 2003, “ Fast Marginal Likelihood Maximization for Sparse Bayesian Models,” Ninth International Workshop on Artificial Intelligence and Statistics, Key West, FL, Jan. 3–6.
Sargsyan, K. , Najm, H. N. , and Ghanem, R. , 2015, “ On the Statistical Calibration of Physical Models,” Int. J. Chem. Kinet, 47(4), pp. 246–276. [CrossRef]
Wu, S. , Angelikopoulos, P. , Papadimitriou, C. , Moser, R. , and Koumoutsakos, P. , 2015, “ A Hierarchical Bayesian Framework for Force Field Selection in Molecular Dynamics Simulations,” Phil. Trans. R. Soc. A, 374(2060), p. 20150032. [CrossRef]
Nagel, J. B. , and Sudret, B. , 2016, “ A Unified Framework for Multilevel Uncertainty Quantification in Bayesian Inverse Problems,” Probab. Eng. Mech., 43, pp. 68–84. [CrossRef]
Wu, S. , Angelikopoulos, P. , Tauriello, G. , Papadimitriou, C. , and Koumoutsakos, P. , 2015, “ Fusing Heterogeneous Data for the Calibration of Molecular Dynamics Force Fields Using Hierarchical Bayesian Models,” J. Chem. Phys., 145(24), p. 244112. [CrossRef]
Ching, J. , and Chen, Y. , 2007, “ Transitional Markov Chain Monte Carlo Method for Bayesian Model Updating, Model Class Selection, and Model Averaging,” J. Eng. Mech.–ASCE, 133(7), pp. 816–832. [CrossRef]
Wu, S. , Angelikopoulos, P. , Papadimitriou, C. , and Koumoutsakos, P. , 2017, “ Bayesian Annealed Sequential Importance Sampling (BASIS): An Unbiased Version of Transitional Markov Chain Monte Carlo,” ASCE-ASME J. Risk Uncertainty Part B, 4(1), p. 011008. [CrossRef]
Finley, S. , Angelikopoulos, P. , Koumoutsakos, P. , and Popel, A. , 2015, “ Pharmacokinetics of Anti-VEGF Agent Aflibercept in Cancer Predicted by Data-Driven, Molecular-Detailed Model,” CPT: Pharmacometrics Syst. Pharmacol., 4(11), pp. 641–649. [CrossRef] [PubMed]


Grahic Jump Location
Fig. 1

Graphical representations of the two HBMs: (a) hierachical prior model and (b) hierarchical stochastic model

Grahic Jump Location
Fig. 2

Three different types of contaminated data sets with x between 0 and 1. The black dash lines denote the actual function without any error.

Grahic Jump Location
Fig. 3

Three different types of contaminated data sets with x between 0.4 and 1. The black dash lines denote the actual function without any error.

Grahic Jump Location
Fig. 4

Different groupings of data sets. Different colors and marker combinations represent different data sets. The dashed lines denote the actual function without any error: (a) actual grouping, (b) grouping across x, (c) over grouping, and (d) random grouping.

Grahic Jump Location
Fig. 5

Eight data sets from y = x2 (P2) and eight data sets from y = x3 (P3). The noises are additive Gaussian error with standard deviation σ̂y=0.1.

Grahic Jump Location
Fig. 6

Posterior robust prediction for different models in different cases (sample size = 100). The dash lines denote the mean prediction and the gray area encloses 90% of the total probability density for the predicted value. Crosses are the data points. The four columns correspond to those in Table 4.

Grahic Jump Location
Fig. 7

Graphical representations of the two alternative HSMs: (a) independent additive error parameters and (b) common additive error parameters

Grahic Jump Location
Fig. 8

Results of the EIM approximation for the likelihood function. Left 2 plots: chosen bases of σy (cross), ϵLJ and σLJ (circles) for experiment 6. The grayscale contour shows the actual likelihood values for this experiment. Right plot: maximum error of the EIM estimate for each experiment as a function of the number of basis. The error is normalized by the maximum value of the actual function.

Grahic Jump Location
Fig. 9

Posterior results for the Krypton MD simulation study: (a) posterior of ϵLJ and σLJ, (b) posterior of σy, and (c) posterior-robust prediction

Grahic Jump Location
Fig. 10

Posterior distributions for parameters of the basic PK model. Upper diagonal: projection of the posterior samples for all pairs of 2D parameter space (colors indicate log-likelihood values of the samples). Diagonal: marginal distributions of the model parameters estimated using kernel histograms. Box-plots denote the means and the 5 and 95 percentiles. Lower diagonal: projected densities in 2D parameter space constructed via a kernel estimate (coloring according to log-posterior values): (a) classical Bayesian and (b) basic HSM.

Grahic Jump Location
Fig. 11

Posterior robust predictions for the plasma concentrations of aflibercept based on the basic HSM. Circles are the data points. Dark gray and light gray regions are the 50% and 90% quantile range of the posterior distribution, respectively: (a) bound aflibercept and (b) free aflibercept.



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Articles from Part A: Civil Engineering
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In