0
Research Papers

Optimal Experimental Design Using a Consistent Bayesian Approach

[+] Author and Article Information
Scott N. Walsh

Department of Mathematical
and Statistical Sciences,
University of Colorado Denver,
Denver, CO 80204
e-mail: scott.walsh@ucdenver.edu

Tim M. Wildey

Sandia National Laboratories,
Center for Computing Research,
Albuquerque, NM 87123
e-mail: tmwilde@sandia.gov

John D. Jakeman

Sandia National Laboratories,
Center for Computing Research,
Albuquerque, NM 87123
e-mail: jdjakem@sandia.gov

Manuscript received October 4, 2016; final manuscript received June 26, 2017; published online September 7, 2017. Assoc. Editor: Yan Wang.The United States Government retains, and by accepting the article for publication, the publisher acknowledges that the United States Government retains, a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this work, or allow others to do so, for United States Government purposes.

ASME J. Risk Uncertainty Part B 4(1), 011005 (Sep 07, 2017) (19 pages) Paper No: RISK-16-1132; doi: 10.1115/1.4037457 History: Received October 04, 2016; Revised June 26, 2017

We consider the utilization of a computational model to guide the optimal acquisition of experimental data to inform the stochastic description of model input parameters. Our formulation is based on the recently developed consistent Bayesian approach for solving stochastic inverse problems, which seeks a posterior probability density that is consistent with the model and the data in the sense that the push-forward of the posterior (through the computational model) matches the observed density on the observations almost everywhere. Given a set of potential observations, our optimal experimental design (OED) seeks the observation, or set of observations, that maximizes the expected information gain from the prior probability density on the model parameters. We discuss the characterization of the space of observed densities and a computationally efficient approach for rescaling observed densities to satisfy the fundamental assumptions of the consistent Bayesian approach. Numerical results are presented to compare our approach with existing OED methodologies using the classical/statistical Bayesian approach and to demonstrate our OED on a set of representative partial differential equations (PDE)-based models.

Copyright © 2018 by ASME
Your Session has timed out. Please sign back in to continue.

References

Cox, D. R. , and Reid, N. , 2000, The Theory of the Design of Experiments, CRC Press, Boca Raton, FL.
Fisher, R. A. , 1966, The Design of Experiments, 8th ed., Oliver & Boyd, Edinburgh, UK.
Pazman, A. , 1986, Foundations of Optimum Experimental Design, D. Reidel Publishing, Dordrecht, The Netherlands.
Pukelsheim, F. , 1993, Optimal Design of Experiments, Wiley, New York.
Ucinski, D. , 2005, Optimal Measurement Methods for Distributed Parameter System Identification, CRC Press, Boca Raton, FL.
Korkel, S. , Kostina, E. , Bock, H. G. , and Schloder, J. P. , 2004, “ Numerical Methods for Optimal Control Problems in Design of Robust Optimal Experiments for Nonlinear Dynamic Processes,” Optim. Methods Software, 19(3–4), pp. 327–338. [CrossRef]
Chung, M. , and Haber, E. , 2012, “ Experimental Design for Biological Systems,” SIAM J. Control Optim., 50(1), pp. 471–489. [CrossRef]
Bock, H. G. , Körkel, S. , and Schlöder, J. P. , 2013, “ Parameter Estimation and Optimum Experimental Design for Differential Equation Models,” Model Based Parameter Estimation, Springer, Berlin, pp. 1–30. [CrossRef]
Horesh, L. , Haber, E. , and Tenorio, L. , 2010, “ Optimal Experimental Design for the Large-Scale Nonlinear Ill-Posed Problem of Impedance Imaging,” Large-Scale Inverse Problems and Quantification of Uncertainty, Wiley, Chichester, UK, pp. 273–290. [CrossRef]
Bauer, I. , Bock, H. G. , Körkel, S. , and Schlöder, J. P. , 2000, “ Numerical Methods for Optimum Experimental Design in DAE Systems,” J. Comput. Appl. Math., 120(1–2), pp. 1–25. [CrossRef]
Haber, E. , Magnant, Z. , Lucero, C. , and Tenorio, L. , 2012, “ Numerical Methods for A-Optimal Designs With a Sparsity Constraint for Ill-Posed Inverse Problems,” Comput. Optim. Appl., 52(1), pp. 293–314. [CrossRef]
Alexanderian, A. , Petra, N. , Stadler, G. , and Ghattas, O. , 2014, “ A-Optimal Design of Experiments for Infinite-Dimensional Bayesian Linear Inverse Problems With Regularized ℓ0-Sparsification,” SIAM J. Sci. Comput., 36(5), pp. A2122–A2148. [CrossRef]
Haber, E. , Horesh, L. , and Tenorio, L. , 2008, “ Numerical Methods for Experimental Design of Large-Scale Linear Ill-Posed Inverse Problems,” Inverse Probl., 24(5), p. 055012. [CrossRef]
Alexanderian, A. , Petra, N. , Stadler, G. , and Ghattas, O. , 2016, “ A Fast and Scalable Method for A-Optimal Design of Experiments for Infinite-Dimensional Bayesian Nonlinear Inverse Problems,” SIAM J. Sci. Comput., 38(1), pp. A243–A272. [CrossRef]
Atkinson, A. , and Donev, A. , 1992, Optimum Experimental Designs, Oxford University Press, Oxford, UK.
Chaloner, K. , and Verdinelli, I. , 1995, “ Bayesian Experimental Design: A Review,” Stat. Sci., 10(3), pp. 273–304. [CrossRef]
Long, Q. , Scavino, M. , Tempone, R. , and Wang, S. , 2015, “ A Laplace Method for Under-Determined Bayesian Optimal Experimental Designs,” Comput. Methods Appl. Mech. Eng., 285, pp. 849–876. [CrossRef]
Loredo, T. J. , and Chernoff, D. F. , 2003, Bayesian Adaptive Exploration, Springer, New York, pp. 57–70.
Müller, P. , Sansó, B. , and Iorio, M. D. , 2004, “ Optimal Bayesian Design by Inhomogeneous Markov Chain Simulation,” J. Am. Stat. Assoc., 99(467), pp. 788–798. [CrossRef]
Solonen, A. , Haario, H. , and Laine, M. , 2012, “ Simulation-Based Optimal Design Using a Response Variance Criterion,” J. Comput. Graphical Stat., 21(1), pp. 234–252. [CrossRef]
Long, Q. , Scavino, M. , Tempone, R. , and Wang, S. , 2013, “ Fast Estimation of Expected Information Gains for Bayesian Experimental Designs Based on Laplace Approximations,” Comput. Methods Appl. Mech. Eng., 259, pp. 24–39. [CrossRef]
Haber, E. , Horesh, L. , and Tenorio, L. , 2010, “ Numerical Methods for the Design of Large-Scale Nonlinear Discrete Ill-Posed Inverse Problems,” Inverse Probl., 26(2), p. 025002. [CrossRef]
Butler, T. , Jakeman, J. D. , and Wildey, T. , 2016, “ A Consistent Bayesian Formulation for Stochastic Inverse Problems Based on Push-Forward Measures,” preprint arXiv:1704.00680. https://arxiv.org/abs/1704.00680
Huan, X. , 2015, “ Numerical Approaches for Sequential Bayesian Optimal Experimental Design,” Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA https://dspace.mit.edu/handle/1721.1/101442.
Stuart, A. M. , 2010, “ Inverse Problems: A Bayesian Perspective,” Acta Numer., 19(5), pp. 451–559. [CrossRef]
Breidt, J. , Butler, T. , and Estep, D. , 2012, “ A Measure-Theoretic Computational Method for Inverse Sensitivity Problems I: Method and Analysis,” SIAM J. Numer. Anal., 49(5), pp. 1836–1859. [CrossRef]
Butler, T. , Pilosov, M. , and Walsh, S. , 2016, “ Simulation-Based Optimal Experimental Design: A Measure-Theoretic Perspective.” SIAM J. Sci. Comput. (in press).
Sargsyan, K. , Najm, H. N. , and Ghanem, R. , 2015, “ On the Statistical Calibration of Physical Models,” Int. J. Chem. Kinet., 47(4), pp. 246–276. [CrossRef]
Kennedy, M. C. , and O'Hagan, A. , 2001, “ Bayesian Calibration of Computer Models,” J. R. Stat. Soc., Ser. B, 63(3), pp. 425–464. [CrossRef]
Huan, X. , and Marzouk, Y. M. , 2013, “ Simulation-Based Optimal Bayesian Experimental Design for Nonlinear Systems,” J. Comput. Phys., 232(1), pp. 288–317. [CrossRef]
Chaloner, K. , and Verdinelli, I. , 1995, “ Bayesian Experimental Design: A Review,” Inst. Math. Stat., 10(3), pp. 273–304.
Huan, X. , and Marzouk, Y. , 2014, “ Gradient-Based Stochastic Optimization Methods in Bayesian Experimental Design,” Int. J. Uncertainty Quantif., 4(6), pp. 479–510. [CrossRef]
Tarantola, A. , 2005, Inverse Problem Theory and Methods for Model Parameter Estimation, SIAM, Philadelphia, PA. [CrossRef]
Wand, M. , and Jones, M. , 1994, “ Multivariate Plug-In Bandwidth Selection,” Comput. Stat., 9(2), pp. 97–116.
Kullback, S. , and Leibler, R. A. , 1951, “ On Information and Sufficiency,” Ann. Math. Stat., 22(1), pp. 79–86. [CrossRef]
van Erven, T. , and Harremoes, P. , 2014, “ Rényi Divergence and Kullback–Leibler Divergence,” IEEE Trans. Inf. Theory, 60(7), pp. 3797–3820. [CrossRef]
Cover, T. A. , and Thomas, J. A. , 2006, Elements of Information Theory, Wiley, Chichester, UK.
Bisetti, F. , Kim, D. , Knio, O. , Long, Q. , and Tempone, R. , 2016, “ Optimal Bayesian Experimental Design for Priors of Compact Support With Application to Shock-Tube Experiments for Combustion Kinetics,” Int. J. Numer. Methods Eng., 108(2), pp. 136–155. [CrossRef]
Zhang, D. , and Lu, Z. , 2004, “ An Efficient, High-Order Perturbation Approach for Flow in Random Porous Media Via Karhunen–Loéve and Polynomial Expansions,” J. Comput. Phys., 194(2), pp. 773–794. [CrossRef]
Heroux, M. , Bartlett, R. , Hoekstra, V. H. R. , Hu, J. , Kolda, T. , Lehoucq, R. , Long, K. , Pawlowski, R. , Phipps, E. , Salinger, A. , Thornquist, H. , Tuminaro, R. , Willenbring, J. , and Williams, A. , 2003, “ An Overview of Trilinos,” Sandia National Laboratories, Albuquerque, NM, Technical Report No. SAND2003-2927. http://www.sandia.gov/~tgkolda/pubs/pubfiles/SAND2003-2927.pdf
Ganis, B. , Klie, H. , Wheeler, M. F. , Wildey, T. , Yotov, I. , and Zhang, D. , 2008, “ Stochastic Collocation and Mixed Finite Elements for Flow in Porous Media,” Comput. Methods Appl. Mech. Eng., 197(43–44), pp. 3547–3559. [CrossRef]
Wheeler, M. F. , Wildey, T. , and Yotov, I. , 2011, “ A Multiscale Preconditioner for Stochastic Mortar Mixed Finite Elements,” Comput. Methods Appl. Mech. Eng., 200(9–12), pp. 1251–1262. [CrossRef]

Figures

Grahic Jump Location
Fig. 1

Approximation of the posterior density obtained using the data Q1 (left), which gives IQ1(πDobs)≈2.015, a set of samples from the posterior (middle), and a comparison of the observed density on Q1 with the push-forward densities of the prior and the posterior (right)

Grahic Jump Location
Fig. 2

Approximation of the posterior density obtained using Q2 (left), which gives IQ2(πDobs)≈0.466, a set of samples from the posterior (middle), and a comparison of the observed density on Q2 with the push-forward densities of the prior and the posterior (right)

Grahic Jump Location
Fig. 3

The approximation of the push-forward of the prior (left), the exact observed density on (Q1, Q2) (middle), the approximation of the posterior density using both Q1 and Q2 (right) which gives IQ(πDobs)≈2.98

Grahic Jump Location
Fig. 4

The push-forward of the prior for the map Q: Λ → (Q1, Q2) introduced in Sec. 3.2 (left), the observed density using the product of the one-dimensional Gaussian (middle), which extends beyond the range of the map, and the normalized observed density that does not extend beyond the range of the map (right)

Grahic Jump Location
Fig. 5

The push-forward and observed densities on Q1 (left) and the push-forward and observed densities on Q2 (right). Notice the support of both of the observed densities is contained within the range of the model, i.e., the observed densities are absolutely continuous with respect to their corresponding push-forward densities.

Grahic Jump Location
Fig. 6

The expected information gained over the design space (which is Ω in this example) approximated using 50, 200, 1000, and 5000 samples from the prior. Notice the higher values in the center of the domain and toward the top right (in the direction of the convection vector from the location of the source); this is consistent with our intuition. Moreover, notice the small changes in the design space as we increase the number of samples from 50 to 5000. This suggests that we compute accurate approximations to the design space using as few as 50 model evaluations.

Grahic Jump Location
Fig. 7

The expected information gain over the design space (which is Ω in this example) approximated using 50, 200, 1000, and 5000 samples from the prior. Notice, the higher values in the corners and the general trend are consistent with Ref. [24].

Grahic Jump Location
Fig. 8

Posteriors, approximated using 5000 samples, using the OED for three realizations of the location of the source. Notice the information gain changes substantially for each posterior, however, this experimental design, placement of the sensor in the bottom left corner, produces the maximum average information gain, E(IQ), over all possible locations of the source.

Grahic Jump Location
Fig. 9

The computational domain and Poisson ratio showing the inclusion for a particular realization of the ellipsoid parameters

Grahic Jump Location
Fig. 10

The set of samples from the posterior and the corresponding kernel density estimate of the posterior for the first sensor location, (3.5294, 1.3049)

Grahic Jump Location
Fig. 11

The set of samples from the posterior and the corresponding kernel density estimate of the posterior for the second sensor location, (1.3902, 1.2100)

Grahic Jump Location
Fig. 12

The expected information gain over the design space (which is Ω in this example) approximated using 10, 50, 100, and 1000 samples from the prior. Notice the higher values near the location of the inclusion.

Grahic Jump Location
Fig. 13

The expected information gain over the design space (which is Ω in this example) approximated using 50, 100, 1000, and 10,000 samples from the prior. Notice the small changes in the design space as we increase the number of samples from 1000 to 10,000. This suggests that we compute accurate approximations to the design space using as few as 1000 model evaluations.

Grahic Jump Location
Fig. 14

The expected information gain over the design space as a function of previously chosen sensor locations. Note that the range of the expected information gained changes in the progression of the figures. In the bottom right, we see the greedy optimal location of eight sensors within the physical domain.

Grahic Jump Location
Fig. 15

In black, we show the possible locations of the sensors and in white, we show the optimal location(s) for 1, 2, 3, 4, and 5 sensors. In the bottom, we see the optimal location of five sensors within the physical domain. Note that the range of the data in each plot is indicative of the expected information gain for the first sensor location, not for the expected information gain for multiple sensors.

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Articles from Part A: Civil Engineering
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In