Multifidelity Optimization Using Asynchronous Parallel Pattern Search and
Multifidelity Optimization Using Asynchronous Parallel
Pattern Search and Space Mapping Techniques
Genetha Gray*, Joe Castro , Patty Hough*, and Tony Giunta
*Computational Sciences & Mathematics Research, Sandia National Labs, Livermore, CA
Computational Sciences and Validation & Uncertainty Quantification Processes, Sandia National Labs, Albuquerque, NM
Purpose of Multifidelity Optimization
In many optimization applications, the objective
function f(x) is expensive to calculate and derivatives
may be inaccurate if they exist at all.
Many of these applications have a high fidelity (or
true) model and a low fidelity (or surrogate) model
which simplifies the high fidelity model in some
An MFO approach optimizes an inexpensive, low
fidelity model while making periodic corrections
using the expensive, high fidelity model.
The relationship between the models can be exploited
in an algorithm by applying space mapping
Outer Loop Optimizer: Asynchronous
Parallel Pattern Search (APPSPACK)
Direct search method
Takes advantage of parallel platform to reduce
Does not assume time needed to evaluate objective
function is constant
Global convergence to a
stationary point under
The Inner Loop as an Oracle
High Fidelity Mode
Low Fidelity Model
1. Start Outer Loop (APPSPACK)
Evaluate N high fidelity response points
Produce table of data with xH and fH(xH)
2. Start Inner Loop
Take data from APPSPACK table
Run LS optimization algorithm
At each iteration, evaluate N low-fidelity responses
At completion obtain , , for our space mapping of (xH) +
Optimize low fidelity model within space mapped high
fidelity space. In other words, minimize fL((xH) + ) with
respect to xH to obtain xH*
3. Return xH* to APPPSPACK to determine if a new best
point has been found.
If the inner loop is viewed as an oracle, the only
significant algorithmic change to APPS is the
addition of the oracle.
Analytically, there are no restrictions on how an
oracle can choose points.
Oracle points are used in addition to the points
defined by the search pattern
The convergence of APPSPACK is not adversely
A convergence proof for the APPS algorithm
exists so no new convergence proof for our MFO
scheme is needed.
Future work may include investigating any
improvement to the convergence of APPS.
A Simple Example
Space Mapping Representation
RHFM ( x0 , x1 ) x0 0.5 x1 0.83
RLFM ( x0 , x1 ) x0 x1
RHFM ( 0.5,0.83)
x0 0 x1 1
f H ( x0 , x1 ) (0.8 x0 0.5) 2 (0.5 x1 0.83) 2
When the number of response points is 8, there are two
calls to the inner loop
0 0.5 , 1 0.83
Gamma ~ O(1), Starting Point = (-2.0,-2.0)
f L ( x0 , x1 ) x x ( x0 0 ) ( x1 1 )
f H ( x0 , x1 ) ( x0 0.5) ( x1 0.83)
(-0.8,-1.2) 1 (-0.76,2.0)
Log Plot of
The numbered white boxes show
approximately where the inner loop
The point in red brackets is the
APPSPACK best point before the
inner loop call.
The point in green was obtained by
the inner loop.
(-0.56,1.6) 2 (-0.61,1.25)
Ongoing and Future Work
Study spaces defined using different constraints.
Implement a generic oracle in APPSPACK.
Include a space mapping that does not require domain
spaces to be defined by the same numbers of parameters.
Apply our multifidelity optimization schemes to some
real world problems:
Earth penetrator analysis
Groundwater problems including well field design &
Circuit system design
Consider a simple example with constant space mapping terms:
Study space mapping sensitivities to various inputs
Number of high fidelity responses used in the mapping
Scaling of the mapping parameters (size of the offset between the low and high fidelity models)
Compare the optimum obtained and the number of high fidelity runs required to reach this
View of High Fidelity
Sandia National Laboratories is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC04-94AL85000.
Bakr, Bandler, Madsen, and Sondergard, An Introduction to Space
Mapping Technique, Optimization & Engineering, 2:369-384, 2001.
Gray and Kolda, APPSPACK 4.0: Asynchronous Parallel Pattern Search
for Derivative-Free Optimization, Technical Report SAND2004-6391,
Sandia National Laboratories, Livermore, CA, Aug 2004. (Submitted to
ACM TOMS.) http://software.sandia.gov/appspack/version4.0/
Kolda, Revisiting Asynchronous Parallel Pattern Search for Nonlinear
Optimization, Technical Report SAND2004-8055, Sandia National
Laboratories, Livermore, CA, Feb 2004. (In revision for SIAM J. Opt.)
Kolda, Lewis, and Torczon, Optimization by Direct Search: New
Perspectives on Some Classical and Modern Methods, SIAM Review,
2. appears in the numerator of Equation 8.4, the lattice energy increases dramatically when the charges increase. Thus, we expect the lattice energy of CaO, which has 2+ and 2- ions, to be the greatest of the three. ... For...
bromley hospitals nhs trust. bromley pct. central and north west london mental health nhs trust. croydon pct. ealing hospital nhs trust. ealing pct. enfield pct. brent teaching pct. great ormond street hospital for children nhs trust. camden and islington mental...
Esta obra de Claude Oscar Monet pertenece al Impresionismo (s. XIX), concretamente es de 1875 y se titula . Madame Monet and Child. Se trata de un óleo sobre lienzo que se encuentra en el Museo de Bellas Artes de...
Water from the reservoir is brought to the turbine through penstocks, at the end of which a nozzle is fitted. The nozzle converts whole of the available head into the kinetic head in the form of a high velocity jet....
Internal Check. One of the Modes of Implementing Internal Control "a system under which the accounting methods and details of an establishment are so laid out that the accounts and the procedure are not under the absolute and independent control...
If you hover or click on the guest number you can see what the patron wrote. Practicing what we call "tough love" is the same as ignoring a patron. The reason these are separate transactions is because the patron must...
Janelle Carper & Taylor Belella. Lesson 1. Harriet Tubman and the Underground Railroad . Standards: SS.4.E.4. Examine and research how slavery and indentured servants influenced. the early economy of the United States by constructing graphics (e.g.,
Ready to download the document? Go ahead and hit continue!