From Theory to Application: Tackling Real-World Uncertainty in Optimization

Design Optimization for Real-World Challenges

In a previous article, we discussed how some organisations perceive design optimization as challenging to implement due to its theoretical nature and perceived complexity. This series aims to address those concerns by exploring advanced techniques that help industrialise and scale design optimization for real-world projects.

In this first installment, we focus on Optimization Under Uncertainty (OUU) — a key methodology for addressing the challenges of uncertainty in design. It plays a vital role in making design optimization both practical and robust by enabling engineers to account for variability and achieve consistent performance in complex systems.

Managing Uncertainty in Engineering Design

Uncertainty is an inherent part of engineering. From imperfections in manufactured components to highly variable operating conditions, few things in complex systems are definite or deterministic.

Optimization Under Uncertainty addresses the inherent variability in engineering by explicitly accounting for randomness in design variables or parameters. This structured approach not only leads to optimal solutions but also enhances robustness, ensuring that objective performance is less affected by variability, and reliability, where constraints remain consistently satisfied despite uncertainties.

By integrating uncertainty into the optimization process, engineers can develop systems that perform predictably and consistently, even when faced with real-world complexities.

Uncertainty Propagation

To produce robust and reliable designs, we need to understand the statistical behaviour of our output functions, such as objectives and constraints. However, direct access to this information is often rare. Instead, we rely on propagating uncertainty from the inputs (e.g., specifications outlining the distribution of manufacturing defects or experimental data detailing varying operating conditions) to the outputs.

First-Order Perturbation Methods

One of the simplest methods for uncertainty propagation is first-order perturbation, which is based on the first-order Taylor Series expansion around the mean input value. In this approach, the mean of the function is approximated by evaluating the function at the mean of the inputs, while the variance is calculated as the expected value of the squared deviations from the mean, assuming independence among the variables and parameters.

While computationally efficient, this method has its limitations. By ignoring higher-order terms in the Taylor Series expansion, it inherently assumes that the underlying distribution is symmetric, which is often not the case in real-world engineering problems. 

Direct Quadrature

A more accurate method is direct quadrature, which evaluates the mean and variance by using their integral definitions and performing numerical integration. This involves discretising the function into a series of predefined shapes, such as trapezoids or rectangles, and summing their areas to approximate the results.

This approach, however, scales poorly with the number of inputs. For instance, computing quadratures for two input variables with 30 samples each requires 900 evaluations. Adding a third input variable significantly increases this number to 27,000 evaluations, demonstrating how quickly complexity grows. While sparse grid techniques can mitigate the computational cost by strategically reducing the number of sampling points, direct quadrature remains impractical for high-dimensional problems due to its exponential growth in computational requirements.

Monte Carlo Simulation

Monte Carlo Simulation offers a more empirical approach to uncertainty propagation. By sampling inputs according to their probability distributions and evaluating the corresponding outputs, this method enables the construction of a representative distribution of the output function values.

Since Monte Carlo Simulation relies on the law of large numbers, it requires a substantial number of samples to achieve accurate results. However, the required sample size is not affected by the dimensionality of the input, making this method particularly suitable for high-dimensional problems, as alternative approaches would be even more computationally prohibitive.

Robust Optimization

With output statistics derived from uncertainty propagation, we can refine our optimisation objectives to enhance the robustness of solutions. For instance, rather than minimising a nominal value, the objective can be redefined to minimise the expected value, a weighted combination of the mean and standard deviation, or to constrain variance within tolerable limits.

These formulations ensure that the design is optimised not just for peak performance but also for consistency and resilience, enabling solutions to perform effectively under varying conditions. For example, in wind farm design optimisation, rather than deterministically maximising power output for a single dominant wind direction, the focus shifts to maximising the expected value of power output across all possible directions, ensuring steady performance regardless of variability in wind patterns.

Reliable Optimization

Reliable optimization focuses on constraints, ensuring they account for uncertainty. A typical approach involves probabilistic constraints, such as ensuring that the probability of constraint satisfaction exceeds a threshold.

This prevents solutions that sit precariously on the edge of feasibility, incorporating a “buffer zone” for added reliability. While safety factors often achieve similar outcomes, integrating uncertainty directly into the optimization process can yield better-informed designs and push performance even further.

Conclusions

Uncertainty is a fact of life — and of engineering. Ignoring it can lead to missed opportunities or even catastrophic failures. By leveraging techniques like Optimization Under Uncertainty, foundational optimization methods can be advanced to address real-world challenges with greater effectiveness and confidence.

In this article, we explored various uncertainty propagation methods and examined how these insights can be applied to achieve more robust and reliable optimisation in practical design challenges.

Have you applied these techniques in your own projects? I’d love to hear about your experiences and insights. Stay tuned for the next installment in this series, where we’ll continue to explore advanced design optimization techniques tailored for real-world challenges.