The statistical foundations for risk-optimal certification by analysis

The traditional ‘make and test’ approach for new products relies on experience, knowledge and large-scale demonstrators to generate certification data.

At best, these are prohibitively costly and time-consuming. For some new products, this approach is impossible.

Sophisticated simulation is increasingly being used by engineers in place of expensive physical testing and at the High Value Manufacturing Catapult we are accelerating these certification by analysis (CBA) capabilities to reduce the time to market for new products.

But how can we strike the right balance between speed and safety?

The trade-off between data collection and costs of experiments has long been acknowledged and a formal solution is offered by the ‘value of information’ analysis method from computational statistics.

The Alan Turing Institute recently demonstrated how an underlying mechanical property can be estimated from a combination of physical testing data and model outputs, whilst simultaneously inferring the imperfect features of the modelling, such as imprecision, bias, reliability and missingness.

The calculation acknowledges that if there is evidence supporting the quality of simulation data, then it will be considered more informative, and more likely to justify reducing physical testing. The converse is also true.

Uncertainty is quantified through a decision problem and the expected value of data collection is obtained. This allows us to identify the sweet spot where the value of additional information outweighs the cost of obtaining it, and a risk-optimal combination of testing and modelling can be identified.

In addition, we completed a sensitivity analysis, demonstrating how the optimal strategy can change based on modelling results of differing quality.

Our approach:

1. Provides a quantitative, risk-based framework for certification decisions
2. Helps justify investments in both physical testing and simulation capabilities
3. Can potentially reduce time-to-market for new products without compromising safety.

But this is not a one-size-fits-all solution.

Each industry and product type will require careful consideration of the specific risks, costs and regulatory requirements involved; but the principles of the analysis are generic. By adopting this method, we can help industries navigate the complex trade-offs between safety, efficiency and innovation in product development.

This work was presented at the FrontUQ uncertainty in aerospace engineering conference in September 2024, and a special issue journal paper will follow.

To find out more about HVM Catapult’s work on certification by analysis, or to speak to our engineers about your industry challenges, get in touch with our team.