Skip to content
Upcoming courses

Uncertainty analysis

Tools
From Fire Safety Academy Wiki
Revision as of 23:17, 1 November 2026 by Mark (talk | contribs) (New FSE wiki article: Uncertainty analysis (written by Claude Opus 5.5))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Contents
  1. 1Sources of uncertainty
  2. 2Aleatory and epistemic uncertainty
  3. 3Switchover and false precision
  4. 4Methods of analysis
  5. 5Propagating input uncertainty
    1. 5.1Worked example
  6. 6Uncertainty in design decisions
  7. 7Limits and practical difficulties
  8. 8See also
  9. 9Notes
  10. 10References

Uncertainty analysis is the part of a fire safety engineering calculation that identifies what is not known exactly and shows how this affects the result. An uncertainty study determines how uncertainties in the input data and in the methods used are reflected in the outputs of the analysis.[1] The sources of uncertainty range from measurement errors and missing data to randomness, simplified models, human behaviour and disagreement about the criteria for a safe design.[2][3] The treatment of uncertainty helps to maintain an appropriate level of public safety while leaving room to reduce costs.[4]

The central question is whether reasonable changes in the inputs, assumptions or criteria could change the design decision. This change of decision is called switchover.[5] Simple approaches use bounding values, safety factors and sensitivity studies. More complete approaches describe uncertain inputs as probability distributions and propagate them through the calculation, for example by Monte Carlo sampling.[6][7] In fire modelling, an uncertainty analysis combines the uncertainty of the inputs with the uncertainty of the model itself, which is estimated from validation studies.[8]

Sources of uncertainty

Uncertainty is often treated as if it meant only measurement uncertainty, that is, doubt about the result of a measurement. Measurement uncertainty has a random part, estimated from repeated observations, and a systematic part, linked to corrections and reference standards. Together they form the total error.[2] Measurements of heat release rate, burning rate or radiative fraction are typical examples of such empirical quantities in fire safety engineering.[9]

Many other sources of uncertainty do not fit the standard treatment of measurements. The heat release rate of a mixed fuel package may simply be unknown. Where and how a fire starts is a matter of chance. The occupancy and furnishings of a building may be different after 10 or 20 years.[2] Occupants may act in many ways after hearing an alarm, and data sources can disagree, for example on the yields of combustion products. Even a goal such as "an equivalent level of fire safety" can be defined in several ways.[2]

For a typical design calculation, scientific uncertainties fall into five groups. These are theory and model uncertainties, data and input uncertainties, calculation limitations, the level of detail of the model, and how well the design fire scenarios represent reality.[10] A two-zone model, for example, assumes that each room consists of two uniform layers. The heat release rate of a three-cushion sofa may be uncertain because no test data exist for a sofa of the same size and materials. Even the choice of the control volume matters. Modelling a single area under draft curtains, instead of the whole building, predicts higher temperatures and faster activation of fire protection devices.[10]

The fire probabilistic risk assessment (PRA) method for nuclear power plants groups uncertainties more simply. It distinguishes modelling uncertainties, including the completeness of the model, from data or parameter uncertainties.[11] Data uncertainties can be described by probability distributions, while modelling uncertainties can be handled by several alternative models, each given a probability of being correct.[12]

Aleatory and epistemic uncertainty

Two kinds of uncertainty are distinguished in most risk analyses. Aleatory uncertainty is due to random variation and chance outcomes. It is also called randomness, stochastic uncertainty or statistical uncertainty. In principle, it cannot be reduced, although it can be characterised better through study.[5] Epistemic uncertainty arises from a lack of knowledge. In principle, it can be reduced by collecting more information or data.[5]

The fire PRA method illustrates the difference with a pump. Whether a pump fails within a given period is a random event, so the probability of at least one failure expresses aleatory uncertainty. The value of the failure rate of that pump is a fixed but unknown number, so the uncertainty about it is epistemic.[13] With few failure data, the epistemic uncertainty in the failure rate is large. After a large enough sample has been collected, the failure rate can be estimated well.[13]

The two kinds should be kept apart where possible. In a fire PRA, the aleatory part is represented in the event tree, while the epistemic part is treated when probability distributions are propagated through the model.[14] Probability itself can be read in two ways. In the frequentist view, it is the frequency of an event in a long series of similar trials. In the Bayesian view, it is the degree of belief of a person, given all the information available to that person.[2]

Switchover and false precision

Direct measurement of the fire safety performance of a building is not usually possible, so designs rely on predictions by fire models and other tools.[5] A single predicted value hides the uncertainty around it. In an example in the SFPE Handbook of Fire Protection Engineering, the upper layer of a room was predicted to reach the limit of 100 °C after 175 s. A sprinkler was predicted to activate after 171.2 s, so the design seemed acceptable.[15] If the predicted time to 100 °C is uncertain by 20 s, however, the limit could be reached after 155 s, before the sprinkler activates.[15]

The choice of criteria is itself uncertain. Two engineers who presented performance-based case studies for real buildings at one conference had both followed the design guidelines. Yet they selected very different performance criteria, in the parameters, the critical values and the use of a time element.[3] Because model predictions are compared with such criteria, a different choice can make the same design pass or fail.[3]

Uncertainty also affects how results should be reported. Input data for fire safety calculations, such as fire loads, heat release curves, walking speeds and material properties, are often uncertain by tens of percent. A result such as "an available escape time of 187.4 seconds" then suggests a precision that the calculation cannot deliver.[16] Two, or at most three, significant figures are realistic for most results. For critical decisions, a sensitivity or uncertainty analysis says more than strict rounding rules.[16]

Methods of analysis

Measured quantities can be handled by classical uncertainty analysis, which uses probability and statistics. A Type A evaluation is based on a statistical treatment of data, such as the standard deviation of a series of observations. A Type B evaluation relies on scientific judgement and other information, such as manufacturer's specifications and calibration reports.[9] Decision variables, such as the acceptable level of fire safety, and value parameters, such as the value placed on a life, have no true value to measure.[9] For these, the effect of different choices on the outcome is shown explicitly.[6]

Several simpler techniques make such effects visible. Bounding evaluates the extremes of the possible range: if both ends give acceptable results, a more costly analysis may be avoided. A sensitivity analysis tests how responsive the results are to changes in individual parameters, and so identifies the most important ones.[6] An importance analysis ranks the uncertain inputs by the rank-order correlation between each input and the output, on a scale from 0 to 1. Where no data exist, the judgement of experts can be gathered in a structured way through expert elicitation.[6]

For complex computer models, three approaches have been compared: direct calculation of derivatives, replacement of the model by a fitted response surface, and Monte Carlo sampling.[7] In Monte Carlo sampling, each uncertain input is described by a probability distribution. The model is run many times with randomly chosen input values, and the results approximate the probability distribution of the output.[7] A comparison by Iman and Helton found that Monte Carlo simulation performed best overall, while response surfaces were not recommended because fire models are too complex for a simple function.[17] Fire PRAs for nuclear power plants propagate epistemic uncertainties with Monte Carlo or Latin hypercube sampling.[18]

In practice, a full analysis is seldom possible for every variable. When the handbook chapter was written, common practice consisted of a series of sensitivity studies that change one variable at a time. This becomes impractical for a design with hundreds of variables.[19] A complete analysis therefore treats quantitatively only the parameters that could cause switchover, and uses best estimates for the rest.[19]

Propagating input uncertainty

For simple relations, the effect of an input uncertainty can be estimated without running a model many times. A validation study for nuclear power plants used empirical correlations for this purpose.[20] According to the correlation of McCaffrey, Quintiere and Harkleroad, the temperature rise of the hot gas layer is proportional to the heat release rate raised to the power 2/3.[20] A small relative uncertainty in the heat release rate therefore produces a relative uncertainty in the temperature rise of two-thirds that size:[21]

ΔT∝Q˙2/3⇒δ(ΔT)ΔT≈23δQ˙Q˙

The same study found the layer height to be much less sensitive. A 15 % to 25 % increase in heat release rate lowers the layer height by only 1 % to 2 %.[21] Such approximations have limits. In a NIST study, a 10 % higher heat release rate should have raised the upper layer temperature by about 6.7 % according to the correlation. A computer simulation gave 9 %, at temperatures above the range of the data behind the correlation.[22]

Independent uncertainties are combined as the square root of the sum of their squares.[23] The same study compared each model prediction with a measurement through the relative difference ε between the predicted and measured rise. It judged the difference significant when it exceeded the combined relative expanded uncertainty:[24]

Uε≈U~M2+U~E2,|ε|>Uε


Symbol Description Unit
ΔT temperature rise of the hot gas layer above ambient K
Q˙ heat release rate kW
δ(⋅) uncertainty of the quantity in brackets unit of that quantity
ε relative difference between predicted and measured rise –
U~M relative expanded uncertainty of the prediction due to its inputs –
U~E relative expanded uncertainty of the measurement –
Uε combined relative expanded uncertainty –

An expanded uncertainty with an expansion factor of two corresponds to two standard deviations and a confidence level of about 95 %.[25] This combined uncertainty does not include the intrinsic uncertainty of the model, such as the two-layer assumption of a zone model or the grid size of a CFD model.[25][24] The two-thirds rule describes small changes, and it holds only within the range of the data behind the correlation.[26][22] The root-sum-square rule assumes that the two uncertainties are independent.[23]

Worked example

The measured heat release rates in the experiments of the validation study had uncertainties of 15 % to 25 %.[20] By the two-thirds rule, an uncertainty of 15 % in the heat release rate gives about 10 % in the predicted temperature rise of the hot gas layer. An uncertainty of 25 % gives about 17 %.[21]

The study illustrates the combination with an upper layer temperature. The predicted rise was about 330 °C and the measured rise about 270 °C, so the relative difference is (330 − 270) / 270, or about 0.22.[27] The two relative expanded uncertainties were about 20 % and 10 %. Combined, they give the square root of 0.20² + 0.10², which is also about 0.22.[27]

The relative difference is therefore about as large as the combined uncertainty. The data cannot show whether the model really over-predicts, because the difference lies at the edge of what the uncertainties allow.[24] With smaller uncertainties of 10 % each, the combined value would be about 0.14, and the same difference would be significant.

Uncertainty in design decisions

Many designs deal with uncertainty through safety factors. A safety factor multiplies a design value, while a safety margin is the difference between the design value and the value that would no longer be safe.[28] Safety factors can be implied, for example in conservative assumptions, or explicit. An example of an explicit factor is multiplying the time needed for escape by 2.[28]

Safety factors behave less simply than they appear. Doubling the heat release rate does not provide a safety factor of 2 on the time available for escape, because the heat release rate is not linearly related to the time to a critical temperature.[29] Factors applied to two parameters do not simply add up. A choice that is conservative for one outcome can work the other way for another. Doubling the heat of combustion is conservative for the upper layer temperature, while doubling the radiative fraction has the opposite effect.[29]

The International Fire Engineering Guidelines (2005) link these choices to the acceptance criteria of a design. For an absolute evaluation, the criteria should take into account the uncertainties in the analysis and the factors of safety employed.[30] The size of the factors should consider, among other things, the redundancy and reliability of the fire safety systems, the methods and assumptions, and the results of an uncertainty analysis. Factors of safety should be applied only at the end of a calculation sequence, because applying them throughout could lead to overly conservative outcomes.[30]

A probabilistic statement of performance makes the role of uncertainty explicit. It contains at least four elements: a probability, a time, a performance criterion and a threshold value.[31] An example is a requirement that the design gives a probability of 0.9 of having 4 min or more before the temperature in a corridor reaches 65 °C.[31] Where values must be rounded for a comparison with limits, the safe direction is preferred: required times are rounded up and available times down.[16]

Limits and practical difficulties

Uncertainty analysis meets several practical obstacles. There is no agreed quantitative method for performance-based designs, and a mathematically rigorous analysis may seem impractical. Large amounts of data would be needed to define the ranges of inputs such as heats of combustion and species yields.[19] Some uncertainties go unrecognised, such as values hard-wired into tools, the choice of tenability criteria and the selection of design fires.[19]

Not every uncertainty can or should be quantified. Some quantities, such as societal values, become policy or regulatory issues rather than engineering issues.[19] In fire PRA, some uncertainties are treated only qualitatively. A basis must then be given for those that are not addressed.[12] An importance analysis can point to the inputs that matter most, but a correlation between an input and an output does not prove that one causes the other.[32]

The uncertainty of the model itself is the hardest part to estimate. Verification and validation studies provide a bias factor and a relative standard deviation for each predicted quantity. These must be combined with the uncertainty of the inputs chosen by the user, and a sensitivity analysis may be needed in addition.[8] The International Fire Engineering Guidelines (2005) suggest that comparing the outputs of several suitable methods can also give an indication of the uncertainty associated with the methods.[1]

See also

Notes

  1. ↑ 1.0 1.1 National Research Council of Canada et al. (2005), p. 1.2-24.
  2. ↑ 2.0 2.1 2.2 2.3 2.4 Notarianni (2002), p. 5-41.
  3. ↑ 3.0 3.1 3.2 Notarianni (2002), p. 5-43.
  4. ↑ Notarianni (2002), p. 5-40.
  5. ↑ 5.0 5.1 5.2 5.3 Notarianni (2002), p. 5-42.
  6. ↑ 6.0 6.1 6.2 6.3 Notarianni (2002), p. 5-50.
  7. ↑ 7.0 7.1 7.2 Notarianni (2002), p. 5-51.
  8. ↑ 8.0 8.1 Stroup et al. (2016), p. 7-2.
  9. ↑ 9.0 9.1 9.2 Notarianni (2002), p. 5-49.
  10. ↑ 10.0 10.1 Notarianni (2002), p. 5-45.
  11. ↑ Nowlen et al. (2005b), p. 15-3.
  12. ↑ 12.0 12.1 Nowlen et al. (2005b), p. 15-4.
  13. ↑ 13.0 13.1 Nowlen et al. (2005b), p. U-3.
  14. ↑ Nowlen et al. (2005b), p. U-5.
  15. ↑ 15.0 15.1 Notarianni (2002), p. 5-55.
  16. ↑ 16.0 16.1 16.2 Van Rossum (2027), §1.7.
  17. ↑ Notarianni (2002), pp. 5-51–5-52.
  18. ↑ Nowlen et al. (2005b), p. 15-2.
  19. ↑ 19.0 19.1 19.2 19.3 19.4 Notarianni (2002), p. 5-44.
  20. ↑ 20.0 20.1 20.2 Hamins and McGrattan (2007), p. 5-1.
  21. ↑ 21.0 21.1 21.2 Hamins and McGrattan (2007), p. 5-2.
  22. ↑ 22.0 22.1 McGrattan et al. (2005), p. 28.
  23. ↑ 23.0 23.1 Hamins and McGrattan (2007), p. 1-10.
  24. ↑ 24.0 24.1 24.2 Hamins and McGrattan (2007), p. 1-11.
  25. ↑ 25.0 25.1 Hamins and McGrattan (2007), p. 1-8.
  26. ↑ Hamins and McGrattan (2007), p. 5-3.
  27. ↑ 27.0 27.1 Hamins and McGrattan (2007), p. 1-12.
  28. ↑ 28.0 28.1 Notarianni (2002), p. 5-47.
  29. ↑ 29.0 29.1 Notarianni (2002), p. 5-48.
  30. ↑ 30.0 30.1 National Research Council of Canada et al. (2005), p. 1.2-26.
  31. ↑ 31.0 31.1 Notarianni (2002), pp. 5-53–5-54.
  32. ↑ Notarianni (2002), p. 5-59.

References

  • Hamins, A., & McGrattan, K. (2007). Verification and validation of selected fire models for nuclear power plant applications: Vol. 2. Experimental uncertainty (NUREG-1824; EPRI 1011999). U.S. Nuclear Regulatory Commission, Office of Nuclear Regulatory Research; Electric Power Research Institute.
  • McGrattan, K. B., Bouldin, C., & Forney, G. P. (2005). Computer simulation of the fires in the World Trade Center towers (NIST NCSTAR 1-5F). National Institute of Standards and Technology.
  • National Research Council of Canada, International Code Council, Department of Building and Housing, & Australian Building Codes Board. (2005). International fire engineering guidelines (2005 ed.). Australian Building Codes Board.
  • Notarianni, K. A. (2002). Uncertainty. In P. J. DiNenno, D. Drysdale, C. L. Beyler, W. D. Walton, R. L. P. Custer, J. R. Hall Jr., & J. M. Watts Jr. (Eds.), SFPE handbook of fire protection engineering (3rd ed., pp. 5-40–5-64). National Fire Protection Association; Society of Fire Protection Engineers.
  • Nowlen, S. P., Najafi, B., Wyant, F. J., Forester, J., Kazarians, M., Kolaczkowski, A., Joglar, F., Funk, D., Hannaman, G., & Anoba, R. (2005b). EPRI/NRC-RES fire PRA methodology for nuclear power facilities: Vol. 2. Detailed methodology (NUREG/CR-6850; EPRI 1011989). Electric Power Research Institute; U.S. Nuclear Regulatory Commission, Office of Nuclear Regulatory Research.
  • Stroup, D., Lindeman, A., McGrattan, K., Peacock, R., Overholt, K., Joglar, F., LeStrange, S., & Montanez, S. (2016). Verification and validation of selected fire models for nuclear power plant applications: Supplement 1 (NUREG-1824, Supplement 1; EPRI 3002002182). U.S. Nuclear Regulatory Commission, Office of Nuclear Regulatory Research; Electric Power Research Institute.
  • Van Rossum, E. (2027). Introduction to Fire Safety Engineering. Fire Safety Academy.
Retrieved from "https://fire-safety.academy/w/index.php?title=Uncertainty_analysis&oldid=130"