Uniform Confidence/Certainty Estimation (UC2)
Uniform Confidence/Certainty Estimation (UC2) is an approach and set of tools that address several issues that plague risk estimation techniques. Deployed between analysis and modeling, UC2 brings uniformity and interoperability that improve risk model results and improve stakeholder engagement. Its unique features correctly capture Confidence and Certainty and improve interoperability between data-driven and Expert-derived risk estimates and the models that consume them. In turn, UC2 increases uniformity, transparency, and stakeholder engagement, without ripping and replacing existing risk models or analytical workflows.
At the heart of risk analysis lurks the particularly problematic issue of Confidence. The problem is widely acknowledged, but often misunderstood and not well managed.
Confidence refers to the ability of an Expert to make subjective accurate predictions that align with the objective truth. If truth is the bullseye of a target, Confidence represents how close risk estimations are to the center. Confidence is the expected proximity to truth.
Certainty is the agreement between multiple estimates from many sources. Tightly grouped estimates indicate more Certainty; more dispersion means less. Certainty is a measure of precision and consistency, not closeness to truth.
Uniform Confidence/Certainty Estimation (UC2 / you-see-two) is a solution to this and other problems in estimating values for use in risk equations and risk models. It is applicable to quantitative, data-driven estimation techniques as well as qualitative techniques. It also acts as a bridge between quantitative, qualitative, objective, and subjective estimations freeing the risk analyst to use most combinations of data sources together in a uniform and transparent manner.
UC2 Scale - Quantitative Example
Usage by risk analysts and subject matter experts (Experts) is straightforward and intuitive. Segments across the top row of the UC2 Scale express the desired level of granularity for an estimate of an arbitrary range. This is the bullseye of objective truth with just enough granularity to assist a decision making stakeholder — or risk model — arrive at an actionable outcome.
When Data-driven or Expert-derived estimates cannot be neatly mapped to the bullseye, the target Confidence/Certainty drops to lower levels. This example also illustrates overlapping Certainties, which are common in both data-driven and Expert-derived estimation.
The bottom row, labeled “Unknown” is the very lowest level of Confidence. In data-driven estimations it denotes that an acceptable estimate was not found within the data. Experts will use this row to indicate “I don’t know” or “I lack context” In both cases it can also indicate that no data or Expert source is available.
Mapping data-driven and Expert estimations to UC2 explicitly captures both Confidence and Certainty in a uniform manner that allows for aggregating estimates from both data-driven and Expert sources. As Further discussed in UC2 Analysis, multiple estimates combine in a way that transparently honors Confidence and Security. UC2 Distributions are more nuanced and accurate outputs that are compatible with nearly any risk model or risk assessment.
With respect to integration with existing models and workflows, UC2 offers incremental improvement without having to rip and replace existing models and estimation techniques. It fits seamlessly into existing data- and expert-driven workflows making them more uniformly compatible through UC2 Scales and UC2 Analysis. And UC2 Distributions integrate just as seamlessly with nearly any model of risk. The outputs are compatible with existing model inputs like Binomials, traditional PERT estimates, PERT Distributions, and other free-form distributions.
Released under the terms of the Creative Commons Attribution-ShareAlike 4.0 International license. You are encouraged to distribute, use, remix, or build upon this work. However, you must give credit as
“Rob Arnold, Acorn Pass, LLC - https://AcornPass.com”
and you must share any derivatives under the same terms along with original credit.
UC2
Download the methodology, build your UC2 Scales and level up your risk management program, for FREE!
Please Contact the Author with feedback, questions, or help with bespoke implementations.