Industry

Cyber security is compared to other industries for its lack of rigorous and quantitative analysis of risk. Risk Measurement focuses on quantitative risk measurement to assist with this imbalance. While cyber security may lacks the forms of rigor found in other industries, this documentation also seeks to mitigate known weaknesses found in those approaches.

What makes other industries quantitative about how they handle risk? How are we so different? Well, here are some examples, and we’ll start with one you see all the time.

Will it rain tomorrow?

Of course, a meteorologist would give you a forecast as a percentage. This would come from their subjective expertise as a meteorologist, supported by an enormous pile of modeled weather data to support their forecast.

Many industries follow similar principles. This means:

• Risks are structured as scenarios.
• Reference class data is gathered as closely to the scenario as possible.
• Forecasts are made with all available information.
• Decisions are made.
• Corrections are made when compare forecasts to reality.

Here are some industry examples which use different language, but with the same principles.

Aerospace #

The FAA has regulation about space launches. All of their risk management effort directs towards a measurement for a simple event: Expected Casualties (Ec).

For all launches, regardless of vehicle type, this final rule requires a single expected number of casualties (Ec) be calculated by aggregating the risk posed to the collective members of the public…

This final rule also revises the acceptable risk threshold for launch from an Ec of 30 × 10E-6 for each hazard to an Ec of 1 × 10E-4 for all three hazards combined.

What does this mean in english?

How many casualties will result from a launch? 

Nuclear Safety #

The NRC (US) certifies nuclear reactors partially based on the completion of Probabilistic Risk Analysis. Nuclear is concerned with events that may result in core damage (Level 1 PRA), how these may leak radioactivity (Level 2 PRA), and how individuals would be put at risk from these exposures (Level 3 PRA). Expert elicitation techniques in nuclear risk measurement are common.

• Will a tube in a steam generator rupture?
• Will there be health effects from resulting raditation leaks around a plant?
• How frequently will the core be damaged?

The nuclear industry relies on extensive data gathering to inform estimation methods, and expert opinion is relied on when adjustments need to be made for innovations without historic data.

Environmental Safety #

Environmental impact organizations use the probabilistic risk assessment.

Scenario: The change in the mortality rate due to Fine Particles (PM2.5) decrease if we pass X regulation.

Outcome: Credible Interval: Reduction of .001 -.05% with 95% confidence.

The CSB organizes investigations that provide transparency into root causes informing probabilistic risk approaches supported in EPA policies.

Meteorology #

The United States spends billions on weather forecasting and its associated infrastructure.

Scenario: Will the east coast hurricane make landfall near our city before we can evacuate?

Outcome: % Likelihood of Yes / No: Yes with a likelihood of 50%.

NOAA and other global organizations build weather infrastructure that makes operational forecasting possible. Meteorologists still make adjustments to their models when publishing forecasts based on known failures of their models, outages in data collection, or local familiarity.

Intelligence Analysis #

All forms of intelligence gathering ultimately desire to inform decision making.

Scenario: Does the image taken by our satellite depict an adversary military aircraft?

Outcome: % Likelihood of Yes / No: Yes with a likelihood of 70%.

Quantitative estimates are a foundational part of the National Intelligence Estimate process, and are used globally by intelligence agencies. Publicly accessible platforms have taken similar approaches with decision markets accepting participation from large groups.