Culture

Many forms of cultural debt exist in the workplace that can hinder a team from really taking advantage of collaborative risk measurement methods. Outside of the principles in this document, the focused norms adopted by a group should help improve probabilistic approaches to risk.

These items are difficult to enforce in the documentation and are better discussed more freely.

Prepare for prevention failures.

Risk measurement is great at measuring “known knowns” and “known unknowns”, but Black Swan Theory is an important aspect to consider when enumerating scenarios of risk.

In any complex environment, there will always be a substantial amount of risk that defies measurement. In which case, attention to the outcomes and the expected response becomes much more worthwhile.

In some industries, “Assume Breach” and “Defense-in-depth” are the rules of thumb that guide this mindset. Risk measurement methods can easily mislead those who wield them into thinking they’ve mitigated all “knowns”.

Some simple tests to understand if you’re thinking about risk correctly, are scenarios like the following:

  • An intruder has bypassed our frontline defenses. (Measure: How likely would you detect them? How long would it take to detect? How much damage would occur?)
  • Business has been interrupted due to a system outage. (Measure: How long until you’d come back online? Would you come online? How many customers would go away?)

A mindset that mitigates a few broad outcomes as opposed to preventing endless amounts of causes is valuable. This mindset helps mitigate eventual failure, and helps mitigate failures of human imagination.

Minds that are open to new information.

One’s ability to update previous beliefs based on sound evidence should be cherished. An unwillingness to hold strong beliefs until equal evidence is shown is a cornerstone of quantitative reasoning. Being wishy-washy is OK so long as one can identify evidence that would influence them.

Seek information, not agreement.

When forming panels, it is important to withhold bullying and attempts to get others to find agreement. Toxicity brews this way and influences risk measurement poorly.

Continuously practice risk literacy.

Opportunities to forecast outcomes are all around us. Find these future events that relate to your interests, your organization, or society and forecast them. Keep score and mind your calibration.

Protect and encourage the “Post Mortem”

Blame should be avoided during post-mortem sessions. Root cause knowledge should be cherished, organized, and shared. Reference data from other organizations should be protected and encouraged as well. Seek non-disclosure agreements and adopt rules like the Traffic Light Protocol or Chatham House rules if necessary.

Remove incentives that corrupt forecasts.

A “Million dollar” management problem that haunts all organization are the incentives to empire build, be promoted, secure a budget, etc. A principle of OKR’s has always been to keep the pursuit of Key Results out of performance management. Decision markets have also seen toxicity brew as individual finances became involved.

Avoiding outside influence that politicize or gamify forecasting will help match the culture and mindset currently seen in meteorology. Weather is not considered a betting environment, nor a market.