The Metrics of Business Aviation Safety
Category: Business Aviation and the Boardroom
Author: Pete Agur
King Rat: The Metrics of Business Aviation Safety
Operating 20,000 hours without an incident and having a zero accident rate are not safety metrics. Did you fly accident-free during the past 20,000 hours because you were incredibly lucky or because you were doing the right things correctly, questions Pete Agur.
Effective safety metrics describe activities and behaviors that create the results you intend. The tools for calculating that reality are emerging today. They are not based on rates of historic failure but on measurable anticipated risks, probabilities and mitigations. In other words, the metrics of safety are shifting from hindsight to forward looking.
That is the good news. The not-so-good news is that the development and use of the Risk Assessment Tool (RAT) is in its infancy. Fully developed and implemented, the RAT will be one of the most effective advances in aviation safety to date. In concept, a RAT is a comprehensive list of risk arenas and their elements. For instance, one major risk source is weather. A portion of the RAT assesses all the weather conditions for a trip leg.
Departure airport, route, destination and alternates are all looked at for conditions that create risk, such as icing, high winds, turbulence, thunderstorms, etc. The aviation department staff identifies specific risk elements and determines whether they should be mitigated.
This assessment is done by a scoring process. When a risk and its probability are deemed high enough to be mitigated, lower risk options are considered and selected. For instance, if thunderstorms are forecast en route, is it better to circumnavigate them or depart at a time that precedes or follows their development? With the selected mitigation implemented, the leg is re-scored to confirm that the resulting plan is acceptable.
In its most basic form, a RAT is a score card that has five-by-five blocks (as illustrated here). One scale of reference is event “Probability” ranging from ‘Unlikely’ to ‘Certainly’. The other scale is severity of “Impact” ranging from ‘No Impact’ to ‘Catastrophic’. These cards are easy to use as guidelines for trip planning. Unfortunately, they are inadequate as effective management tools because they are neither comprehensive nor sufficiently detailed to provide management information.
There are a number of commercial RATs being offered. Some have as few as a couple-dozen data points, and others boast hundreds. How good are they? Our firm recently conducted a Flight Risk Assessment Tool (FRAT) research project. The findings were presented at the Flight Safety Foundation’s Business Aviation Safety Seminar this past April. Our observation is that the current crop of FRATs is better than nothing, but are not nearly as good as they will be in an iteration or two.* From your perspective, there are several things you need to consider in the selection and use of a RAT:
DEPTH AND BREADTH
Most RATs include a number of basic data points that are relatively easy to acquire. Aircraft performance, weather, airport information, etc., are routinely included. The better ones include some crew data points. This is a good start, because studies by the National Transportation Safety Board tell us human performance (error) is a major contributor in about 70% of all professionally-flown aircraft accidents.
But none of the RATs we have seen have sufficient crew data. Your Flight RAT should gauge the competence of each crew member as well as the collective competence of the crew as a team. Are they highly experienced in the aircraft? What is the crew’s chronic and acute fatigue status prior to the trip and at the end of each leg? These and many more factors should be identified, measured and scored.
RISK WEIGHT AND ACCRUAL
Most Flight RATs use a simple, linear scoring system. Unfortunately, linear calculations are not reflective of the real potential of a Risk/Probability impact. Therefore, there should be a weighting and accrual algorithm at the heart of the tool. For instance, operating into international airspace includes some communication challenges. If your crew has been to a remote location a number of times, the risk goes down as crew members gain familiarity. But if the trip is a first time arrival into a foreign airport in the mountains at night in low weather, the algorithm must adjust the score for the compounded and accrued risks.
USE THE RAT WIDELY
Most RATs are used only once - as a pre-flight or pre-trip information device. Our study found substantial benefit was gained by scoring the Flight RAT three times, including:
1. Initial trip request (creating a baseline score, and allowing the scheduling team to measure the impact of their contribution to reducing trip risks, and promoting inter-team collaboration - scheduling, maintenance, flight and management).
2. Pre-flight briefing (to provide a guide for the crew’s communications and preparation).
3. Post-flight debriefing (actual trip score changes are identified and trends are noted for future consideration).
You may have noticed I have distinguished between the term RAT and Flight RAT. That is because the vast majority of safety measurement effort thus far has been on the flight side. However, the greatest risk of damage to your aircraft is on the ground. More damage is done to aircraft while they are being moved or while they are parked than when they are in motion for flight. Additionally, more people are injured around aircraft on the ground than while they are in flight. Even so, the development of a Ground RAT for Business Aviation is way behind the Flight RAT.
You may have read the above information and be wondering, “as a Board Member, what do you suggest I do?” You need to confirm that:
1. Your Business Aviation department is using the most advanced Flight RAT available.
2. The Flight RAT is being scored three key times - trip request, pre-flight and post-flight.
3. Management reports based on Flight RAT data are developed and used effectively.
4. Your ground operations will be included in a RAT process of their own, ASAP.
In Business Aviation, there are many ways to skin a cat. That is why it is rare for me to make a specific recommendation. But, in this case, the RAT is king.
*To receive a copy of the FRAT paper, contact the Flight Safety Foundation or our offices. www.vanallen.com or www.flightsafety.org
Do you have any questions or opinions on the above topic? Get them answered/published in World Aircraft Sales Magazine. Email feedback to: Jack@avbuyer.com