How seriously does your flight department take its safety audits? You’ll need to move beyond ticking the boxes to stop serious safety breaches from accumulating unnoticed.
The word ‘audit’ traces its root to the ancient Latin verb ‘audire’ which means ‘to hear’, or ‘to hear well’. In essence, an audit is a structured set of activities performed to get sufficient appreciation of [to hear well about] how an organization is performing in comparison to a given set of requirements.
It’s important to note that the audit function is only effective if it is undertaken properly. That means avoiding the mere ‘box-ticking’ exercise that demonstrates formal compliance with standards and regulations. For this to happen it’s absolutely necessary that the organization is committed to best practices and that it turns to professional auditors.
The reason that audits are performed and have such significance and frequency is given by the critical mass in management thinking achieved by modern quality management's desire for high-quality results and continuous improvement. Management principles and the ISO standards for quality management (developed in the 1980s), are built around the ‘Plan–Do–Check–Act’ quality circle. Audits are the means to accomplish the ‘Check’ function.
From a safety and operational point of view Business Aviation operations have traditionally been audited against national requirements on aviation safety, which are derived from ICAO Annex 6, Part 1 or 2 - depending on whether the organization is involved in charter operations or in non-commercial operations (including corporate flight departments).
With an almost exclusive focus on safety regulatory requirements, until recently safety audits in Business Aviation have been used eminently as a means to ascertain the degree of compliance of operators with applicable aviation safety regulations. With the advent of safety management concepts and precepts, however, during the last ten years safety auditing has started to take an additional approach. Audits will continue to be performed to monitor compliance with prescriptive regulatory requirements, but auditing efforts will increasingly focus on performance-based safety management to verify that operations are inherently safe, and move beyond pure compliance.
According to ICAO’s safety management manual (Doc. 9859, 3rd edition 2013) audits are an integral part of a safety management system (SMS). More precisely they are part of the safety performance monitoring and management element (which itself falls under the safety assurance component).
“Audits focus on the integrity of the organization’s SMS and its supporting systems. Audits provide an assessment of safety risk controls and related quality assurance processes. Audits may be conducted by entities that are external to the service provider or through an internal audit process having the necessary policies and procedures to ensure its independence and objectivity. Audits are intended to provide assurance of the safety management functions, including staffing, compliance with approved regulations, levels of competency and training.” (Document 9859)
In the SMS framework there is also an increasing need to audit (check) for performance (or risk) based requirements. This task is also accomplished with the safety assurance function of the SMS, which allows the organization to select its own safety monitoring (performance) indicators.
Safety Performance Indicators
Learning to define, interpret and monitor safety performance indicators is the next challenge of safety auditing as part of performance-based safety management in Business Aviation.
Safety performance indicators serve to continuously track the safety performance of the organization against its safety targets, which need to be set by the organization in agreement with the oversight authority. “Parameters for such performance tracking may be occurrence outcomes, deviations or any event types that reflect the safety, quality or risk level of the process,” ICAO clarifies.
Safety performance indicators to be defined and targeted by Business Aviation operators can refer to both higher-level consequences, such as individual/combined fleet monthly serious incident rates (e.g. per 1,000 flight hours), and lower-level consequences, such as percentages of findings per audit; voluntary reports rates; and dangerous goods incident report rates.
Monitoring & Measurement
With regard to performance-based monitoring and measurement, ICAO stresses that data trending charts should be used to track outcomes defining safety performance. “Outcome occurrences should normally be tracked as occurrence rates rather than absolute numbers,” ICAO outlines. “In conjunction with such indicators, alert as well as desired improvement target levels should be set for each indicator. These will serve as markers to define what the abnormal/unacceptable occurrence rate is as well as the desired target (improvement) rate for the indicator.”
On top of setting safety performance targets, ICAO also underlines the importance of setting alert levels serving as the demarcation line between the acceptable the unacceptable region for a safety indicator.
“So long as the occurrence rate for a process does not trend beyond or breach the set alert level criteria, the number of such occurrences is deemed to be acceptable (not abnormal) for that monitoring period,” ICAO highlights. “On the other hand, the aim of a targeted improvement level is to achieve the desired improvement level within a defined future milestone or monitoring period. With such defined alert and target settings, it becomes apparent that a qualitative/quantitative performance outcome can be derived at the end of any given monitoring period. This may be done by counting the number of alert breaches and/or the number of targets achieved for an individual indicator and/or a package of safety indicators.”
Ultimately, auditing for performance-based requirements needs time to be developed as an individual skill and a corporate function, even if it is already required as part of the SMS. Its effects may not be fully evident yet, but we can expect it will eventually lead to an industry that is better aware of the risks faced, and learns to manage them better.