Pillars of Aviation Summit 2010
Director of Aviation Safety John McCormick
- Ensuring the Highest Standards of Safety
30 June 2010
I would like to thank the Malaysian Department of Civil Aviation and Malaysia Airports for the invitation to speak to you this afternoon and for hosting this event.
As you know, aviation regulation is a dynamic area because aviation is a dynamic industry. I would like to briefly outline for you some of the key international drivers of change in aviation safety regulation that I see affecting the aviation industry and how we and other civil aviation safety regulators are responding to these challenges.
I had the pleasure of attending the recent International Civil Aviation Organization’s (ICAO) High Level Safety Conference in Montreal earlier this year.
At that conference it was made very clear to me that compliance with ICAO Standards and Recommended Practices has been, and will continue to be, a cornerstone of international civil aviation safety.
However it was also clear that ICAO believes that international civil aviation faces challenges in sustaining an approach to the management of safety based exclusively on compliance with prescriptive requirements.
Where appropriate, a prescriptive-based approach has to be complemented by a performance-based approach. An initial example of this performance-based approach is the implementation of Safety Management Systems (SMS) within the aviation industry.
Such an approach has now been extended by ICAO to member States through the State Safety Program, which we are currently in the process of developing in Australia.
An effective State Safety Program is essential, because ICAO is planning to adopt a continuous monitoring approach to these State Safety Programs as an extension to its Universal Safety Oversight Audit Program (USOAP).
USOAP provides a useful benchmark of States' safety oversight processes and abilities, largely because of the independent nature of the program.
While USOAP has provided us with an important global benchmark of State safety oversight, it is also acknowledged that this type of audit program will inevitably evolve over time. Australia sees logic in moving to a Continuous Monitoring Approach (CMA) and expects to actively participate in this process.
This CMA approach is to include the development and implementation of an on-line reporting and data management system that will allow ICAO to monitor the safety oversight capabilities of States on an ongoing basis. This will allow ICAO to have a clear, accurate and up-to-date profile of States' safety oversight systems. This is something that was not possible under the current USOAP regime. The continuous monitoring approach will become the primary means by which ICAO will assess the performance of States in terms of their respective Safety Programs.
A key point for us to consider in looking at SMS and Australia's own State Safety Program is how we collect, analyse and use data.
As highlighted by ICAO, both SMS and State Safety Programs need constant in-flows of relevant safety data to measure the extent to which the activities contemplated by these systems meet their objectives. Neither system will function without a steady flow of current, accurate and meaningful safety data.
Of course, the identification of such 'relevant' data, and the way in which this data is interpreted and analysed are critical elements in this process-how we handle this data is a key question that ICAO and participants in the High Level Safety Conference recognised requires considerable attention.
The High Level Safety Conference reinforced my belief that, for aviation safety regulators, including my own, to be effective regulators, they need to be data-driven organisations.
A key point of discussion at the conference on the use of data revolved around the protection of sources of information.
While there was general agreement that the disclosure of accident investigation information for the purposes of a criminal prosecution warrants a high level of protection, it was recognised that the disclosure of information for other purposes may warrant a different approach. The relevant ICAO standard on Aircraft Accident and Incident Investigation-paragraph 5.12 of Annex 13 to the Chicago Convention, does not establish a protection regime that is necessarily appropriate for the varying nature of accident investigation information or for the very different purposes for which that information might be disclosed.
Disclosure under standard 5.12 is only allowed if the appropriate authority for the administration of justice ('the justice authority') determines that the need for the disclosure outweighs the adverse domestic and international impact such information may have on the investigation or any future investigation ('the balancing test').
While the balancing test allows flexibility in considering the nature of accident investigation information and the purpose of its disclosure, a problem with this regime is that the Annexes do not provide sufficient guidance to facilitate implementation. Attachment E to Annex 13 explains the principles behind the need for protection but it does not provide guidance on steps which may be taken for the effective administration of the balancing test in a variety of situations.
Moreover, while paragraph 5.12 relates specifically to information derived from accident and incident investigation under the cover of Annex 13, Attachment E to that Annex-which is technically limited to the subject matter of that Annex-purports to extend to the use and application of a much wider-range of information.
There is no shortage of safety data available in international civil aviation. The key issue is to ensure that a steady flow of relevant safety data continues by protecting such data from inappropriate use. Here, of course, the crucial questions must be what constitutes 'inappropriate use' and who decides whether, and in what cases, a particular use is appropriate or inappropriate. Whether the latter question can and should be answered only by judicial authorities is another important question-and one that Australia raised in its well-received Working Paper we presented at the High Level Safety Conference.
In our view, the basic starting premise should be that protection should be to a level commensurate with the nature of the data each source generates, and that such protection should not interfere unduly with the administration justice or the operation of a State's legal system. Beyond this, however, important consideration needs to be given to the extent to which other important safety-related interests might be served by the disclosure and use of otherwise 'protected' data.
Determining the sensitivity of information should involve considering and balancing the significance of two factors - the nature of the information, and the purpose for which it is proposed to be disclosed.
These issues will undoubtedly continue to generate considerable discussion and debate. The High Level Safety Conference recommended that a special working group be established to consider their implications, and the matter will surely arise again at the forthcoming ICAO Assembly later this year in Montreal (28 September to 8 October).
Outcome Based or Prescriptive Regulation
As I mentioned earlier, the prescriptive-based approach to aviation safety has to be complemented by a performance-based, or outcomes-based approach. It is a policy of the Civil Aviation Safety Authority that, in the process of legislative development:
'Wherever possible, the regulations must be drafted to specify the safety outcome required, unless, in the interests of safety, and to address known or likely safety risks, more prescriptive requirements need to be specified'
These same considerations can usefully be taken into account in determining whether an outcome-based or prescriptive approach to the interpretation and application of safety regulations should be adopted, where the legislation allows for such discretion.
In deciding whether to adopt an Outcome-Based or Prescriptive Approach (or some combination of these) in such circumstances - there are five Factors that can and should be considered:
- The nature and complexity of the activity and the outcome;
- The stability (or instability) of the operational environment;
- The willingness and ability of the regulated organisation to take responsibility for their conduct with good judgement and necessary expertise;
- The willingness and ability of the regulator to exercise corresponding measures of expertise and good judgement; and
- A sufficient level of trust between the regulator and the regulated organisation.
These principles are adapted from the work of Professor John Braithwaite, of the Australian National University and highly-regarded expert on regulatory theory and its practical application.
Let me touch briefly on these five factors:
Firstly, we need to consider the nature and complexity of the regulated activity and the outcome we want to be achieved.
Generally, the more complex the activity and the more variable an acceptable outcome might be, the more amenable that situation may be to the specification of an outcome-based expectation or obligation.
For example, the contents of particular provisions of the operations manual of a large, dynamic and sophisticated air transport operation involving the use of several different types of aircraft - should be more outcome-based.
Secondly, consider the stability of the operational environment in which the relevant activities will be carried out.
The greater the frequency and rapidity with which critical features of the operational environment may change, the more amenable that situation may be to an outcome-based requirement or obligation.
For example, in assessing the acceptability of procedures for refuelling with passengers on board for an international charter operator that commonly flies into remote locations with extreme weather conditions, it is more likely that an outcome-based rather than a specific prescriptive requirement will conduce to optimal safety results.
Thirdly, consider the willingness and ability of the organisation to take responsibility for its conduct with good judgement and professional expertise.
Because outcome based obligations require organisations to understand and implement the open-framed requirements by which they will be bound, it is essential that the regulator be confident that they have both the ability and the willingness to do so responsibly.
We must ask ourselves, is this an operator that has consistently demonstrated a high-level of commitment to safety, meeting and exceeding regulatory requirements? Or is this an operator known to seek only to meet minimum requirements, and perhaps to 'creatively' approach its regulatory obligations with a view to minimising their safety-orientated efforts? Are they able (and willing) to devise and refine their own conduct, in response to - and in anticipation of - important safety-related considerations?
To the extent the regulator cannot be confident about these things, there will be a greater need for a more prescriptive approach - specifying in detail, what it is the operator must do, or refrain from doing.
Fourthly, consider also the ability and willingness of the regulatory authority to demonstrate a corresponding level of expertise, integrity and judgement.
The regulatory authority responsible for overseeing and assessing the way in which the organisation fulfils its obligations must itself be able and willing to make competent, informed and even-handed judgments about the sufficiency and effectiveness of the means by which the operator will fulfil its obligations; and whether those obligations are, in fact, being fulfilled.
Take for example where an operator is required to have and operate in accordance with a sophisticated Safety Management System (which will invariably involve a range of 'outcome-based' elements); We must ask ourselves, is the regulator able and willing to understand and make meaningful and reasonable judgements about what is, or is not, being done?
If the regulatory authority is NOT in a position to monitor and assess an organisation's efforts to fulfil outcome-based obligations, it will be necessary - fairer and more conducive to effective safety oversight - to rely on a more prescriptive approach.
Finally, there is a need to consider whether or not a sufficient level of trust exists between the regulatory authority and a regulated organisation to allow for an outcome-based approach to the imposition and assessment of regulatory obligations.
Where provision is made for outcome-based compliance so much of the detail of what would otherwise be specified in prescriptive obligations is 'given over' to the organisation subject to those requirements, the regulatory authority must be confident that the organisation can and will effectively and meaningfully take on what amounts to a higher degree of responsibility for their own safe operation.
The corollary to this proposition, of course, is that the organisations involved in such arrangements need to be confident that the regulatory authority will be fair and reasonable in assessing the organisation's efforts to comply with broadly-framed obligations and expectations - that the regulator will not be arbitrary, capricious, pedantic or inconsistent in its assessments.
In other words, operators must be confident that the regulator will not resort to inappropriately prescriptive forms of assessment and analysis, after having invited or allowed an operator to approach its obligations in an outcome or performance based manner.
This consideration, the need for mutual trust, is very closely related to the 3rd and 4th factors. It might even be identified as the most important factor. But if it is, we need to be cautious in our consideration of the difference between legitimate and tendentious claims-by regulators and the regulated alike-of a deficient display of competence and/or willingness on the other's part, to operate fairly, maturely, honestly, and responsibly under manifestly more flexible outcome-based regulatory regimes.
As a practical matter, then, when considering whether to adopt an outcome-based or more prescriptive approach to the formulation of legislative requirements and in the exercise of discretion in the performance of regulatory functions - a 'prudent mix' of the two will almost certainly be the most appropriate course.