• Operations
  • Individuals
    • Flight Crew
    • Air Traffic Controllers
    • Cabin crew
    • Exams
    • ASICs & AVIDs
    • English Language proficiency
    • Aviation medicals
    • Contact information
  • Aircraft owners
  • Operators
  • Flight Training
  • Aerodromes
  • Office of Airspace Regulation
  • Sport Aviation
  • GAAP
  • Non-towered aerodromes
  • Dangerous goods

Location:

• Home
• Operations
• Individuals
• Exams
• » The CASA Examination System Policies, Processes & Participants

The CASA Examination System
Policies, processes and participants

Introduction

The Australian Civil Aviation Safety Authority (CASA) has an obligation under the Civil Aviation Act 1988 to regulate for aviation safety by issuing flight crew licences. It achieves this by ensuring that flight crew have achieved the minimum standard of skill and knowledge required to operate safely in the complex aviation environment.

Under the same Act, CASA also has a function to promote Australia's civil aviation capabilities. It does this by ensuring that the Flight Crew Licensing (FCL) system should have sufficient quality and integrity not only to promote safety but also to ensure international recognition of the high standard of an Australian licence.

To these ends CASA publishes syllabi of training to enable applicants for flight crew licences and ratings to undergo the required (skill and knowledge) training in the industry, and to attain the required minimum standards.

Determining Aeronautical Knowledge Standards

CASA determines and ensures the achievement of aeronautical knowledge standards of Australian flight crew licence/rating holders through a system of flight crew examinations.

However, the industry is responsible for designing and conducting structured training programmes to achieve the required standards. A well-structured training programme should have its own internal evaluation system which provides for detailed feedback. The CASA regulatory examinations are not part of the industry training evaluation system. CASA examinations serve as public audit tools to determine the standards of knowledge achieved from industry training.

The Examinations

A typical CASA flight crew examination consists of a number of questions that tests the published syllabus of aeronautical knowledge training in as comprehensive a coverage as may be possible under a practical duration of examination session.

CASA has sole discretion in re-structuring the syllabus coverage of an examination, including emphasizing safety focus as necessary on any area of the syllabus, provided the appropriate allowable time for the examination is not compromised.

Currently, CASA examinations consist of two types of questions, namely the multi-choice type and the supply-short-answer type. The latter is restricted to answers involving numerical values only. The web-based CyberExams practice examination has examples of these types of questions.

The questions in an examination may all be of equal marks, or of different marks. CASA strongly advises candidates to check the marks of each question.

The aggregate marks of all the questions in an examination form its maximum possible score. The passing score for the examination is a defined percentage of the aggregate marks of its questions. Therefore all questions in an examination have to be answered to obtain the best possible score.

If a candidate elects not to answer a question, the question will be scored as answered wrongly. A candidate may not appeal against the questions, answered or otherwise, on the basis of his/her personal preference or perception, unless the questions are unanswerable due to incomplete data or information necessary for its solution, or lie outside the syllabus.

The time allocated to an examination is based on the need for a ‘well-prepared’ candidate to answer all the questions. The candidate him/herself has sole discretion and responsibility in managing the examination time.

Candidates are strongly advised to familiarise themselves thoroughly with the CASA style of question and associated rules including the scoring of the examination. For details, read here.

The Examination Assurance Process

The examination system itself has a rigorous structure to ensure that the examination questions and the scoring of candidates' answers are correct and fair. It consists of a multi-step assurance process that regularly involves industry participation.

Development

The process starts with the development of a question by a question writer. The development (or writing) of examination questions is largely contracted out to industry experts. In some limited cases, the writer may be a CASA FOI. After the writer has received instructions from CASA as to what the task involves, he writes the questions in accordance with the following:

• The published syllabus - its scope and standards.
• The "Flight Crew Licensing Specifications to Contractors Writing Examination Questions" Manual.
• Any additional specifications by CASA such as the use of a technical reference publication that is available to the industry (eg. CAR, CAO, AIP, AC Kermode's 'Mechanics of Flight', Roll-Royce's 'The Jet Engine', Hawkin's 'Human Factors in Flight', etc)

The writer is also at liberty to refer to any authoritative books on educational assessments and aeronautical knowledge subjects. If the writer assesses that a topic technical reference, other than that specified by CASA, may be referred to regularly, he/she recommends to CASA for the publication(s) to be listed on the 'Useful References for Examinations'. If CASA accepts the recommendation, the publication would be added to the list, which is available on the CASA website.

During the development phase, the writer applies his/her knowledge and experience in consort with the above specifications and guides to create the required examination question. The writer also regularly consults CASA FOIs or other experts such as, say, a Human Factors/CRM expert in a university, the Bureau of Meteorology, ATSB, etc.

This is a fairly involved process, at times requiring several re-writes before the product is accepted by CASA.

Question Developer

The development phrase of the examination assurance process requires question developers or writers to be qualified industry personalities. Question developers (and moderators) shall have a combination of aviation and academic/educational/teaching qualifications and experience that is appropriate to the level of the examination or rating.

The qualifications for a question writer e:

Mandatory:

• No criminal record or history of repetitive regulatory violations.
• A (current or former) member of the flight crew profession/community.
• Hold or have held the licence, rating or other qualifications at least equivalent to the level of the exam the questions are being written for.*

• * in some limited cases, as for a Human Factors/CRM expert, the aviation qualification need not be as high as the examination level (eg. a HF/CRM expert from a university writing for the ATPL examination may only be required to hold a flight crew icence)

• Good written communication skills in English.
• Excellent knowledge of subject

Preferable:

• Extensive (civil or military) aviation operational experience.
• Extensive (civil or military) instructional, teaching, examining or presentation-skill experience, particularly in aeronautical knowledge subjects.
• Good research capabilities.
• Good industry reputation.

Moderation

After a writer has developed and submitted a question to CASA, the next process is 'moderation'. This involves an independent scrutiny of the question, that is, by a person other than the writer him/herself. Moderation must take place in order for the question to be approved and used in the examination. If the writer is a CASA FOI, an industry person shall moderate that question.

An important assessment in the moderation process is that the question is testing the published syllabus. The process also scrutinizes the question for technical accuracy, presentation, language, and a list of other criteria that ensure the question is 'correct and fair'.

'Correct and fair' means that the question is within the syllabus, comprehensible and answerable by a candidate who has undergone a well structured course of training in accordance with the published syllabus, and who has been assessed by a qualified instructor to have attained the required standards of knowledge.

Therefore CASA examination questions are written to test a "well-prepared candidate" which is defined as follows:

A 'well-prepared' candidate for a particular flight crew examination is one who has completed a well structured course, either under supervision or through self-study, based on a comprehensive coverage of the relevant syllabus, and who has demonstrated achievement sufficient, from a theoretical knowledge consideration, to ensure competent and safe standard of operations for all privileges of the licence or rating.

Validation

Validation is an additional layer in the assurance process of vetting the examination question for correctness and fairness. It is similar to the moderation process, except that it shall be performed wholly by experienced industry instructors. CASA FOIs do not participate in validation exercises.

The validation process has sometimes been incorrectly perceived as a training exercise for aeronautical knowledge instructors, or an opportunity to 'see' and subsequently 'teach to' CASA examination questions.

Apart from ensuring the correctness and fairness of the examination questions, the process of deliberately engaging industry participation brings valuable industry inputs and perspectives into the examination assurance process.

Validation is an on-going activity, occurring at periodic intervals. CASA determines and organizes the validation exercises, and invites industry experts to vet new batches of questions.

Scoring

During the moderation process each question is given a score that is placed in the question library and fully computerised for accurate and extremely fast counting following an examination. No human intervention or interference is possible during the microseconds between the candidate's submission of the examination and the issue of result.

Feedback

The CASA examination system has an automatic feedback mechanism that advises the candidate of the syllabus areas in which he/she has failed to answer questions correctly. This guidance, called a knowledge-deficiency-report (KDR), is issued together with the examination result. The KDR supports the candidate's re-training effort.

The aim of the re-training effort is for the candidate to properly acquire the required knowledge for the licence or rating. Thus, the candidate (and the instructor) is provided the KDR so that they may review and restudy the areas of knowledge deficiency.

Even for a candidate who passes the examination, CASA requires that he/she be re-tested on these noted deficiencies before a flight test. The candidate has to present the KDR to the school's chief flying instructor (CFI) or/and the testing officer (ATO) conducting the flight test for the licence or rating. As part of the test, the CFI or/and the ATO will orally examine the candidate on the noted deficient areas of knowledge. This procedure addresses and enhances safety standards.

Remedial Study

It has sometimes (incorrectly) been perceived that in addition to the examination feedback process, CASA assumes a re-training/tutorial responsibility. However, CASA is not an aeronautical knowledge training organization. CASA as the regulator provides basic feedback in the form of a KDR to help the candidate and the instructor focus their re-training efforts on the areas of noted deficient knowledge. The candidate's instructor continues to be responsible for training and any required re-training.

Another expectation sometimes raised is that the feedback should include the questions that were answered incorrectly as well as their correct answers. Supply of questions and answers would result in narrow rote type learning of specific questions and not contribute to understanding the subject matter as a whole. This would be counter-productive to the proper acquisition of the required knowledge for the licence or rating.

Appeal

CASA has an appeal process that has functioned satisfactorily for nearly two decades. The process enables a candidate to address directly to FCL any concern of being disadvantaged by a perceived error or omission in an examination question.

The sole purpose of this facility is for FCL to examine the merits of the candidate's report/observations and determine whether there are valid grounds for reassessing the examination.

FCL will investigate all proper appeals that contain sufficient information for proceeding with the investigations. If the candidate's report is assessed as valid, FCL will re-mark the exam and advise the candidate of the outcome.

The facility is not to enable a candidate to seek aeronautical knowledge training (or answers to questions) from CASA. Training should be sought from the instructor. Therefore, as a general rule, CASA does not enter into any correspondence with the reporting candidate.

Performance Reporting

Performance reporting indicates to CASA how well an examination, a question, a region, a school or an examination venue, etc has performed over a period of time.

This enables CASA to:

• fine-tune an examination or a question for greater testing effect.
• provide useful knowledge-deficiency feedback to industry for improvement of training courses.
• provide business information to supervisors for improvement of examination services.

The CyberExams system gathers examination data to report on and update examination performance at periodic intervals.

Examination Delivery

What is 'Examination Delivery'?

It means the tasks of processing applications to sit examinations, collecting fees, booking (and re-booking) preferred examination dates/times, administering examination venues, supervising the conduct of the examination, and printing (and reprinting) the result and KDR.

The aim of the 'Examination Delivery' concept is to institute a one-stop-shop for the convenience of the candidate, so that every examination needs of the candidate (other than the 'appeal') may be conducted expeditiously with the industry supervisor.

At the command of the supervisor, the Internet-based examination system delivers the candidate's examination in a timely manner to an approved exam venue. There are several hundreds of such venues located throughout Australia. Similarly, on submission (with the click of a button), the examination is instantaneously marked and the results together with the KDR issued immediately back to the candidate at the venue.

Examination Types

CASA manages the different exams types under two categories, 'professional examinations' and the Private Pilot Licence (PPL) examinations.

The professional examinations include those for the Commercial Pilot Licence (CPL), Air Transport Pilot Licence (ATPL), Flight Engineer Licence (FE), as well as the Flight Instructor, Agricultural and Command Instrument ratings.

The PPL examinations include those for the Aeroplane PPL, Helicopter PPL, and conversion for overseas PPL Aeroplane and Helicopter licences, and the Private IFR rating

Supervisor

CASA has delegated the task of Examination Delivery of the professional examinations to Assessment Services Proprietary Limited (ASL) under contract. ASL may also deliver the PPL examinations.

CASA has approved Chief Flying Instructors (CFI) of approved flying training schools throughout Australia to deliver PPL examinations for the aircraft category (aeroplane or helicopter) that is authorised on the school's AOC. The PIFR examination set by CASA may also be available at schools that are approved to conduct instrument flying training.

Examination Information

An important factor in the successful and stress free sitting of an examination is a thorough understanding of the flight crew procedures and processes involved. These procedures ensure that CASA or its delegate, the supervisor, is aware of the candidate's requirements, and that the candidate in turn will know what to expect at the examination.

Therefore a candidate should familiarize him/herself with all available examination information that supports training as well as successful application for an examination.

CASA Website

CASA posts examination information on its website for the public. Aeronautical knowledge instructors and students are advised to refer regularly to the CyberExams page on the CASA website (www.casa.gov.au).

CASA strongly advises candidates to do this prior to taking up an aeronautical knowledge-training course (or embarking on a self study course), and particularly before attempting an examination.

ASL Website

ASL has its own website (www.aslexam.com), where it posts information on various aspects of 'examination delivery' for which it is responsible. It is essential that candidates and instructors familiarize themselves with the rules, procedures and conditions governing its supervision of examinations.