FITS - Teaching Methods

For Scenario Based Training (SBT) to be effective, it is vital that the pilot in training and the Instructor communicate the following information well in advance of every training flight:

• Purpose of flight
• Scenario destination(s)
• Desired pilot in training learning outcomes
• Desired level of pilot in training performance
• Desired level of automation assistance
• Possible in-flight scenario changes (during later stages of the program)

With the guidance of the Instructor, the pilot in training should make the flight scenario as realistic as possible. This means the pilot in training will know where they are going and what will transpire during the flight. While the actual flight may deviate from the original plan, it allows the pilot in training to be placed in a realistic scenario.

Prior to the flight, the Instructor will brief the scenario to be planned. The Instructor will review the plan and offer guidance on how to make the lesson more effective. Discussion, in part, will reflect ways in which the Instructor can most effectively draw out a pilot in training's knowledge and decision processes. This enables the Instructor to analyze and evaluate the pilot in training’s level of understanding. After discussion with the Instructor, the pilot in training will plan the flight to include:

• Reason to go flying
• Route
• Destination(s)
• Weather
• Notams
• Desired pilot in training learning outcomes
• Possible alternate scenarios and emergency procedures

Consider the following example: The Instructor provides a detailed explanation on how to control for wind drift. The explanation includes a thorough coverage of heading, speed, angle of bank, altitude, terrain, and wind direction plus velocity. The explanation is followed by a demonstration and repeated practice of a specific flight maneuver, such as turns around a point or S turns across the road until the maneuver can be consistently accomplished in a safe and effective manner within a specified limit of heading, altitude, and airspeed. At the end of this lesson, the pilot in training is only capable of performing the maneuver.

Now, consider a different example: The pilot in training is asked to plan for the arrival at a specific uncontrolled airport. The planning should take into consideration the possible wind conditions, arrival paths, airport information and communication procedures, available runways, recommended traffic patterns, courses of action, and preparation for unexpected situations. Upon arrival at the airport the pilot in training makes decisions (with guidance and feedback as necessary) to safely enter and fly the traffic pattern. This is followed by a discussion of what was done, why it was done, the consequences, and other possible courses of action and how it applies to other airports. At the end of this lesson the pilot in training is capable of explaining the safe arrival at any uncontrolled airport in any wind condition.

The first example is one of traditional learning, where the focus is on the maneuver. The second is an example of scenario-based training, where the focus is on real world performance. Many learning developers in flight training have built on the former option. Traditional training methods in many instances are giving way to more realistic and fluid forms of learning. The aviation industry is moving from traditional knowledge-related learning outcomes to an emphasis on increased internalized learning in which learners are able to assess situations and appropriately react. Knowledge components are becoming an important side effect of a dynamic learning experience.

Reality is the ultimate learning situation and scenario-based training attempts to get as close as possible to this ideal. In simple terms, scenario-based training addresses learning that occurs in a context or situation. It is based on the concept of situated cognition, which is the idea that knowledge cannot be known and fully understood independent of its context. In other words, we learn better, the more realistic the situation is and the more we are counted on to performs.

Michael Hebron, a well-known golf instructor, suggests that there is little the expert can do in the way of teaching the learner particular motions of the golf swing. Instead, learning has to be experiential and feedback based; only a handful of basic principles are involved. The same goes, he says, for any and all kinds of learning. “It’s about learning, not about golf.”

Scenario-based training (SBT) is similar to the experiential model of learning. The adherents of experiential learning are fairly adamant about how people learn. They would tell us that learning seldom takes place by rote. Learning occurs because we immerse ourselves in a situation in which we are forced to perform. We get feedback from our environment and adjust our behavior. We do this automatically and with such frequency in a compressed timeframe that we hardly notice we are going through a learning process. Indeed, we may not even be able to recite particular principles or describe how and why we engaged in a specific behavior. Yet, we are still able to replicate the behavior with increasing skill as we practice. If we could ask Mark MacGuire to map out the actions that describe how he hits a home run, he would probable look at us dumbfounded and say, “I just do it.” On the other hand, I am sure Mark MacGuire could describe in detail the size and characteristics of every one of the baseball diamonds he was playing in as well as the strengths, weaknesses and common practices of every one of the pitchers he faced.

Scenario-based training best fits an open philosophy of blended and multiple learning solutions in which change and experience are valued and the lines between training and performance improvement are blurred. For scenario-based training to be effective it must generally follow a performance improvement imperative. The focus is on improved outcomes rather than the acquisition of knowledge and skills. Success requires a blended, performance-based, and reinforced solution.

An athletic exercise such as Basketball might prove to be a very good example. Clearly, the team’s objective is to win, which means scoring more points than the other team. That’s the performance objective. Each member of the team also has personal performance goals. The coach can stand at a blackboard and explain defensive and offensive diagrams with players, the rules of the game, and so forth. By doing that, he has identified a set of learning subjects (rules and play patterns) that are best delivered in a traditional fashion.

On the other hand, the application of these subjects and the level of proficiency required in their use can only be learned on the court. The scenario in this example is a scrimmage. During a typical scrimmage, experienced players are mixed with nonexperienced players and matched against a similarly constituted practice team. The two teams play a game, and the coaches stop the action at appropriate intervals to offer feedback. Learning takes place in a highly iterative fashion often without the player realizing that specific bits of learning are taking place. The scrimmage provides a player with the opportunity to make several decisions, engage in complex and fast-paced behaviors, and immediately see impact. The coach may have some general ideas of basketball in mind and perhaps some specific learning objectives for the day, but in most cases does not know precisely which of them will be addressed during the scrimmage – that depends on the flow of practice.

Similarly, most flight training consists of both kinds of subjects: those amenable to traditional instructional design techniques and those better approached through scenario-based training. Neither is all that useful without the other. Before a learner can engage in a scenario, he or she needs some basic subject knowledge and skill. However, the strongest adherents of the scenario-based approach suggest very little subject knowledge is needed in order to take advantage of SBT. The main point is that knowledge without application is worth very little.

The first step in the scenario design process is to engage a number of subject matter experts in a series of discovery sessions and interactive meetings for the purpose of identifying issues and learning objectives including higher-level and performance objectives. With clearly identified learning objectives, appropriate techniques and where to use they can be specified. In the basketball example, players need some rudimentary knowledge of the game and basic skill in order to make the practice session efficient and effective. Consequently, the required knowledge and skill objects need to be integrated into the actual sessions of practice. So, like a train pulling a number of boxcars, a traditional piece of learning precedes or is integrated into a scenario, with the scenario dictating what information is covered in the traditional piece. If, as described in the scrimmage session above, you don’t precisely know what will come up in the practice, you shouldn’t waste time in the traditional preparation. It’s more efficient to share very basic principles and devote your resources to preparing to teach any situation that may arise. What is important, however, is to establish the boundaries of the scenarios. These are done using performance-based learning objectives (Internalized Responses) as opposed to knowledge-based learning objectives, and are worded as performance objectives rather than skill-based behavior objectives.

For example, in the traditional, more repetitive, intensive flight training sessions, objectives are knowledge-based and tend to be specific and limited. On the other hand, in scenario-based training we are simply trying to determine whether the learner has the minimum necessary knowledge/skill to qualify for the scenario. With scenario-based objectives, we are looking for performance behaviors and indicators of internalized responses, which are usually situational recognition indicators.

We can see this clearly illustrated in an automobile driver-training example (Table 1). The traditional Behavior (skill) objective is knowledge based and the SBT Performance objective is performance-based (responses which are situational recognition indicators).

Table 1: Driving Learning Objectives

Scenario design sessions should resemble focus groups in which participants work through a series of issues, from broad scenario outlines to very specific scenario details. Direct participants to address two general areas: content and style.

Sessions to determine content usually ask participants to:

• Share experiences about the subject event
• Describe desirable outcomes
• Share best practices or known instances of consistent achievement of the desired outcomes
• Create indicators of successful outcomes
• Create strategies expected to lead to successful outcomes
• Establish descriptions of successful and unsuccessful performance behaviors related to these strategies (note that outcome measures and performance behaviors will constitute the evaluative criteria for assessing performance in the scenario).

After the content discussion, ask participants to review the look, feel, and flow of the scenario. This is much like the process used for instructional design. Develop a storyboard with a general beginning and end, using the boundaries established earlier. Talk through the scenario in the session and, through iteration, create a flow script from the results.

With these two elements in place, you can begin the actual construction of the scenario. A subcommittee of Flight Instructors and subject matter experts (SMEs) should review and revise the scenario to fit into the whole course of instruction.

Scenarios are meant to be real situations. In an ideal world, an assessment team would evaluate behavior and agree on several critical performance dimensions. The key indicators should come from the initial SMEs, in which they also create strategies expected to lead to successful outcomes and establish descriptions of successful and unsuccessful performance behaviors. Outcome measures and performance behaviors will constitute the evaluative criteria for assessing performance in the scenario.

Examples of indicators of successful outcomes are whether an airplane arrived and was secured at the destination airport and how safe were all aspects of the flight or were there any regulatory violations. Strategies are clusters of internally consistent behaviors directed toward the achievement of a goal. Performance behaviors are the key behaviors in those strategies. Establishing these dimensions should be a group process and is usually completed in the subject matter expert design session.

Review, obtain learner feedback, and revise. All learning, even the most traditional, is iterative. The key to creating a useful scenario is to see it as a learning experience for the designers as well as the learners. This means that results and comments about the learning experience are shared with the SMEs and the designer so that they can review and modify the scenarios as necessary. Obtain open –ended qualitative data from the learner and the Flight Instructor about the experience and review the data with the SME's and the designer.

Based on this kind of feedback, scenarios can be revised to better target the learner population. That process mirrors the original design steps. There are some cautions, however, in the revision process. First, there is an old saying: “It doesn’t take a cannon to blow away a tin can.” Basically, revisions should not needlessly complicate the scenario or the technology needed to employ it. It is crucial to weigh the risks of complication against the genuine learning needs. Before any revision, affirm the original purpose statement and the categorization of learning elements.

Also, do not let principles and main points become diluted by revisions. It is tempting to add more items and nuances in a scenario, but doing so further complicates the learning process. Save complexity for a full-scale “capstone” experience. Remember, adding an item in traditional learning complicates the learning process in a linear fashion. In scenarios, complication grows non-linearly with the addition of learning items. So, beware. A rule of thumb is to reduce rather than increase principles and main points in a revision.

Always review success and failure paths for realism. Remember that any change in a scenario item complicates all items on the path following it. Any time a decision node is altered, chances are that the decision nodes and information items following it must change. With every revision, follow and ensure the consistency of associated paths.

Finally, remember that traditional learning elements should service the scenario-based learning elements, which are situated in a real context and based on the idea that knowledge cannot be known and fully understood independent of its context. It is essential to place boundaries around scenarios to make the transitions between scenarios and traditional learning as efficient as possible.

• Scenario-based training (SBT) is situated in a real context and is based on the idea that knowledge cannot be known and fully understood independent of its context.
• SBT accords with a performance improvement and behavior change philosophy of the learning function.
• SBT is different from traditional instructional design and one must be aware of the differences to successfully employ SBT.
• All learning solutions should employ both traditional and scenario-based training.
• Traditional learning elements should service the scenario-based training elements.
• It is essential to place boundaries around scenarios to make the transitions between scenarios and traditional learning as efficient as possible.
• Use interactive discovery techniques with subject matter experts (SMEs) and designers to establish the purpose and outcomes of scenarios create the scenarios and appropriate strategies and performance behaviors, and develop learner evaluation criteria.
• SBT occurs by following success and failure paths through a realistic situation. Typically, these paths must be limited to stress the main learning objective. Otherwise the scenario can become too complex and unwieldy.
• Open-ended qualitative learner feedback is key to successful scenario revision, but revisions should not further complicate the scenario unless highly justified.

Kindley, R. (2002). Scenario-Based E-Learning: A Step Beyond Traditional E-Learning. Retrieved 02/02/05 from http://www.learningcircuits.org/2002/may2002/kindley.html

The art and science of managing all the resources (both on-board the aircraft and from outside sources) available to a single-pilot (prior and during flight) to ensure that the successful outcome of the flight is never in doubt. Most of us remember a favorite Instructor from our past that showed us the best way to solve in-flight problems and unforeseen circumstances. The FITS team has combined much of this collective CFI body of knowledge with some innovative teaching methods to give pilots practical tools to teach aeronautical decision-making and judgment. It is called Single Pilot Resource Management (SRM). SRM includes the concepts of Aeronautical Decision Making (ADM), Risk Management (RM), Task Management (TM), Automation Management (AM), Controlled Flight Into Terrain (CFIT) Awareness, and Situational Awareness (SA). SRM training helps the pilot maintain situational awareness by managing the automation and associated aircraft control and navigation tasks. This enables the pilot to accurately assess and manage risk and make accurate and timely decisions. This is what SRM is all about, helping pilots learn how to gather information, analyze it, and make decisions.

Teaching pilots to identify problems, analyze the information, and make informed and timely decisions is one of the most difficult tasks for Instructors. By way of comparison, the training of specific maneuvers is fairly straightforward and reasonably easy to understand. We explain, demonstrate, and practice a maneuver until proficiency is achieved. We are teaching the pilot in training “what to think” about each maneuver and sign them off when they demonstrate proficiency. Teaching judgment is harder. Now we are faced with teaching the pilot in training “how to think” in the endless variety of situations they may encounter while flying out in the “real world.” Often, they learn this by watching Instructors. They observe reactions, and more importantly, actions, during flight situations and they often adapt the styles of the Instructor to their own personalities.

The SRM scenarios, developed by the FITS team, incorporate several maneuvers and flight situations into realistic flight scenarios. The scenarios are much like the Line Oriented Flight Training (LOFT) employed by the major corporate and airline training organizations for years. Pilots in training may range from 100-hour VFR-only pilots, all the way to multi-thousand hours ATP’s. The strength of this format is that the participants learn not only from their Flight Instructor, but from each other as well. The collective knowledge of many pilots, when guided by an experienced CFI, is much greater than the knowledge of each participant, including the Flight Instructor. In these scenarios, there are no right answers, rather each pilot is expected to analyze each situation in light of their experience level, personal minimums, and current physical and mental readiness level, and make their own decision.

Table 2: Single Pilot Resource Management (SRM)

IMC during VFR flight. b. During system and navigation failures and physiological incidents during IFR flight. |Note: All tasks under SRM will be embedded into the curriculum and the training will occur selectively during all phases of training. SRM will be graded as it occurs during the training scenario syllabus. |

SRM sounds good on paper, however, it requires a way for pilots to understand and deploy it in their daily flights. This practical application is called the “Five P’s (5P’s)” The 5P’s consist of “the Plan, the Plane, the Pilot, the Passengers, and the Programming”. Each of these areas consists of a set of challenges and opportunities that face a single pilot. And each can substantially increase or decrease the risk of successfully completing the flight based on the pilot’s ability to make informed and timely decisions. The 5P’s are used to evaluate the pilot’s current situation at key decision points during the flight, or when an emergency arises. These decision points include, pre-flight, pretakeoff, hourly or at the midpoint of the flight, pre-descent, and just prior to the final approach fix or landing.

The 5P’s are based on the idea that the pilots have essentially five variables that impact his or her environment and that can cause the pilot to make a single critical decision, or several less critical decisions, that when added together can create a critical outcome. These variables are the Plan, the Plane, the Pilot, the Passengers, and the Programming. The authors of the FITS concept felt that current decision-making models tended to be reactionary in nature. A change has to occur and be detected to drive a risk management decision by the pilot. For instance, many pilots ascribe to the use of risk management sheets that are filled out by the pilot prior to takeoff. These catalog risks that may be encountered that day and turn them into numerical values. If the total exceeds a certain level, the flight is altered or cancelled. Informal research shows that while these are useful documents for teaching risk factors, they are almost never used outside of formal training programs. The number of pilots who use them before each and every flight approaches zero. The 5P concept is an attempt to take the information contained in those sheets, and in the other available models and operationalize it.

The 5P concept relies on the pilot to adopt a “scheduled” review of the critical variables at points in the flight where decisions are most likely to be effective. For instance, the easiest point to cancel a flight due to bad weather is before the pilot and passengers walk out the door and load the aircraft. So the first decision point is Pre-Flight in the flight planning room, where all the information is readily available to make a sound decision, and where communication and FBO services are readily available to make alternate travel plans.

The second easiest point in the flight to make a critical safety decision is just prior to takeoff. Few pilots have ever had to make an “emergency take-off”. While the point of the 5P check is to help you fly, the correct application of the 5P before takeoff is to assist in making a reasoned go-no-go decision based on all the information available. That decision will usually be to “go”, with certain restrictions and changes, but may also be a “no-go”. The key point is that these two points in the process of flying are critical go-no go points on each and every flight.

The third place to review the 5Ps is at the mid point of the flight. Often, pilots may wait until the ATIS is in range to check weather, yet at this point in the flight many good options have already passed behind the aircraft and pilot. Additionally, fatigue and low altitude hypoxia serve to rob the pilot of much of their energy by the end of a long and tiring flight day. This leads to a transition from a decision-making mode to an acceptance mode on the part of the pilot.

The last two decision points are just prior to decent into the terminal area and just prior to the final approach fix as preparations for landing commence. Most pilots execute approaches with the expectation that they will land out of the approach every time. A healthier approach requires the pilot to assume that changing conditions (the 5Ps again) will cause the pilot to divert or execute the missed approach on every approach. This keeps the pilot alert to all manner of conditions that may increase risk and threaten the safe conduct of the flight. Diverting from cruise altitude saves fuel, allows unhurried use of the autopilot, and is less reactive in nature. Diverting from the final approach fix, while more difficult, still allows the pilot to plan and coordinate better, rather than executing a futile missed approach. Now lets look in detail at each of the “Five P’s”.

The “Plan” can also be called the mission or the task. It contains the basic elements of cross country planning, weather, route, fuel, publications currency, etc. Unlike risk management sheets that pilot fill out before a flight, the “Plan” should be reviewed and updated several times during the course of the flight. A delayed takeoff due to maintenance, fast moving weather, and a short notice Temporary Flight Restriction (TFR) may all radically alter the plan. Several excellent flight planning software packages are available that automate this process, allowing the pilot additional time to evaluate and make decisions. Some include real time and graphical TFR depictions. The “plan” is not just about the flight plan, but the entire days events surrounding the flight and allowing the pilot to accomplish the mission. The plan is always being updated and modified and is especially responsive to changes in the other four remaining P’s. If for no other reason, the 5P check reminds the pilot that the day’s flight plan is a “living” document, subject to change at any time.

Obviously the weather is a huge part of any “plan.” The addition of real time data link weather information give the TAA pilot a real advantage in inclement weather, but only if the pilot is trained to retrieve, and evaluate the weather in real time without sacrificing situational awareness. And of course, weather information should drive a decision, even if that decision is to continue on the current “plan.”

Both the “plan” and the “plane” are fairly familiar to most pilots. The “plane” consists of the usual array of mechanical and cosmetic issues that every aircraft pilot, owner, or operator can identity. However, with the advent of the Technically Advanced Aircraft (TAA), the “plane” has expanded to include database currency, automation status, and emergency backup systems that were unknown a few years ago. Much has been written about single pilot IFR flight both with, and without, an autopilot. While this is a personal decision, it is just that, a decision. Low IFR in a non-autopilot equipped aircraft may depend on several of the other “P’s” we will discuss. Pilot proficiency, currency, and fatigue are among them. The TAA offers many new capabilities and simplifies the basic flying tasks, but only if the pilot is properly trained and all the equipment is working as advertised.

This is an area all pilots are learning more and more about each day. Technically Advanced Aircraft (TAA), especially when used for business transportation, expose the pilot to more high altitude flying, long distance and endurance, and more challenging weather simply due to their advanced capabilities. The traditional “IMSAFE” checklist is a good start. However, each of these factors must be taken in consideration of the cumulative effect of all of them together and the insidious effects of low altitude hypoxia. The authors informal survey of TAA pilots show that almost half fly with pulse oxymeters to display the effects of low altitude hypoxia in a graphic manner.

The combination of late night, pilot fatigue, and the effects of sustained flight above 5,000 feet may cause pilots to become less discerning, less critical of information, less decisive and more compliant and accepting. Just as the most critical portion of the flight approaches (for instance a night instrument approach, in the weather, after a four hour flight) the pilot’s guard is down the most. The “5P” process emphasizes that pilot recognize the physiological situation they are placing themselves in at the end of the flight, before they even takeoff, and continue to update their condition as the flight progresses. Once identified, the pilot is in an infinitely better place to make alternate plans that lessen the effect of these factors and provide a safer solution.

One of the key differences between CRM and SRM is the way passengers interact with the pilot. In the airline industry the passengers have entered into a contractual agreement with the pilots company with a clearly defined set of possible outcomes. In corporate aviation, the relationship between crew and passengers is much closer, yet is still governed by a set of operating guidelines and the more formal lines of corporate authority. However, the pilot of a highly capable single engine aircraft has entered into a very personal relationship with the passengers, in fact, they sit within an arms reach all of the time.

It may be easy, especially in business travel, for the desire of the passengers to make airline connections or important business meetings to enter into the pilot’s decisionmaking loop. If this is done in a healthy and open way, it is a very positive thing. However, this is not always the case. For instance, imagine a flight to Dulles Airport and the passengers, both close friends and business partners, need to get to Washington D.C. for an important meeting. The weather is VFR all the way to southern Virginia then turns to low IFR as the pilot approaches Dulles. A pilot employing the 5P approach might consider reserving a rental car at an airport in northern North Carolina or southern Virginia to coincide with a refueling stop. Thus, the passengers have a way to get to Washington, and the pilot has an out to avoid being pressured into continuing the flight if the conditions do not improve.

Passengers can also be pilots. The old joke says that when four Certified Flight Instructors (CFI) board a light general aviation, a NOTAM should be posted. There is some truth to this. If no one is designated as pilot in command and unplanned circumstances arise, the decision-making styles of four self confident CFI’s may come into conflict. Another situation arises when an owner pilot flies with a former CFI in the right seat on a business trip. Unless a clear relationship is defined and briefed prior to the flight, the owner pilot may feel some pressure to perform for the Individual Learning Manager (possibly beyond his or her capability), and the Individual Learning Manager may feel inhibited from intervening in small decisions until it is clearly evident that the pilot is making poor decisions. This is actually a CRM situation and requires clear preflight understanding of roles, responsibilities, and communication. Non-Pilots can also cause the pilot to review the SRM process.

Pilots need to understand that non-pilots may not understand the level of risk involved in the flight. There is an element of risk in every flight. That’s why SRM calls it risk management not risk elimination. While a pilot may feel comfortable with the risk present in a night IFR flight, the passengers may not and may manifest this during the flight. The human reaction to fear and uncertainty is as varied as the shapes of our ears. Some become quiet, some talk incessantly, and in extreme cases anger and fear are strongly manifested. This may be the last thing the pilot needs to deal with while shooting the ILS to 400 feet and a mile visibility at midnight.

A pilot employing SRM should ensure that the passengers are involved in the decisionmaking and given tasks and duties to keep them busy and involved. If, upon a factual description of the risks present, the passengers decide to buy an airline ticket or rent a car, then a good decision has generally been made. This discussion also allows the pilot to move past what he or she “thinks” the passengers want to do and find out what they “actually” want to do. This removes a load of self-induced pressure from the pilot.

The TAA adds an entirely new dimension to the way General Aviation aircraft are flown. The Glass Cockpit, GPS, and Autopilot are tremendous boons to reduce pilot workload and increase pilot situational awareness. And frankly, the programming and operation of these devises is fairly simple and straightforward. However, unlike the analog instruments they replace, they tend to capture the pilot’s attention and hold it for long periods of time (like a desktop computer). To avoid this phenomenon, the pilot should plan in advance when and where the programming for approaches, route changes, and airport information gathering should be accomplished…as well as times it should not. Pilot familiarity with the equipment, the route, the local air traffic control environment, and their own capabilities vis-à-vis the automation should drive when, where, and how the automation is programmed and used.

The pilot should also consider what his or her capabilities are in response to last minute changes of the approach (and the reprogramming required) and ability to make largescale changes (a re-route for instance) while hand flying the aircraft. Since formats are not standardized, simply moving from one manufacturer’s equipment to another should give the pilot pause and require more conservative planning and decisions.

The SRM process is simple. At least five times, before and during the flight, the pilot should review and consider the “Plan, the Plane, the Pilot, the Passengers, and the Programming” and make the appropriate decision required by the current situation. It is often said that failure to make a decision is a decision. Under SRM and the 5P’s, even the decision to make no changes to the current plan, is made through a careful consideration of all the risk factors present.

The teaching of SRM is best accomplished in a seminar environment. Recently, the authors conducted a set of classroom seminars that presented real time flight scenarios to a room full of qualified pilots of varied experiences. The first scenario presented was a night MVFR/IFR flight from St Augustine Florida to Washington Dulles Airport. The original “Plan” called for a non-stop flight with a 45-minute fuel reserve. The “Plane” was a well-equipped TAA with a minor navigation light problem that delayed departure by an hour. The “Passengers” were one pilot and one non-pilot. The non-pilot seemed nervous about the trip and a little ill. Both passengers needed to get to Washington DC for an important meeting the next day. The “Pilot” had spent a full day at a flight refresher clinic, including a two-hour flight and a three-hour class, and felt reasonably refreshed at the 5 PM departure time. And finally, the GPS/MFD, the “Programming,” combination looked like it would make the flight a snap. However, there were questions about the currency of the database that required the pilot’s attention.

The discussion that followed revolved around the reliability of the weather data, the fatigue of the pilot landing at Dulles at 9 PM, alternate ways to get the passengers to their meeting, minimum requirements for aircraft night flight, and a more complete understanding of the benefits and challenges posed by GPS programming and database currency. The 5p’s ensured that each pilot looked at the entire picture prior to making the critical decisions that would lay the groundwork for success or failure over four hours later in Washington.

Predictably, the destination weather deteriorated slowly as the flight proceeded northbound. The pilot’s fatigue level, low altitude/long duration hypoxia, a succession of minor annoyances caused by the airplane and the passengers, began to become a factor. Again, the pilots applied the 5p’s, and many decided to land short of Washington Dulles, check the weather, and secure a rental car as a backup for the Monday morning meeting (in fact many decided this prior to takeoff).

For the purposes of the discussion, this aircraft was equipped with a ballistic parachute system. For those that proceeded to Dulles, the scenario ended with a spatial disorientation incident at 1500 feet, 10 miles short of the airport caused by pilot fatigue, latent hypoxia, and failure to use the autopilot. For many, it was the first time they had considered all the options available, and the criticality of quick and accurate decisions. In the background, another Individual Learning Manager began calling out altitudes and speeds as the aircraft descended to the ground, providing an added dose of realism and pressure. Should the class initiate an unusual attitude recovery, and if it did not work should they attempt another? How much will the passengers help or hinder the pilots thought processes? When, and how, should the ballistic parachute system be deployed, and what are its limitations. This scenario sparked questions about the capabilities and limitations of the autopilot, cockpit automation, and the parachute system. More importantly, it caused the pilots in the room to examine how they should gather critical information, assess the risks inherent in the flight, and take timely action. All agreed that a few accurate decisions before and during the early part of the flight reduced the risk to pilot and passengers.

All these questions were discussed in a lively thirty-minute session following the scenario. In this type of Scenario Based Training, the group discussion is just as important as the actual situation, for it is during the discussion that the pilots are most ready to learn, and begin to develop a mental model of how they might react to situations. Instead of encountering a once in a lifetime, life or death, situation alone on the proverbial dark and stormy night, the participants could examine how the situation had developed, understand the options available to them, and begin to develop a general plan of action well ahead of time.

The third component of the FITS training method, following each flight scenario, is to use the concept of “learner-centered grading.” Learner centered grading includes two parts: learner self assessment and a detailed debrief by the instructor. The purpose of the self assessment is to stimulate growth in the learner’s thought processes and, in turn, behaviors. The self-assessment is followed by an in-depth discussion between the instructor and the pilot in training which compares the instructor ratings to the pilot in training’s self-assessment.

To improve learning, it is recommended that learners prepare to learn from their experiences both before and after key events. This preparation should increase learning and enhance future performance. Pre-briefs are essential for setting goals. During key events, especially those that require high levels of attention, there may be little time for learning; most individuals allocate the bulk of their cognitive resources to performing the actual task; however, they may also dedicate some cognitive resources to self-monitoring, learning, and correction.

How facilitation and feedback occur is important to the learning process. In order for feedback to be useful for both informational and motivational purposes, it should be designed systematically. For example, the facilitator (Flight Instructor) should avoid lecturing the learner, and should withhold their observations and opinions of the exercise until the learner has given their opinion. The use of closed-ended questions may stymie the usefulness of the feedback process as well, as they encourage one-word/yes/no types of answers that do not elicit opinions of performance or suggestions for improvement. It is more effective to use open-ended questions that probe the learner to assess their own performance. Allotting enough time for the feedback is also important. Debriefs that are rushed often turn into one-way “lectures” due to time constraints. Referring to prior pre-briefs when conducting subsequent debriefs provides a sense of continuity, reliability, and consistency, all of which are desirable attributes of a feedback source. Reminding learners of goals and lessons learned from prior exercises helps them plan for future events. Learners may also be more receptive to feedback during a debrief if they were appraised of the goal criteria in a pre-brief.

The FITS approach utilizes scenarios to teach Single Pilot Resource Management (SRM) while simultaneously teaching individual tasks such as landings and takeoffs. The authors quickly realized that this required a new approach to the pilot in training's performance measurement. Traditional grading approaches are generally teacher centered and measure performance against an empirical standard. The following example of a traditional flight syllabus demonstrates.

A Traditional Grading Scale

• Excellent - the pilot in training has performed in an excellent manner
• Good – the pilot in training has exceeded basic requirements
• Satisfactory – the pilot in training has met basic standards
• Marginal – the pilot in training has failed to perform the task standards
• Unsatisfactory – the pilot in training has demonstrated significant performance difficulties

A Traditional Lesson

This type of grading scale is in wide use throughout the aviation training industry. While it appears to be based on published standards, in reality it is often used as a tool to determine pilot in training progress and provide motivation. Thus, on the first lesson a pilot in training may receive an “Excellent” grade for attempting to plan the flight and accomplishing the weight and balance with a few minor errors. However, by the third flight, that same performance may only earn a “Satisfactory” grade due to lack of pilot in training progress (note that while performance remained the same, the grade changed). Additionally, the Flight Instructor awards the grade based on his or her observation of the pilot in training's performance. This observation, while accurate, may not be based on an understanding of the pilot in training’s level of knowledge and understanding of the task. Lastly, the pilot in training has been conditioned since grade school to look at grades as a reward for performance and may feel that there is a link between grades earned and their self-esteem. In reality, none of this aids pilot in training performance in any meaningful way.

The learner centered grading approach addresses the above concerns. First, the grade is now a “Desired Scenario Outcome.” These outcomes describe pilot in training-learning behavior in readily identifiable and measurable terms. They reflect the pilot in training’s ability to see, understand, and apply the skills and tasks that are learned to the scenario.

For instance, a pilot in training who can “explain” a successful landing has achieved the basic level of competence to begin the learning process. Once the pilot in training can “explain” the effect of crosswind and speed reduction on rudder effectiveness, they have achieved a level of learning that will allow for meaningful “Practice.” The “Perform” level denotes unsupervised practice and self-correction of errors. These grades are equally applicable to the first scenario to the last since they are not lesson dependent.

The grade of “Manage/ Decide” is used solely for SRM grading and the grade of “Perform” is used solely for task grading. A pilot in training who is becoming proficient at aeronautical decision-making and risk management would be graded first at the “Explain” level, then at the “Practice”, and finally at the “Manage/Decide” level. A Manage/Decide or Perform grade does not describe perfection. Rather, these grades simply show a proficient pilot who corrects their own errors so that the outcome of the flight is never in doubt. Realistically, this is the performance level we desire. All pilots make mistakes, it is in learning to identify and correct mistakes that they become proficient pilots.

The objective of scenario-based training is a change in the thought processes, habits, and behaviors of the pilot in training during the planning and execution of the scenario. Since the training is learner centered, the success of the training is measured in the following desired pilot in training outcomes.

(a) Maneuver Grades (Tasks)

• Describe – at the completion of the scenario, the PT will be able to describe the physical characteristics and cognitive elements of the scenario activities. Instructor assistance is required to successfully execute the maneuver.
• Explain – at the completion of the scenario the PT will be able to describe the scenario activity and understand the underlying concepts, principles, and procedures that comprise the activity. Significant instructor effort will be required to successfully execute the maneuver.
• Practice – at the completion of the scenario the pilot in training will be able to plan and execute the scenario. Coaching, instruction, and/or assistance from the CFI will correct deviations and errors identified by the CFI.
• Perform – at the completion of the scenario, the PT will be able to perform the activity without assistance from the CFI. Errors and deviations will be identified and corrected by the PT in an expeditious manner. At no time will the successful completion of the activity be in doubt. (“Perform” will be used to signify that the PT is satisfactorily demonstrating proficiency in traditional piloting and systems operation skills)
• Not Observed – Any event not accomplished or required

(b) Single Pilot Resource Management (SRM) Grades

• Explain – the pilot in training can verbally identify, describe, and understand the risks inherent in the flight scenario. The pilot in training will need to be prompted to identify risks and make decisions.
• Practice –the pilot in training is able to identify, understand, and apply SRM principles to the actual flight situation. Coaching, instruction, and/or assistance from the CFI will quickly correct minor deviations and errors identified by the CFI. The pilot in training will be an active decision maker.
• Manage/Decide - the pilot in training can correctly gather the most important data available both within and outside the cockpit, identify possible courses of action, evaluate the risk inherent in each course of action, and make the appropriate decision. Instructor intervention is not required for the safe completion of the flight.
• Not Observed – Any event not accomplished or required

Grading will be conducted independently by the pilot in training and the instructor, and then compared during the post flight critique.

Learner centered grading (outcomes assessment) is a vital part of the FITS concept. Previous syllabi and curriculum have depended on a grading scale designed to maximize pilot in training management and ease of instructor use. Thus the traditional: “excellent, good, fair, poor” or “exceeds standards, meets standards, needs more training” often meet the instructor’s needs but not the needs of the pilot in training. The learner centered grading described above is a way for the instructor and pilot in training to determine the pilot in training’s level of knowledge and understanding. “Perform” is used to describe proficiency in a skill item such as an approach or landing. “Manage-Decide” is used to describe proficiency in the SRM area such as ADM. Describe, explain, and practice are used to describe pilot in training learning levels below proficiency in both.

Grading should be progressive. During each flight, the pilot in training should achieve a new level of learning (e.g. flight one, the automation management area, might be a “describe” item by flight three a “practice” item, and by flight five a “manage-decide” item.

Immediately after landing, and before beginning the critique, Flight Instructor Linda asks her pilot in training Brian to grade his performance for the day. Being asked to grade himself is a new experience but he goes along with it. The flight scenario had been a two-leg IFR scenario to a busy class B airport about 60 miles to the east. Brian had felt he had done well in keeping up with programming the GPS and the MFD until he reached the approach phase. He had attempted to program the ILS for runway 7L and had actually flown part of the approach until ATC asked him to execute a missed approach.

When he went to place a grade in that segment he noticed that the grades were different. Instead of satisfactory or unsatisfactory he found, “Describe, Explain, Practice, and Perform”. He decided he was at the Perform level since he had not made any mistakes.

When Linda returned Brian discovered that she had graded his flight as well, with a similar grade sheet. Most of their grades appeared to match until the item labeled “programming the approach”. Here, where he had placed a “Perform” Linda had placed a “Explain”. This immediately sparked a discussion. As it turned out, Brian had selected the correct approach, but he had not activated it. Before Linda could intervene, traffic dictated a go around. Her explain grade told Brian that he did not really understand how the GPS worked and he agreed. Now, learning could occur.

In the table that follows, the desired outcome table denotes a pilot in training near the beginning of training and the grades reflect proficiency of the pilot in training to an expected level of performance in each of these areas. These grades are not self-esteem related since they do not describe a recognized level of prestige (such as A+ or “Outstanding”), rather a level of performance. You can’t flunk a lesson. However, you can fail to demonstrate the required flight and SRM skills. By reflecting on the lesson and grading their own performance, the pilot in training becomes actively involved in the critique process. Pilot in training participation in the process also reduces the self-esteem issue. But most importantly, this establishes the habit of healthy reflection and self-criticism that marks most competent pilots.

Learner Centered Scenario Grading-Desired Outcome Table