Aircraft checklists are the foundation of pilot standardization and flight deck safety. Following a series of air carrier accidents in the late 1980s, safety professionals began to recognize and have concerns that the improper use, non-use, or poor design of checklists could potentially contribute to an aircraft accident. Until this point, the design of the checklist and the philosophy of its use escaped input or analysis from human factors professionals.
The NTSB first recognized the importance of checklist use and its critical role in flight safety in 1969 following the no-flap takeoff crash of a Pan American Airways Boeing 707-321C from Elmendorf Air Force Base in Alaska. The crash of Pan Am’s Clipper Racer—an all-cargo flight destined for South Vietnam—killed all three crewmembers.
NTSB discovered that “Flaps” appeared on Pan Am’s Before Taxi Checklist but was not included on the Before Takeoff Checklist. It was noted that the first officer lowered the flaps prior to taxi, but as the aircraft was preparing to be de-iced, the captain retracted the flaps. Afterward, the first officer remarked, “Okay, let’s not forget those.”
Among the probable causes of the crash, according to the Safety Board, were a defective checklist, the 707’s defective takeoff warning system, and stress caused by a rushed schedule. In its recommendation, the NTSB called for: “Air carrier cockpit checklist to be reviewed in an effort to ensure that each list provides a means of reminding the crew, immediately prior to takeoff, that all items critical for safe flight have been accomplished.”
Unfortunately, it took 18 years and three air carrier accidents during a 15-month period for the NTSB and industry to recognize the problems that relate to the human factors aspects of checklist design and procedures.
In the first accident, in May 1987, an Air New Orleans British Aerospace J-3101 (Jetstream) twin-turboprop crashed after takeoff in New Orleans. During the initial climb, the flight crew felt a severe yawing motion and engine torque fluctuations. The captain attempted to land and overran the runway, crossing a highway striking several vehicles.
The NTSB concluded that “the engine rpm levers were either advanced to a position less than full forward or they were not advanced at all before takeoff, indicating a lack of checklist discipline on the part of the aircrew.”
A contributing factor, according to the report, stated that “The typeface of the Air New Orleans checklist was 57 percent smaller than the recommended human engineering criteria. This smaller typeface reduced the legibility of the print even under optimum conditions.” Furthermore, the NTSB would formally recommend that the FAA issue an advisory circular to commercial operators recommending the use of a procedural checklist that incorporates human engineering design criteria for size and style of print.
The NTSB would also suggest that frequent change (change fatigue) in the checklist can have a negative effect on the checklist’s importance to the flight crew. Air New Orleans had recently introduced the Jetstream into its fleet, the report noted. “Frequent revisions of checklists for newly-acquired aircraft are understandable, but the fact that this normal checklist had been changed seven times between January and May 1987 [only five months] suggests to the Safety Board that its original design (the manufacturer) and approval (FAA) may have been inadequate and may have caused confusion among flight crews.”
In the second accident, only three months after the Air New Orleans crash, a Northwest McDonnell Douglas MD-82 crashed shortly after takeoff from Detroit Metro Airport following a no-flap/no-slat takeoff. The six crew members and 148 of the 149 passengers were killed in the crash. Two individuals were killed on the ground. The NTSB concluded that the probable cause of the accident was the flight crew’s failure to use the taxi checklist to ensure that the flaps and slats were extended for takeoff.
One year later, a Delta Air Lines Boeing 727-200 crew attempted a no-flap/no-slat takeoff and crashed 22 seconds after liftoff at Dallas Fort Worth International Airport. Of the 108 passengers and crew onboard the aircraft, 17 were killed and 76 were injured. The NTSB concluded that “the flight crew did not extend the airplane’s flaps or slats for takeoff.”
Human perception errors were likely in this accident considering the cockpit voice recorder captured the second officer reading the checklist, stating “flaps,” and the first officer responding, “fifteen, fifteen, green light”—a normal response. Presumably, the first officer would visually reference the inboard and outboard flap/slat position indicators (“fifteen, fifteen”) and the illumination of the leading-edge flaps and slats indicator (“green lights”). In this case, however, the position indicators would show “zero” and “no lights” without the flaps or slats extended. Active monitoring requires the pilot to “look for something” rather than “look at something.”
NTSB Wants Research
During the NTSB hearings into the Northwest Airlines MD-82 crash, the late Earl Wiener—a human factors specialist and NASA scientist—testified at the time, “that he did not know of any human factors research on how a checklist should be designed.” Further research by NASA concluded that in 1989, there was essentially no human factors research available that pertained to aircraft checklists—anywhere, not in the U.S., Western Europe, or elsewhere.
Following the Air New Orleans Jetstream and Northwest Airlines MD-82 accidents, the NTSB recommended that the FAA convene a human-performance research group to determine “…if there is any type of method of presenting checklist which produces better performance on part of user personnel.” Additionally, the Safety Board recommended that the FAA specify checklist typography criteria for commercial operators.
As a result, NASA began its journey to investigate the many human factors elements related to aircraft checklist design and procedures. Researchers found problems related not only to the physical design of the checklist but also to the social issues that led pilots to misuse it or not use it at all.
The output of this research included two research documents: NASA’s Human Factors of Flight-Deck Checklist: The Normal Checklist (Degani and Wiener, May 1990) and NASA’s On the Typography of Flight-Deck Documentation (Degani, December 1992). Afterward, the UK CAA produced a document (now in its third edition—2006), CAP 676: Guidance on the Design, Presentation, and Use of Emergency and Abnormal Checklist.
NASA defines the major function of the checklist as to ensure that the crew will properly configure the aircraft for flight and maintain this level of quality throughout the flight and in every flight.
Generally, a checklist serves as a “job aid” for very complex operations: it aids in recall, verifies aircraft configuration (even when pilots are physically or psychologically “degraded”), provides a proper sequence of actions, provides a sequential framework for operational requirements, allows cross-checking, keeps all crewmembers “in the loop,” facilitates optimal crew coordination and workload, and is a quality management tool.
An overlooked objective in checklist design is its ability to promote a positive attitude towards the use of this document or procedure. For the front-line operator—pilots in this case—a checklist must be “well-grounded” in the current operational environment with a sound realization of its importance and not regarded as a nuisance. By design, it should be an effective interface between the human and the machine.
There are two methods or philosophies in conducting or executing a checklist—the “challenge and response” or the “do-list.” Each method has its merit.
Challenge and response is more accurately termed “challenge-verification-response.” Using these methods, pilots configure the aircraft for the appropriate flight phase using a “flow” pattern. A flow pattern is an ingrained sequence of actions performed from memory. After completion of the flow, pilots then use the checklist to verify that the critical items have been correctly configured; one pilot reads the “challenge” portion of the checklist—both verify—and the other pilot provides the appropriate response.
Generally, most modern aircraft utilize a “dark, quiet concept” (configuration redundancy) where if the aircraft is configured properly, the overhead panel (or systems panels) will not have any lights illuminated. In this case, all items in the flow pattern may not be included in the checklist—a simple “overhead panel” challenge and a “set” response may be appropriate if each crewmember verifies that no lights are illuminated.
The challenge and response method is very efficient; each crewmember can accomplish their assigned flow patterns and then when everyone is “caught up,” the crew can collectively complete the checklist.
Step-by-Step
A “do-list” is more accurately termed a “call-do-response.” Using this method, the checklist is used to lead the pilots to a step-by-step (like a cookbook) procedure where one pilot directs the other pilot to configure a cockpit control. In theory, every flight deck control (switch, lever, or otherwise) would be listed on the checklist and both crewmembers would have to be present to accomplish the checklist.
The advantages of the do-list are that both crewmembers can verify the appropriate switch activation. Disadvantages of the do-list are that they are very detailed, and the checklists are time-consuming. As intended, each pilot must be in the seat for the entire process.
According to NASA, philosophies toward checklist design may vary based on the type of operation. A short-haul operator—greater than three segments per day—may choose to avoid a long meticulous checklist. Highly repetitive checklists may lead pilots to develop “workarounds” that only include those critical items or skip the checklist altogether. For the long-haul operator—one to two flights per day— pilots may be less resistant to a more detailed checklist.
UK CAA’s CAP 676 incorporates a checklist audit tool (CHAT) to determine if a checklist complies with the best human factors practices as defined within the document. The CHAT is most effectively used as a gap analysis tool to identify areas where a checklist can be improved. Using this resource, an operator can grade a checklist in areas such as its physical characteristics, content, layout, and format.
Physical characteristics look at items such as the size of the actual document, contrast and color, and other details such as typography. One of the most common errors in checklist design is typography. There are two main factors that need to be considered, legibility of print and readability, to account for less-than-optimal reading conditions on the flight deck.
Legibility of print or discriminability involves the proper selection of alphanumeric characters to enable the reader to identify them quickly and positively from other letters or characters. Readability relates to the quality of the word or text to allow rapid recognition of a single word, word groups, abbreviations, or symbols. These factors, according to the research, are crucial for flight deck documents such as checklists.
Today there are thousands of typefaces or fonts available. Two major groups of fonts are applicable to flight deck documents: Roman and sans-serif. Roman is the style of font typically used in newspapers. Sans-serif includes contemporary fonts (Calibri or Arial, for example) that do not include the little strokes (serifs) that project horizontally from the top or bottom of a main stroke. Research has shown that sans-serif is more legible than Roman. The absence of serifs presents a more simple and clean typeface. Arial or Helvetica are preferred.
Upper or Lower
Many checklists are published using ALL UPPER-CASE letters—this is usually an attempt to add emphasis. The consensus of researchers is that lowercase is preferred and recommended over uppercase. Lowercase is more legible because the pattern of a word is stored in the human memory (think of “sight words” that are taught to children). Lowercase words have ascenders (the vertical stroke of a “b” or “d’) and descenders (“p” or “q”) that contribute to the unique pattern of a word. Lowercase words appear “characteristic” whereas upper case words appear like a “RECTANGULAR BOX.” The use of UPPERCASE words for checklist titles is allowable, but the up and down style is preferred (See image on previous page).
The type size is important for readability, especially for older pilots. At about age 50, there is a 50 percent reduction in retinal illumination as compared to a 20-year-old. In general, a font size between 14 and 20 is recommended. CAP 676 and FAA 8900.1 recommend headings to be printed at type size 14 (minimum 12) and the normal body of a checklist at type size 12 (minimum 10). Any font size less than 10 is not recommended. Smoke-related checklists should use a larger font for improved readability.
For contrast, black text on a white background is preferred (FAA and CAA) and black text on a yellow background is acceptable (CAA).
Designing a checklist should be easy, right? Thirty years ago, researchers often encountered the following statement, “Checklists are simple and straightforward, so what is there to study about them?” For air carriers (121 operators in the U.S.) the design of a checklist that incorporates human factors considerations was not a priority until the early 2000s.
In business aviation, only recently have manufacturers embraced serious human factors research to revamp the design and philosophies of checklists and quick reference handbooks to create useful cockpit tools for the end user. Bombardier was one of the first to embrace these concepts with the introduction of the Global 7500. The idea was to make a checklist useful for the everyday line pilot as opposed to a flight test pilot. z