How Usability Testing Resulted in Improvements to Ground Collision Software for General Aviation: Improved Ground Collision Avoidance System (IGCAS)

Document

HOW USABILITY TESTING RESULTED

IN IMPROVEMENTS TO GROUND

COLLISION SOFTWARE FOR

GENERAL AVIATION

IMPROVED GROUND COLLISION AVOIDANCE SYSTEM (IGCAS) MICHAEL LAMARR CHRIS CHINSKE, ETHAN WILLIAMS, CAMERON LAW, MARK SKOOG, PAUL SOROKOWSKI

OUTLINE

• Background

• System Description

• Design Philosophy

• Evolution of Displays

• Methods of Testing

• Results and Recommendations

• Future Research

WHY DO WE NEED THE IMPROVED GROUND

COLLISION AVOIDANCE SYSTEM?

• Controlled flight into terrain (CFIT) remains a leading cause of fatalities in aviation • Night • Weather • Spatial Disorientation and loss of situation Awareness • Enhanced ground proximity warning and terrain awareness and warning systems have substantially reduced CFIT for large commercial air carriers • The problem still remains for fighter aircraft, helicopters, and general aviation resulting in roughly 100 deaths each year in the United States alone

WHERE ARE WE GOING WITH THIS?

• Improved ground collision avoidance system (iGCAS) for General Aviation (GA) • Reliable collision avoidance for all aircraft • Manual and eventually automatic versions • Tailorable to user’s price point • Walk - on tablet/phone warning system • Downloadable app • Tablet/Phone warning system with wireless sensor inputs Current Work • Downloadable app coupled to avionics with wireless interface • Glass cockpit warning system • S/W available to commercial avionics manufactures Note: • Full Auto GCAS iGCAS was derived from the F - 16 • S/W running on onboard avionics Auto GCAS which began USAF • Integrated with commercial digital autopilot fielding in 2014

IGCAS PROGRAM FLOW

• Runs in background of existing navigation app • Setup iGCAS for specific aircraft • Switches to Viable Maneuver Display (VMD) (Caution State) near terrain • Switches to Avoidance Director Display (ADD) (Warning State) when impact is imminent • Switches back to VMD once imminent impact is resolved • Switches back to navigation app once clear of terrain App Start Up Nav ADD ADD VMD Nav VMD

VIABLE MANEUVER DISPLAY

Note: Appears as the aircraft approaches terrain (about 30 seconds from impact)

AVOIDANCE DIRECTOR DISPLAY

Note: Appears when an avoidance maneuver must be executed to avoid a terrain collision

VIDEO PLACEHOLDER

DESIGN PHILOSOPHY

• Design phone first philosophy • Displays need to be as simple and easy to understand as possible • Must not issue nuisance warnings • Warn pilot when a recovery MUST be performed • Can’t Expect pilot to make a maneuver if warned at last second • Provide cautionary alert (less urgent) to get attention prior to warning • Must be reliable and usable by all types of pilots • It was not expected that a pilots would be highly trained on iGCAS • It was not expected (or hoped for) that iGCAS would be a tool used that often

EVOLUTION OF DISPLAYS

EVOLUTION OF THE VIABLE MANEUVER DISPLAY

Viable Maneuver Display

METHODS OF TESTING

OBJECTIVES

• Assess the overall appropriateness and acceptability of iGCAS as a warning system for General Aviation aircraft • Usability of the iGCAS displays • Effectiveness of audio cues • Test terrain avoidance • Comparison of pilot - flown trajectory to planned trajectory • Terrain miss distance • Pilot response time

PARTICIPANTS

• 24 general aviation pilots attending the Experimental Aircraft

Association (EAA) airshow participated in the usability study

• Ages ranged from 18 - 70 (Mean 44 years old) • 35 to 10,000 (Median 1475 hours) flight hours

• Each session typically lasted 1.5 hours

SIMULATOR SETUP AND THE TEAM

Team of multi - discipline engineers Flight simulator running X - Plane 10 and the iGCAS app on a smart phone

TEST PROCEDURE

• iGCAS brief (3 - 5 minutes)

• Simulator familiarization (5 - 8 minutes)

• Brief think aloud protocol and flight cards (2 - 3 minutes)

• 8 scenarios flown (35 - 45 minutes)

• After each scenario the pilot elaborated on the session providing feedback on the VMD, ADD, and audio cues

• Debrief (10 - 30 minutes)

• post - test questionnaire • interview

EXAMPLE SCENARIO WITH ADD MANEUVER

• Maintain until ADD maneuver: • 9,500’ MSL • 10˚ Mag Heading • 140 KIAS • Please think aloud throughout scenario while monitoring flight conditions and iGCAS Displays • Follow Avoidance Director Display guidance • Once maneuver is complete fly, wings level for 10 seconds

EXAMPLE SCENARIO WITH A VMD MANEUVER

• Note: Cloud layer at 11,000’ MSL • Maintain until one VMD maneuver is available and follow without activating ADD: • 10,000’ MSL • 215˚ Mag Heading • 125 KIAS • Please think aloud throughout scenario while monitoring flight conditions and iGCAS Displays • Follow VMD guidance once one maneuver is available

RESULTS AND RECOMMENDATIONS

VMD AND ADD USABILITY SCORES WITH

STANDARD DEVIATION

1 is Very Unsatisfactory to 6 is Very Satisfactory

VMD PROMINENT ISSUES

• VMD does not provide sufficient feedback when maneuver options are no longer viable • Arrows and the BABS can change too quickly • After ADD transitions to VMD there is a lack of directive guidance which could lead to a secondary activation

ADD PROMINENT ISSUES

• ADD ended slightly earlier than expected with a chance of a second actuation • After avoidance maneuver the “wings level” audio cue is incorrectly interpreted as straight and level flight • No feedback that avoidance maneuver is being conducted correctly • ADD did not provide as much reaction time as expected • Bank information not apparent on ADD

SUMMARY AND CONCLUSIONS

• At no time did the pilots consider the system to provide nuisance warnings or impede normal flight operations • Three styles of using iGCAS were observed • Mostly audio only with head up flying • Audio with head up and down as a cross reference for decision making • Mostly head down • Pilots were able to avoid terrain using iGCAS with about 3 - 5 minutes of training • Pilots recommended a training session ranging from 10 to 60 minutes • Pilots did not want to add any additional features and considered their favorite par of iGCAS to be its simplicity

SUMMARY AND CONCLUSIONS

• iGCAS was tested in a way to generate the maximum feedback on the displays, audio cues, maneuvers, and performance data.

• In doing so, pilots knew that they would be avoiding terrain and when to expect the avoidance • The system as tested has known limitations and more testing is needed to characterize and identify all existing limitations.

1. System should be continued to be matured towards transition to public availability 2. Considerable care should be taken to not adversely affect the clarity and simplicity of the system if any changes are made .

Overview

Document