Aircraft Loss-of-Control: Analysis and Requirements for Future Safety-Critical Systems and Their Validation

Aircraft Loss-of-Control: Analysis and Requirements for

Future Safety-Critical Systems and their Validation

Christine M. Belcastro NASA Langley Research Center Dynamic Systems and Control Hampton, Virginia christine.m.belcastro@nasa.gov Abstract —Loss of control remains one of the largest contributors 42 hull loss accidents and 3241 fatalities. The analysis of this to fatal aircraft accidents worldwide. Aircraft loss-of-control reference groups the accidents into the categories aerodynamic accidents are complex, resulting from numerous causal and stall, flight control system, spatial disorientation of the crew, contributing factors acting alone or more often in combination.

contaminated airfoil, and atmospheric disturbance. There is Hence, there is no single intervention strategy to prevent these also a detailed discussion of accidents in each of these accidents. This paper summarizes recent analysis results in identifying worst-case combinations of loss-of-control accident categories and a comparison with older accidents that occurred precursors and their time sequences, a holistic approach to prior to 1993 in order to identify emerging trends. This preventing loss-of-control accidents in the future, and key reference also provides a definition of aircraft upset requirements for validating the associated technologies.

conditions, which is defined therein as “any uncommanded or inadvertent event with an abnormal aircraft attitude, rate of Keywords-Aircraft loss of control; Loss-of-control accident change of aircraft attitude, acceleration, airspeed, or flight analysis; Integrated systems solution to aircraft loss of control; Validation of safety-critical systems operating under off-nominal trajectory”. Due to the complexity of LOC accidents, no conditions single intervention strategy can be identified to prevent them.

This paper summarizes key recent results presented in I. I NTRODUCTION References [5], [6], and [7] to address aircraft loss of control.

Ref. [5] presents a detailed analysis of LOC accidents in Aircraft loss of control (LOC) is one of the largest which worst case combinations of causal and contributing contributors to fatal accidents across all vehicle classes and factors are identified as well as how they sequence in time.

operational categories [1], [2], and [3]. The 2010 Boeing Future potential risks are also identified. Ref. [6] presents a report of Ref. [1] summarizes commercial jet airplane future integrated systems concept for preventing aircraft LOC accidents that occurred worldwide between 1959 and 2009 accidents, and Ref. [7] presents requirements and a process for involving aircraft that are heavier than 60,000 pounds their validation and verification (V&V), with an emphasis on maximum gross weight. In this report, aircraft loss-of-control validation. Key results from these references are summarized is the leading fatal accident category, with 20 accidents in Sections II, III, and IV, respectively. Section V provides a occurring in this time period that resulted in 1,848 fatalities.

summary and some concluding remarks.

The 2008 report of Ref. [2] on worldwide fatal accidents by the Civil Aviation Authority (CAA) Safety Regulation Group II. A IRCRAFT LOC A CCIDENT A NALYSIS in the United Kingdom (UK) determined LOC to be the A review of 126 LOC accidents (predominantly from Part second-leading consequence (38.9%, 110 of 283 accidents) 121, including large transports and smaller regional carriers) resulting from numerous causal and contributing factors – occurring between 1979 and 2009 (30 years) that resulted in second only to post-crash fire. The report by Alliant 6087 fatalities was performed for the analysis, and a listing of Techsystems of Ref. [3] looked across all U.S. operational these accidents is provided in the Appendix of Ref. [5]. This categories (Part 121, Scheduled Part 135, Non-Scheduled Part accident set does not represent an exhaustive search 135 and Part 91) between 1988 and 2004 and found that throughout this time period, and it does not include military, aircraft loss of control accidents “were responsible for more private, cargo, charter, and corporate accidents. Russian than half of the aviation fatalities during that time period” – aircraft accidents were also excluded due to a general lack of despite having contributed to less than 20% of the U.S.

detailed information in the associated reports. Of this total aviation accidents in the data set.

accident set, 91 accidents resulting in 4190 fatalities occurred Aircraft LOC is also complex, resulting from numerous between 1994 and 2009 (15 years). The review was based on causal and contributing factors acting alone or more often in accident reports available on the Aviation Safety Network [8] combination. In Reference [ 4 ], 74 LOC accidents were and National Transportation Safety Board (NTSB) [ 9 ] reviewed for the time period 1993 – 2007, which resulted in websites. The level of detail in analyzing each accident was The following subsections A and B address combinations and therefore dependent on the level of detail provided in the sequencing of LOC causal and contributing factors, accident reports. Information from each report was respectively. Subsection C addresses future risks.

transcribed into a categorized set of causal and contributing factors, using the following scheme. The causal and Table 1. Contributions to LOC Accidents and Fatalities by Individual Causal and Contributing Factors [5] contributing factors were grouped into three categories: Factor Accidents % Fatalities % adverse onboard conditions, vehicle upsets, and external Adverse Onboard Conditions 119 94.4 5683 93.4 hazards and disturbances.

Vehicle Impairment 33 26.2 1134 18.6 Adverse onboard conditions included: System Faults / Failures / Errors 57 45.2 2807 46.1  vehicle impairment (including inappropriate vehicle Vehicle Damage 23 18.2 1780 29.2 configuration, contaminated airfoil due to icing, and improper Inappropriate Crew Response 54 42.8 2818 46.3 vehicle loading); Vehicle Upsets 98 77.8 4523 74.3  system faults, failures, and errors (resulting from design flaws, Abnormal Attitude 18 14.3 219 3.60 software errors, or improper maintenance actions); Abnormal Airspeed / Angular Rates / Asymmetric Forces 14 11.1 701 11.5  vehicle damage to airframe and engines (resulting from fatigue Abnormal Flight Trajectory 4 3.2 272 4.47 Uncontrolled Descent 15 11.9 773 12.7 cracks, foreign objects, overstress during upsets or upset Stall / Departure 49 38.9 2622 43.1 recovery, etc.); and External Hazards & Disturbances 61 48.4 3246 53.3  inappropriate crew response (including pilot-induced Poor Visibility 9 7.1 556 9.1 oscillations, spatial disorientation, mode confusion, ineffective Wake Vortices 4 3.2 402 6.6 recoveries, crew impairment, and failures to take appropriate Wind Shear / Gusts / Thunderstorms 18 14.3 1126 18.5 actions).

Snow / Icing 28 22.2 595 9.8 Abrupt Maneuver / Collision 3 2.4 189 3.1 External hazards and disturbances included:  poor visibility;  wake vortices; A. Worst Case Analysis  wind shear, turbulence, and thunderstorms; In order to identify worst case combinations of LOC causal  snow and icing conditions; and and contributing factors (as defined by number of accidents  abrupt maneuvers for obstacle avoidance or collisions.

and resulting fatalities), 3-dimensional scatter plots were Vehicle upsets included: generated. Figure 1 shows the key result from this analysis.

 abnormal attitude; 9Accidents/370Fatalities  abnormal airspeed, angular rates, or asymmetric forces; (5Accidentswith219FatalitiesAlso 14Accidents/778Fatalities InvolvedInappropriateCrew (10Accidentswith620Fatalities  abnormal flight trajectory; Fatalities SphereSizeis Response) AlsoInvolvedFailures, 0 DirectlyProportionalto Damage,orImpairment) 17Accidents/326Fatalities  uncontrolled descent (including spiral dive); and 1– 99 NumberofAccidents (4Accidentswith55Fatalities 100– 199 200– 299 AlsoInvolvedInappropriate  stall/departure from controlled flight.

300 More CrewResponse) Vehicle Upset Conditions A basic analysis of the contributions of each Stall / Departure Uncontrolled Descent causal/contributing factor to the 126 accidents is given in Abnormal Trajectory Ab. Va / Rates / Asym Table 1. It should be noted in Table 1 that the factors are not Abnormal Attitude None / Unknown mutually exclusive. For example, 119 LOC accidents involved one or more adverse onboard conditions, and the 3Accidents/204Fatalities External (2Accidentswith196 frequency of each individual factor within this category is Hazards / FatalitiesAlsoInvolved Failures,Damage,or Disturbances Impairment) listed. These numbers do not add up to 119, however, because 15Accidents/522Fatalities Adverse Onboard Conditions (1Accidentwith47FatalitiesAlsoInvolved there were many accidents involving more than one subfactor.

InappropriateCrewResponse) Similarly, adding the number of accidents listed for the three categories exceeds the 126 total because many accidents Figure 1. Identification of Overlap in LOC Causal and involved multiple categories. The 23 accidents related to Contributing Factor Combinations, 1979 – 2009 [5].

vehicle damage consisted of 20 airframe and system damage conditions, and 3 engine damage conditions. Table 1 is useful The three dimensions are aligned with the three categories for determining the number of accidents and fatalities identified in Table 1. Sphere size is directly proportional to associated with individual causal and contributing factors, but the number of accidents, and sphere color depicts the number it does not provide any information on combinations or of fatalities as indicated by the legend. As indicated in Figure sequencing of these factors. Nonetheless, this table identifies 1, worst case combinations include: system faults and failures System Faults/Failures/Errors, Vehicle Impairment/Damage, occurring alone and in combination with upsets, icing Inappropriate Crew Response, Stall/Departure, Atmospheric conditions resulting in vehicle impairment, and inappropriate Disturbances related to Wind Shear/Gusts, and Snow/Icing as crew response combined with upset conditions. There are also the most significant contributors to the number of fatalities. a significant number of accidents and fatalities resulting from: vehicle damage occurring alone and combined with upsets, rather an outcome of an external hazard or adverse onboard icing combined with inappropriate crew response and upsets, condition. Within adverse onboard conditions, system faults, and wind shear and turbulence combined with inappropriate failures, and errors are the leading initial factor, and crew response and vehicle upsets. As noted in Figure 1 with inappropriate crew response is the second most likely initial red text, there is some overlap (i.e., some combinations that event. Relative to external hazards and disturbances, the are not mutually exclusive) in the scatter plot, especially leading initial factor is icing, followed by wind shear, gusts, within the adverse onboard conditions dimension. This and thunderstorms. Adverse onboard conditions are also the overlap is due to a significant number of accidents that most likely factor to occur second in the chain of events involved multiple adverse onboard conditions. For example, leading to aircraft LOC, with vehicle impairment being the some of the accidents shown for system faults and failures also most likely secondary factor to occur. This is due to vehicle involved inappropriate crew response. Alternatively, many of impairment resulting from icing conditions (i.e., contaminated the accidents shown for inappropriate crew response also airfoil or reduced engine performance), faults or damage.

involved other adverse onboard conditions, such as vehicle Vehicle upsets most often occur as the second, third, or fourth impairment, failure, or damage. While there is some overlap factor in the LOC sequence. Only one 5-factor sequence was in the external hazards and disturbances and the vehicle upset identified in this data set.

dimensions, it is generally much smaller that the onboard An analysis was also performed of each LOC sequence.

dimension. This analysis is summarized in Table 3.

Ref. [5] also analyzed the most recent 15 years of accident data in the set, as well as those involving no fatalities, in order Table 3. Summary of LOC Accident Sequences [5] to identify any emerging issues. No significant emergent Initial Factor in LOC Sequence Accidents % Fatalities % trends were evident.

Adverse Onboard Conditions 69 54.8 3733 61.3 B. Time Sequence Analysis Vehicle Impairment 3 2.4 186 3.1 System Faults / Failures / Errors 42 33.3 1544 29.0 An analysis of the time sequencing of the LOC causal and Vehicle Damage 6 4.8 908 14.9 contributing factors was performed for the 30-year data set.

Inappropriate Crew Response 18 14.3 1095 14.3 Table 2 provides a summary of this sequencing.

External Hazards & Disturbances 54 42.8 2228 36.6 Poor Visibility 7 5.5 438 7.2 Table 2. Sequencing of LOC Causal & Contributing Factors [5] Wake Vortices 3 2.4 137 2.2 Wind Shear / Gusts / Thunderstorms 14 11.1 874 14.4 Factor 1st 2nd 3rd 4th 5th Snow / Icing 27 21.4 590 9.7 Adverse Onboard Conditions 69 69 24 6 0 Abrupt Maneuver / Collision 3 2.4 189 3.1 Vehicle Impairment 3 29 3 0 Vehicle Upsets 3 2.4 126 2.1 System Faults / Failures / Errors 42 11 4 0 Abnormal Attitude 0 0 0 0 Vehicle Damage 6 7 5 5 0 Abnormal Airspeed / Angular Rates / 0 0 0 0 Inappropriate Crew Response 18 22 12 1 0 Asymmetric Forces External Hazards & Disturbances 54 6 0 0 0 Abnormal Flight Trajectory 1 0.8 117 1.9 Uncontrolled Descent 0 0 0 0 Poor Visibility 7 0 0 0 0 Stall / Departure 2 1.6 9 0.2 Wake Vortices 3 1 0 0 0 Totals 126 100 6087 100 Wind Shear / Gusts / Thunderstorms 14 3 0 0 Snow / Icing 27 1 0 0 Abrupt Maneuver / Collision 3 1 0 0 Table 3 provides the number of accidents and fatalities (and Vehicle Upsets 3 36 47 15 1 associated percentages) relative to each causal and Abnormal Attitude 0 12 3 3 0 Abnormal Airspeed / Angular Rates / 0 3 7 4 contributing factor as the initial factor in the LOC sequence.

Asymmetric Forces Defining the LOC sequences in terms of the initiating factor Abnormal Flight Trajectory 1 1 3 1 0 allowed a comprehensive assessment without overlap. As Uncontrolled Descent 0 5 7 2 1 Stall / Departure 2 15 27 5 0 indicated in Table 3, LOC events initiated by adverse onboard conditions comprised 54.8% of the accidents and 61.3% of the fatalities within the data set considered in this analysis. Of It should be noted that these sequences were identified without these, system failures, faults, and errors initiated 33.3% of overlap. That is, there is no “double bookkeeping” of accidents and 29% of fatalities, followed by inappropriate sequences in Table 2. Thus, the total number of initiating crew response, vehicle damage, and vehicle impairment.

factors under column 1 sums to the total number of LOC External hazards and disturbances initiated 42.8% of the accidents, since all LOC accidents result from at least 1 causal accidents and 36.6% of the fatalities in the LOC accidents or contributing factor. Table 2 indicates that LOC events are considered. Within this category, icing represented 21.4% of usually first precipitated by an adverse onboard condition or accidents and 9.7% of fatalities, whereas wind shear, an external hazard or disturbance. Moreover, external hazards turbulence, and thunderstorms initiated 11.1% of accidents and disturbances rarely occur further downstream in LOC and 14.4% of fatalities. These factors were followed in sequences. Vehicle upsets are rarely the initial factor but frequency of occurrence by poor visibility, wake vortices, and Dashed boxes in Figure 2 represent factors that occurred in abrupt maneuver or collision (with the last two having the some subset within the sequence. These 7 generalized same frequency of occurrence). It is interesting to note that sequences represent 112 accidents (88.9%) and 5529 fatalities icing initiated more accidents, but wind-related disturbances (90.8%).

resulted in more fatalities. This is because the predominance C. Future Potential Risks of icing-induced accidents in the data set of this study In addition to looking at historical accident data, potential involved smaller aircraft, whereas the preponderance of wind- future LOC accident risks should be identified relative to induced accidents in this data set involved large transports. As known (as well as new) precursors. This is more difficult, indicated previously, vehicle upsets are rarely the precipitating because (without data) it becomes more speculative.

factor in the LOC sequence, with these comprising 2.4% of the However, the identification of potential areas of vulnerability accidents and 2.1% of the fatalities considered in this study.

might enable the development of a comprehensive Within this category, stall/departure initiated 1.6% of the intervention strategy that anticipates and mitigates these future accidents and 0.2% of fatalities, and abnormal flight trajectory potential risks. One area of consideration is airspace operation initiated 0.8% of accidents and 1.9% of fatalities in the data under the Next Generation (NextGen) Air Transportation set. While upsets are not usually the precipitating factor, System [10]. The NextGen concept of operations provides an many LOC sequences include vehicle upset somewhere in the integrated view of airspace operations in the 2025 timeframe chain of events (as indicated in Table 2).

and includes high-density, all-weather, and self-separation Ref. [5] identified 52 unique LOC sequences from the operational concepts. There is also expected to be mixed- accident data. In order to condense these sequences into capability aircraft operating within the same airspace, smaller, more actionable groupings, they were also combined including piloted aircraft and unmanned aircraft systems.

and generalized. The generalized sequences from Ref. [5] are High-precision 4-D trajectories are envisioned that will enable shown in Figure 2 along with the associated number of safely flying with closer spacing to inclement weather, terrain, accidents and fatalities.

and other aircraft, and these trajectories can be altered if A.43Accidents,1855Fatalities:(I,III) necessary during the flight. Other areas of consideration LOC Normal Event include increasing airspace and vehicle system complexity Flight Vehicle VehicleProblem/ Upset ExternalHazard without developing comprehensive methods for their • AbnormalAttitudes • VehicleImpairment/Fault/Failure/Damage validation and verification (V&V), and increased automation • AbnormalTrajectory • ExternalHazardorDisturbance • Stall/Departure without improved crew interfaces.

B.20Accidents,907Fatalities:(V,VIII,IX) In an effort to identify areas of potential future LOC risk in Normal LOC Flight Event terms of known precursors, Figure 3 illustrates several areas of VehicleProblem/ Inappropriate Vehicle ExternalHazard CrewResponse Upset possible increase in causal and contributing factors with the • VehicleImpairment/Fault/Failure/Damage • PoorSituationalAwareness/Distraction • AbnormalAttitudes potential for increased LOC accidents or incidents.

• ExternalHazardorDisturbance • SpatialDisorientation(PoorVisibility) • AbnormalTrajectory • ModeConfusion(SystemComplexity) • Stall/Departure IncreasedSystem HighDensity C.17Accidents,1095Fatalities:(II) ComplexityWithout MixedVehicle IncreasedAutomation WithoutImprovedCrew ComprehensiveV&V Operations Normal LOC Interfaces Flight Event Process Vehicle Inappropriate VehicleProblem/ AllWeather Upset CrewResponse ExternalHazard Operations Vehicle Upset Conditions Stall / Departure Normal LOC Uncontrolled Descent D.16Accidents,484Fatalities:(IV,3,32) Flight Event Abnormal Trajectory VehicleProblem/ Ab. Va / Rates / Asym ExternalHazard Abnormal Attitude None / Unknown • VehicleImpairment/Fault/Failure • ExternalHazardorDisturbance E.8Accidents,569Fatalities:(VI) External Normal LOC VehicleUpset/ Flight Hazards / Event External VehicleDamage/ Disturbances Hazard InappropriateCrew Response Adverse Onboard Conditions F.7Accidents,569Fatalities:(VII,X) HighDensity Operations Normal LOC Flight VehicleUpset Event VehicleProblem/ Inappropriate Failure/ ExternalHazard CrewResponse Figure 3. Potential Areas of Future Increased LOC Risk [5].

Damage • VehicleImpairment/Fault/Failure • ExternalHazardorDisturbance If all-weather operations and highly precise trajectories that G.1Accident,50Fatalities:(42) enable closer spacing to inclement weather increase the Normal LOC Flight Event VehicleProblem/ Vehicle Inappropriate probability of an aircraft actually encountering a weather ExternalHazard Upset CrewResponse hazard during flight, this could result in a larger number of • Stall/Departure • VehicleImpairment/Fault/Failure • InabilitytoRecover weather-related LOC accidents (particularly in the terminal • ExternalHazardorDisturbance area). If airspace and vehicle system complexity is increased Figure 2. Generalized LOC Accident Sequences [5]. without comprehensive methods for their V&V, this could lead to a larger number of LOC events initiated by system detection and isolation onboard the aircraft. Mitigation faults, failures, and errors. If high-density mixed-vehicle technologies include failsafe guidance and control systems operations and high-precision tracking that enables closer that ensure stability, maximize vehicle performance and spacing between aircraft increase the probability of aircraft handling qualities, and enable safe maneuvering under LOC encountering other aircraft during flight, this could result in a conditions, as well as flight deck interface systems that larger incidence of wake-induced LOC events or ultimately provide improved crew situational awareness, those initiated by vehicle damage resulting from mid-air countermeasures for crew errors, and variable autonomy to collisions. Increased automation without improved crew optimize the synergy between the crew and automation. The interfaces could result in a higher incidence of LOC events mitigation technologies would also include specific functions precipitated by inappropriate crew actions. New LOC for upset prevention under LOC conditions. Recovery precursors associated with failure modes of future vehicle and technologies include guidance and control algorithms for safe airspace systems must also be identified and considered during and reliable upset recovery. These algorithms would prevent V&V of these systems, and their potential ramifications entry into unrecoverable conditions, and would include vehicle considered under off-nominal operating conditions. New constraints during the recovery (e.g., normal structural loading types of crew-induced LOC precursors must also be constraints as well as vehicle constraints resulting from considered.

impairment or damage). Variable autonomy interface III. F UTURE S YSTEM C ONCEPT functions would be utilized to optimize involvement between the crew and the system. Figure 5 presents a holistic approach Due to the complexity of aircraft LOC events (i.e., accidents for developing the technologies needed to prevent LOC and incidents), no single intervention strategy can be identified accidents using this intervention strategy. Advanced modeling to effectively prevent them. Moreover, there are currently no coordinated or integrated systems or research efforts for and simulation technologies must be developed for addressing aircraft LOC. Current aircraft control systems are characterizing off-nominal condition effects on vehicle primarily designed for operation under nominal conditions, dynamics and control characteristics, including vehicle and often disengage (i.e., return control authority to the pilot) failures and damage, vehicle upset conditions, wind shear and under off-nominal conditions. Current flight deck systems turbulence, wake vortices, icing, and key combinations of provide limited information under off-nominal conditions these (as identified in Reference [3]). This capability can be associated with aircraft LOC. While many current systems utilized for improved crew training under off-nominal have built-in tests for assessing system, subsystem, or conditions, and for the development and validation of component health, these lack the integrated capability for advanced onboard integrated systems technologies.

assessing vehicle health across them, or for the prevention of Databases, models, and real-time modeling methods can also cascading failures across multiple systems. There is also no be utilized onboard the aircraft for characterizing and existing capability to assess vehicle health and external assessing the effects of off-nominal conditions. Enhanced hazards in terms of their impact on flight safety. Improved models, databases, and simulations can also be utilized for crew training and operational procedures for off-nominal improved crew training under LOC conditions.

conditions might enable improved crew response during LOC events, but this is dependent on the capability to effectively Mitigate Avoid/Detect Recover characterize vehicle dynamics and control characteristics under off-nominal conditions. Advanced onboard systems that Normal LOC Vehicle Inappropriate Flight Event Vehicle provide effective detection and resilience under off-nominal Impairment/ Crew Upset conditions could enable improved situational awareness and ExternalHazard Response vehicle response under LOC events, but this requires the • VehicleImpairment/Fault/Failure/Damage • PoorSituationalAwareness/Distraction • AbnormalAttitudes • ExternalHazardorDisturbance • SpatialDisorientation(PoorVisibility) • AbnormalTrajectory effective integration and validation of the associated • ModeConfusion(SystemComplexity) • Stall/Departure technologies.

The analysis of Section II can provide insight into Figure 4. Example Generalized LOC Accident Sequence and preventing LOC accidents. Figure 4 shows an example Intervention Strategy [6].

generalized sequence from Figure 2 with an intervention strategy defined to break the sequence at all stages via Vehicle health management (VHM) technologies must be avoidance, detection, mitigation, and recovery technologies. developed for continually assessing and predicting the health Avoidance technologies include enhanced models and of the airframe, propulsion system, and avionics systems in simulations for characterizing LOC conditions for improved real-time, as well as remaining useful life. In-situ sensing and crew training, advanced vehicle and system design methods estimation methods are needed for distinguishing between that reduce failures and damage, and forward-looking sensors anomalous system behavior and external disturbances. These for avoidance of external hazards and disturbances. Detection technologies provide the capability to prevent catastrophic technologies include vehicle health management technologies failures and damage through the early detection of anomalies, that provide the capability to prevent catastrophic failure as well as the capability to rapidly detect, identify, through early anomaly detection as well as rapid failure characterize, and contain failures and damage when they do occur. (shown in green), vehicle flight safety management and Flight safety assurance (FSA) technologies must be resilient control (shown in blue), and crew-system interfaces developed to provide the capability of continually assessing (shown in yellow).

and predicting the impact of off-nominal conditions on vehicle ExternalHazards AirspaceManagement DistributedRedundantDynamicSystems flight safety, and to provide resilient guidance and control • Avionics(FCS&GNC) • HIRF,Lightning,&IonizingParticles • TrafficMonitoring • Sensing&Actuation • Turbulence,WindShear,Icing • CollisionAvoidance capabilities under off-nominal conditions. These capabilities • AirframeStructure • WakeVortices • Safe/SecureCommunications • Propulsion&ElectricalPower • Terrain,Obstacles,OtherVehicles • Safe/SecureAirspaceOperations can be utilized onboard the aircraft for mitigation of system IntegratedVehicleHealthManagement Vehicle/CrewInterfaceManagement failures and vehicle impairment or damage, external Onboard • InformationFusion Crew/ • VariableAutonomyInterface Environment • Failure/DamageDetection&Isolation • CrewNotificationandCueing Human • Diagnostics&Prognostics disturbance rejection, and upset prevention and recovery. • CrewStateAssessment Operator • Cockpit • Redundancy&Architecture • OptimalRoutePlanning • Cabin Management • ImprovedSituationalAwareness They can also be utilized to support improved crew training, • CargoBay • Equipment Bay especially for providing insight into non-intuitive control FlightSafetyAssessment&Management VariableAutonomyAssessment&Management Onboard • AutonomyRequirementsAssessment • ProbabilityofLossofControlAssessment& Hazards • RecoveryTime&ManeuverRequirements strategies required for upset recovery. Resilient guidance Prediction Analysis • DynamicEnvelopeIdentification&Prediction • EMC • VehicleConstraintsImpactAnalysis • Vibrations • Stability/PerformanceMargins&RiskEstimation functions, such as trajectory generation under vehicle • HandlingQualitiesAssessment • UpsetPrediction&Detction • Recovery&HandlingQualitiesAssuranceunder • RecoverabilityEstimation&Prediction constraints (e.g., vehicle impairment or damage), must also be EmergencyConditions&VehicleConstraints developed. OnboardModeling,Simulation,& ResilientGuidance&ControlforOffNominalConditions Databases • FeasibleTrajectoryGenerationforRecovery&SafeManeuvering,andLanding • VehicleDynamicsModels • RobustAdaptiveControl&TrajectoryTrackingunderOffNominalConditions • AdverseVehicleEffectsModels – RobustnesstoAtmosphericDisturbances • AdverseEnvironmentalEffectsModels • ImprovedUnderstandingofSafe/UnsafeSystemOperations – UpsetPrevention&Recovery • AchievableDynamicsEstimation • ValidationofIntegratedTechnologiesunderHazardousConditions – Failure/DamageMitigation • DamageGrowthPrediction& • StabilityAssurance&PerformanceOptimization AeroservoelasticEffectsAssessment Validation of High-Confidence Systems & Technologies • OptimalControlEffectorRedundancyManagement IntegratedGroundTesting • Database&ModelUpdates Analytical Methods &Tools IntegratedFlightTesting– Subscale &FullScale Unsafe 0.02 Nonlinear 0 a Region -0.02 12 -0.04 Safe Safe -0.06 Invariant Set -0.08 -0.1 1 0.5 0 0.5 0 Figure 6. Aircraft Integrated Resilient Safety Assurance & -0.5 -0.5  -1 -1 Xcg  GW Unsafe Uncertainty Scenario i Failsafe Enhancement (AIRSAFE) System Concept and Functions [6]. Vehicle Health Safe Flight Deck Systems Vehicle Dynamics Flight Safety Management Management Modeling for Off-Nominal & Operations & Resilient Control Conditions VehicleHealthState FlightSafety Airframe Propulsion Upsets Damage Control Authority Display Control Surface Status Display ROLL Current Setting L AIL R AIL PITCH New Vehicle Fai lures / Limi ts Upse Damage 08 -12 L R Onboard modeling capability is reflected by purple. These ts ELEV ELEV AbnormalFlight YAW Atmospheric 03 03 R AIL STUCK 8 DEG Control Fl ight Condi tions RUD ROLL LMTS SET Surface Conditions Icing -11 DGD A/C PRF – POT ICING SIG INJ – TE ST R ELEV Displacement Scale Disturbances Integrated Message Area core functions and capabilities directly correlate to those SafeTrajectories RemoteSensing WindShear AircraftSystems InSituSensing /Gusts Wakes Vehicle depicted in Figure 5. Multi-colored boxes represent shared Failures& Damage ExternalHazards functions between the associated subsystems. A detailed • VehicleFlightSafety • ImprovedSituational • ImprovedCrewTraining • CatastrophicFailure Assessment/Prediction/Assurance Awareness&CrewResponse • OnboardModelUpdates& PreventionthroughEarly description of the AIRSAFE System concept, including • RealTimeControlMitigation& underLOCConditions EffectsAssessmentSupport Anomaly/FaultDetection UpsetPrevention/Recovery • AvoidanceofExternalHazards • SystemTechnology • RapidFailure/Damage subsystem interfaces, is given in Reference [11], and Ref. [6] • SupportforImprovedCrew • Informed/Synergistic Development&Validation Detection,Identification, Training/Guidance/Cueing ResponsebetweenCrew& Support Characterization,& Systems Containment provides a synopsized description as well as an initial assessment of the potential effectiveness of the AIRSAFE Figure 5. A Holistic Approach to Prevent Aircraft LOC System concept in providing LOC sequence interventions.

Accidents [6].

IV. V ALIDATION AND V ERIFICATION (V&V) Effective crew-system interface technologies must be V&V becomes much more difficult for safety-critical developed for providing improved situational awareness and resilient systems operating under off-nominal conditions, such crew response under off-nominal conditions. These as the AIRSAFE System Concept of Section III. Due to the technologies include effective visual and aural methods for huge operational space, there are too many conditions to fully notification and cueing, and variable autonomy systems that analyze, simulate, and test. While there are numerous enable optimal partitioning of authority between the crew and technical challenges associated with this problem, some key automation. Effective information exchange and coordination technical challenges are summarized below.

between the vehicle and airspace operations must also be  Development and Validation of Physics-Based Off-Nominal achieved. Remote sensing technologies must be developed for Conditions and Effects Models avoidance of external hazards and disturbances.

– Requires modeling of Validation and verification (V&V) technologies must be » adverse onboard conditions (e.g., faults, failures, damage) » abnormal flight conditions (e.g., unusual attitudes, stall, developed for the comprehensive evaluation of these stall/departure, other vehicle upset conditions) technologies for operation under off-nominal conditions, and » external hazards and disturbances (e.g., icing, wind shear, to enable the identification of system limitations and wake vortices, turbulence) » Worst-Case Combinations (as Determined from LOC constraints as well as safe and unsafe operating conditions Accident/Incident Data) (and their boundaries).

– Requires data and/or experimental methods for off- Based on the holistic approach of Figure 5, an onboard nominal conditions, which may not be available integrated systems concept can be developed. One such – Can involve multidisciplinary coupled effects concept, called AIRSAFE, is presented in Figure 6, including – Cannot fully replicate in-flight loss-of-control environment a detailed depiction of subsystem functions and capabilities.

 V&V of Adaptive Diagnostic, Prognostic, and Control The core subsystems include vehicle health management Algorithms Operating under Off-Nominal Conditions – Involves a variety of nonlinear mathematical constructs developed for assessment of each core component using the (inference engines, probabilistic methods, physics-based, appropriate methods. Although Figure 7 shows some example neural networks, artificial intelligence, etc.)

metrics for algorithm validation, and illustrates that these are – May involve onboard adaptation that can result in dependent on the algorithm type, metrics are needed for each stochastic system behavior core V&V component.

– Involves fusion and reasoning algorithms for sensor data, information processing, and decisions Subsystem/SystemTechnologies – Requires methods for establishing probabilities of • VehicleHealthManagement » false alarms and missed detections • ResilientControl&VehicleSafetyStateManagement » incorrect identifications and decisions • VariableAutonomyFlightDeck • IntegratedAIRSAFETechnologies » loss of stability, recoverability, and control – Requires methods & metrics for establishing off-nominal V&VComponents condition coverage, reliability, and accuracy for diverse V&VPredictive Subsystem/System Subsystem/System algorithms & multiple objectives Validation CapabilityAssessment Verification – Requires integrated multi-disciplinary system assessment • V&VValidity&ConfidenceLevel • Detection/Prediction • Software(SW) • Diagnostics/Prognostics – NominalConditions • Hardware(HW) methods – BoundaryConditions • ControlTheoretic • IntegratedSW/HWSystem • VariableAutonomy – OffNominalConditions » performance assessment » error propagation and effects assessment V&VMethods » inter-operability effectiveness assessment Simulation/ Flight Analysis Simulation/ Flight  System Verification and Safety Assurance GroundTesting Testing Analysis Ground Testing • ValidatedModels,Simulations,andEmulationsof – Involves large-scale complex interconnected software • Stability Testing • FullScale OffNominalConditions • Performance • Batch • Subscale – Individual/Combinations systems • Robustness • RealTime – Linear/NonlinearEffects • Reliability • Piloted – Involves potentially fault tolerant and reconfigurable – MultidisciplinaryEffects • Hardware – Normal/Boundary/Abnormal Flight hardware intheLoop • CrossCorrelation/UtilizationofAnalysis, • LinkedLab Simulation/Ground Test,andFlightTestResults – May involve adaptive and reasoning algorithms with • Subsystem&IntegratedSystemAssessment stochastic behavior – Requires verification methods for a complex system of Simulation/Ground Analysis FlightTesting systems Testing • FormalMethods • Representative • Code/Module – Requirements Flight  V&V Predictive Capability Assessment • Subsystem/System – Logic/Code Environment • SafetyAssurance • Representative – Requires methods to demonstrate compliance to Hardware certification standards for an extensive set of off-nominal V&VMetrics ExampleValidationMetrics: conditions (and their combinations) that cannot be fully Detection/Prediction/Diagnostics/Prognostics: ControlTheoretic: VariableAutonomy: replicated • ConvergenceRate&Accuracy • Stability,Performance,Recoverability • Handling • Robustness(Uncertainties&Disturbances) • Robustness(Uncertainties&Disturbances) Qualities – Requires methods for determining (and quantifying) level • Coverage(OffNominalConditions) • Coverage(OffNominalConditions) • Interface • Reliability • Reliability&Coverage Effectiveness of confidence in V&V process and results for – ProbabilityofFalseAlarms&MissedDetections – ProbabilityofUnsuccessfulMitigation or • Susceptibilityto – ProbabilityofIncorrectIdentification, Prediction, Recovery Aircraft/Pilot demonstrating compliance orDecision – Probabilityof Instability orLossofControl Coupling • Others • Others • Others These technical challenges can be utilized in defining V&V Figure 7. V&V Process Requirements for the process requirements. Key components of the V&V process AIRSAFE System Concept [7].

include system/subsystem validation, system/subsystem verification, and V&V predictive capability assessment. Each Based on the V&V process requirements of Figure 7, a of these V&V components requires the development of detailed V&V process can be developed for complex methods, tools, and testbeds to perform analysis, simulation / integrated resilient systems, such as the AIRSAFE System ground testing, and flight testing. Moreover, each method, concept of Figure 6. A high-level overview of the integrated tool, and testbed must be developed to assess system V&V process is presented in Figure 8. The colors of the mitigation effectiveness under off-nominal precursor blocks correlate to the associated AIRSAFE subsystem conditions to aircraft loss-of-control accidents in order to functions depicted in Figure 6 – that is, blue correlates to reduce (or prevent) them in the future. V&V metrics must be integrated resilient control and flight safety management defined for the diverse set of algorithms associated with the functions, green represents vehicle health management subsystems and integrated system, and new methods, tools, functions, and yellow is associated with crew interface and testbeds developed (as needed) to assess these metrics.

functions. Multi-colored boxes in Figure 8 represent Based on an analysis of the V&V problem [12], the V&V evaluation of the associated integrated subsystem functions.

process requirements for future systems designed for operation Analysis, simulation, and experimental V&V components are under off-nominal conditions (such as the AIRSAFE System organized in the V&V process of Figure 8 moving from left to concept) can be defined as depicted in Figure 7. This figure right, and system evaluation becomes more highly integrated shows V&V process components, methods, and some example moving to the center (from above and below) and to the right.

algorithm validation metrics that are required for AIRSAFE Also as indicated in Figure 8, results from the V&V process subsystem and integrated system technologies. The core V&V are utilized as an iterative process for refining the algorithm methods of analysis, simulation/ground testing, and flight design of each subsystem. Ref. [7] presents a more detailed testing are applicable to each of the core V&V components description of the controls-related components of the V&V and take on different meanings for each. Metrics must be process (including methods and interfaces). This is depicted in Figure 8 by the red box around the lower two rows of the off-nominal condition effects and their mitigation; and (iv) process. A detailed summary of recent research crew interface technologies for improved situational accomplishments is also provided in Ref. [7]. Reference [12] awareness and variable autonomy under off-nominal provides a detailed description of the entire process.

conditions.

The AIRSAFE System technologies are being developed for Analysis Simulation Testing safety-critical operation under off-nominal conditions, and VehicleHealthManagement(VHM)AlgorithmsDesign their V&V poses significant technical challenges. The V&V problem and the research approach being taken to address it Stability& Stability& VHM Stochastic Hardware Software Performance Performance Performance /Flight Verification& Analysis Robustness were described. V&V process requirements were presented, Analysis Testing SafetyAnalysis Analysis which integrated analytical, simulation, and experimental methods, software tools, and testbeds. A detailed V&V VHM/ Software Piloted Multidisciplinary Multidisciplinary Integrated Verification Nonlinear IRC/ process was defined for application to the AIRSAFE System Nonlinear Hardwareinthe Hardwareinthe Linear& & Simulation Simulation LoopNonlinear Loop Nonlinear Safety CVI Evaluation Simulation Flight Evaluation Analysis Analysis concept.

Evaluation Evaluation A CKNOWLEDGMENT Stability& Stability& Flying Software The AIRSAFE System concept and V&V process presented Performance Nonlinear Flight IRC/ Performance Qualities Verification& Robustness Analysis Testing Analysis Analysis SafetyAnalysis Analysis in this paper were developed in collaboration with Dr. Celeste CVI M. Belcastro of NASA Langley Research Center, who lost her IntegratedResilientControl(IRC)AlgorithmsDesign Crew/VehicleInterface(CVI)AlgorithmsDesign courageous and selfless battle with cancer and passed from this life on August 22, 2008. Continued work in these areas, as F IGURE 8. V&V P ROCESS O VERVIEW [7].

well as in aircraft LOC accident prevention, is dedicated to her memory.

V. C ONCLUSION Aircraft loss of control is the largest aircraft accident category, and results in the highest number of fatalities among R EFERENCES the worldwide commercial jet fleet. It is also the most [1] “Statistical Summary of Commercial Jet Airplane Accidents, Worldwide Operations, 1959-2009”; Boeing Commercial Airplanes, July 2010.

complex accident category, resulting from numerous causal [2] “Global Fatal Accident Review 1997-2006”, Civil Aviation Authority, and contributing factors that act individually or (more often) Safety Regulation Group, CAP 776, July 21, 2008.

combine to result in a loss of control event (accident or URL: http://www.caa.co.uk/docs/33/CAP776.pdf incident). These factors are off-nominal conditions that occur [3] Evans, Joni K., “An Examination of In Flight Loss of Control Events During 1988–2004”, Alliant Techsystems, Inc., NASA Langley onboard the aircraft, as external disturbances, or as abnormal Research Center, Contract No.: TEAMS:NNL07AM99T/R1C0, Task flight conditions. To address aircraft loss of control, a detailed No. 5.2, 2007.

LOC accident analysis was performed to identify worst-case URL: http://www.boeing.com/news/techissues/pdf/statsum.pdf combinations of causal and contributing factors as well as [4] Lambregts, A. A., Nesemeier, G., Wilborn, J. E., and Newman, R. L., “Airplane Upsets: Old Problem, New Issues”, AIAA Modeling and their temporal ordering or sequencing in time. The data set Simulation Technologies Conference and Exhibit , 2008, AIAA 2008- used in the analysis consisted of 126 LOC accidents that 6867.

resulted in 6087 fatalities during the 30-year period 1979 – [5] Belcastro, Christine M. and Foster, John V., “Aircraft Loss-of-Control 2009. Scatter plots were used in identifying worst-case Accident Analysis”, AIAA Guidance, Navigation, and Control Conference , Toronto, Canada, 2010.

combinations of LOC accident precursors, and a set of 7 [6] Belcastro, Christine M. and Jacobson, Steven R., “Future Integrated generalized LOC sequences was defined, which represent Systems Concept for Preventing Aircraft Loss-of-Control Accidents”, 88.9% of the accidents and 90.8% of the fatalities considered AIAA Guidance, Navigation, and Control Conference , Toronto, Canada, in this study. Future risks with the potential to increase LOC 2010.

[7] Belcastro, Christine M., “Validation and Verification of Future accidents were also considered.

Integrated Safety-Critical Systems Operating under Off-Nominal A holistic research and technology development approach Conditions”, AIAA Guidance, Navigation, and Control Conference , was presented for reducing aircraft LOC accidents, as well as Toronto, Canada, 2010.

an associated integrated system concept, called the Aircraft [8] Aviation Safety Network (ASN) Database Available at http://aviation- safety.net/database/ Integrated Resilient Safety Assurance and Failsafe [9] National Transportation Safety Board (NTSB) Database Available at Enhancement (AIRSAFE) System. The holistic approach http://www.ntsb.gov/ntsb/query.asp requires the development of (i) modeling and simulation [10] Joint Planning and Development Office, “Concept of Operations for the Next Generation Air Transportation System”, Version 3, October 2009, technologies for characterizing vehicle dynamics and control Available at http://www.jpdo.gov/library.asp characteristics under off-nominal precursor conditions [11] Belcastro, Christine M., and Belcastro, Celeste M.: Future Research associated with LOC events; (ii) vehicle health management Directions for the Development of Integrated Resilient Flight Systems to Prevent Aircraft Loss-of-Control Accidents, Part I: System technologies for the detection, identification, characterization, Technologies; NASA TM (being finalized for publication).

and containment of vehicle and system failures and damage [12] Belcastro, Christine M., and Belcastro, Celeste M.: Future Research (as well as their prevention through improved maintenance, Directions for the Development of Integrated Resilient Flight Systems to inspection, and vehicle design); (iii) flight safety management Prevent Aircraft Loss-of-Control Accidents, Part II: Validation and Verification; NASA TM (in final preparation).

and resilient control technologies for the rapid assessment of