Analytical and simulator study of advanced transport

NASA CR NASA Contractor Report 3572 c. 1

Analytical and Simulator Study of

Advanced Transport Handling Qualities

William H. Levison and William W. Rickard CONTRACT NASl-16410 JUNE 1982 I..

TECH LlBkWtY KAFB, NM

l~lllulBl~lllmlhlll#l

OOb21b8 NASA Contractor Report 3572

Analytical and Simulator Study of

Advanced Transport Handling Qualities

William H. Levison Bolt Berunek and Newman Inc.

Cambridge, Mussachsetts William W. Rickard Douglas Aircraft Compazy Long Beach, California Prepared for Langley Research Center under Contract NASL-16410 National Aeronautics and Space Administration Scientific and Technical Intormation Office -- LIST OF TABLES 2. Pilot-Related pilode Parameters Appropriate to Glideslope 5.

6. Subjective Estimates of Attention to Cockpit Instruments Longitudinal Dimensional Stability Derivatives for Bl.

B2. Longitudinal Dimensional Stability Derivatives for Lateral-Directional Dimensional Stability Derivatives . . 44 B3.

El. Pilot Debriefing Form ................ .52-54 Fl.

Analysis of Pilot Rating Scores: T-Tests of Paired G2.

Differences .......................

............

G4. Objective Performance Scores. -68-74 LIST OF FIGURES 1. Procedures for Predicting Pilot Ratings 2.

Predicted Performance/Workload Tradeoffs.

.............

3. Six-Axis Motion Base Simulator.

4. Simulator Cockpit 5. Simulator Control Box Approach Geometry 6.

7. Comparison of Measured and Predicted Criteria 8. Comparison of Predicted and Measured "Error" Variability .......................

.26-27 Scores.

Effect of Throttle Control on Predicted Performance 9.

Index..........................2 7 ii LIST OF FIGURES (cont.)

Bl. Augmentation System for Configuration 5. . . . . . . . 43 Cl. Layout of Motion Base Simulator. . . . . . . . . . . . 46 Captain's Main Instrument Panel. . . . . . . . . . . . 47 c2.

. . . . . . . . . . 48 c3. Center Main Instrument Panel . . .

. . . . . . 57 Fl. Pilot Comment Card . . . . . . . . . . . .

. . . .

Cooper-Harper Pilot Rating Scale . . . . . . . 58 F2.

. . I . . . . . . . 59 F3. Pilot Rating Form . . . . . . . .

. . . . . . . . . . 61-62 F4. MBS Line Printer Page. . . . . . .

iii PREFACE This report summarizes the work performed for NASA Langley Research Center under Contract No. NASl-16410 by Bolt Beranek and Newman Inc. (prime contractor) and Douglas Aircraft Company (subcontractor). Dr. William H. Levison was Principal Investigator for Bolt Beranek and Newman, William W. Rickard was Project Engineer for Douglas and Mr.

Aircraft Company. Mr. Martin T. Moul and Mr. David B. Middleton served as NASA Technical Monitors.

iv LIST OF SYMBOLS A Relative attention to flight control task A Aircraft stability derivative pertaining to elastic mode B Aircraft stability derivative pertaining to elastic mode BW Bandwidth F Control force acceleration of gravity, 32.17 feet/second2 h Height error, feet L Aircraft stability derivative pertaining to roll axis M Aircraft stability derivative pertaining to pitch axis N Aircraft stability derivative pertaining to yaw axis n normal acceleration, g's OCM Optimal Control Model roll rate P degrees/second Pitch rate, q R predicted Cooper-Harper rating r yaw rate S Laplace frequency variable, radians/second Time constant, seconds T Reference airspeed, feet/second u.

incremental aircraft velocity along the longitudinal body U axis, feet/second U. Airspeed error relative to moving air mass, feet/second aircraft speed, knots V V incremental aircraft lateral velocity, feet/second W incremental aircraft velocity along the vertical body axis, feet/second Aircraft stability derivative pertaining to longitudinal X body axis Aircraft stability derivative pertaining to lateral body Y axis Aircraft stability derivative pertaining to vertical body z axis V LIST OF SYMBOLS (Cont.)

Angle of attack, radians vertical flight path angle, degrees thrust deviation from reference, pounds elevator deviation from reference, degrees generalized control input variable damping ratio (dimensionless) Generalized aircraft response variable associated with elastic mode Pitch error, degrees standard deviation roll angle heading angle frequency, radians/second Subscripts a aileron control elevator control cc e elevator control gust input g n normal acceleration roll rate P pilot P aircraft phugoid response mode ph pitch rate q r rudder control RP rudder control S elevator control aircraft short period response mode sP U X-axis velocity Y-axis velocity V W aileron control (wheel) W Z-axis velocity a angle of attack vi Conversion Factors To Obtain Multiply by - . 3048 meters feet meters inches . 02540 knots . 5148 meters/second . 01745 radians degrees newtons pounds 4.448 kilograms pounds .4536 vii An analytic methodology, based on the optimal-control pilot is demonstrated for assessing longitudinal-axis handling model, qualities of transport aircraft in final approach. Calibration of the methodology is largely in terms of closed-loop performance requirements, rather than specific vehicle response is based on a combination of published characteristics, and criteria, pilot preferences, physical limitations, and engineering judgment.

Six longitudinal-axis approach configurations were studied covering a range of handling qualities problems, including the The analytical procedure was presence of flexible aircraft modes.

used to obtain predictions of (a) Cooper-Harper ratings, (W a scalar quadratic performance index, and (c) rms excursions of important system variables. A subsequent manned simulation study yielded objective and subjective performance measures that varied across vehicle configurations in the manner predicted by model analysis. In particular, flexible modes for the specific configurations explored in this simulation study were correctly predicted to have no significant effect on handling qualities.

Although performance trends were adequately predicted, experimental error scores were consistently greater than predicted.

Therefore, the analytic scheme is recommended for use in obtaining handling qualities estimates.

comparative, rather than absolute, 1. INTRODUCTION 1.1 Background A certain dichotomy is associated with the topic of flying qualities assessment. From the pilot's point of view, the flying qualities of an airplane, in a given task, relates to the degree to which adequate performance can be achieved with reasonable levels of pilot workload Nevertheless, flying qualities r1,21 l specifications are written in terms of open-loop vehicle response order to aid the airplane manufacturer in characteristics in determining compliance with the specifications. Accordingly, with only partial success, considerable effort has been expended, to find the combination of aircraft response parameters that will, reliably predict closed-loop performance and pilot workload.

criteria, In : contrast to open-loop vehicle-centered pilot/vehicle model analysis allows one to explore issues related to closed-loop performance as well as to workload demands made on the pilot. The effects of external disturbances and control/display parameters, as well as inherent pilot limitations, Perhaps most important, predictive schemes can be considered.

not constrained to based on pilot/vehicle analysis are "conventional" dynamics and can therefore be applied to flying having high-order response qualities studies of aircraft characteristics.

Hess [3] and Levison [4,5] have proposed two similar schemes, based on the optimal-control model (OCM) for pilot/vehicle systems Levison's scheme was for predicting pilot opinion ratings.

161, recently tested against data obtained in a previous simulation Results of study of commercial transport handling qualities [5,71.

this test were sufficiently encouraging to warrant the follow-on study that is the subject of this report.

The prediction scheme is based on the following assumptions: (a) pilot rating is a function of the flig'ht task: (b) for a given flight task there exist one or more critical subtasks which serve (c) performance as the primary determinants of pilot rating: requirements are well defined for each critical subtask; (d) pilot opinion is based partly on the degree to which desired performance is achieved and partly on the information-processing workload associated with the task; and (e) a reliable model exists for predicting performance/workload tradeoffs for relevant flight tasks.

These assumptions lead to the procedure diagrammed in Figure In effect, the analytic prediction scheme parallels the 1.

procedure that would be followed in performing a well-controlled handling qualities simulation study, the major difference being the use of the optimal control pilot/vehicle model to obtain pilot ratings and other performance measures.

DEFINE TASK DEFINE SUBTASK DEFINE I PERFORMANCE CRITERIA DEFINE PERFORMANCE VS.

WORKLOAD $ PREDICT PILOT RATING f Figure 1. Procedure for Predicting Pilot Rating The following empirical rating expression was developed in the initial study [4,51 and used in the subsequent effort: l<R<lO -- where R is the predicted Cooper-Harper pilot rating.; c is the probability that one or more important system variables will exceed its maximum acceptable value, and A is a measure of the relative attention (i.e., workload) associated with the task.

The pilot is assumed to operate on the performance/workload tradeoff curve, predicted by the OCM, so as to minimize R.

A good fit to experimental data in the preceding study was found with0 o=O.l and Ao=2.

"Attention" is reflected in the OCM by a signal-to-noise parameter -- one of the independent model parameters that account for the human operator's information processing limitations [8,91.

Since attention is referenced to a level inferred from data obtained in a standardized laboratory tracking task, rather than to some assumed capacity, values greater than unity are possible.

1.2 Objectives This report summarizes the results of a study by Bolt Beranek and Newman Inc. and Douglas Aircraft Company to provide a rigorous test of the handling-qualities assessment scheme reported previously by Levison [4,5].

Study goals included (a) development of closed-loop performance criteria, (b) a tightly-constrained manned simulation to yield Cooper-Harper opinion ratings with minimal inter-pilot variability, and (c) compilation of a data base of objective performance measures suitable for methodological development. An additional goal was to explore the effects on pilot opinion rating of simulating flexible modes of transport aircraft, and to determine whether or not the analytic scheme would predict these effects.

The study summarized in this report was limited to a single area of application: longitudinal-axis handling qualities of commercial transports in final approach.

This particular application was selected to provide a tie to previous results and does not reflect a limitation of the methodology.

In theory, the methodology should be equally applicable to lateral-axis handling qualities and to other aircraft types and other flight phases.

Some of the experimental configurations explored in this study were selected to induce specific handling qualities problems and are not representative of aircraft either currently or prospectively operational. The simulation results presented in this report, therefore, are intended as a basis for testing the predictive capabilities of the analytic scheme and not as a definitive study of transport handling qualities.

This study was performed in three sequential phases: 1. Pre-experiment analysis was performed to (a) assist in defining the experimental task, (b) make an initial selection of aircraft configurations to be explored in the manned simulation, (c) define independent parameters of the analytic rating prediction scheme, and ('da) predict pilot ratings and objective performance measures. This phase is described in Section 2.

2.

A manned simulation was performed with four test pilots to obtain Cooper-Harper pilot opinion ratings for six vehicle configurations. Means and standard deviations were computed for various system variables of interest. The simulator characteristics and the testing procedures are described in Section 3.

3. Post-experiment statistical analysis was performed on both subjective and objective performance measures, and experimental trends were compared to those predicted by the analytic scheme.

Experimental results are summarized in Section 4.

Additional details relating to experimental procedures are - given in Appendices A-F, and detailed tabulations of experimental results are given in Appendix G. Readers not familiar with the model-based scheme are directed to the literature [4,51 for a review of the methodology that includes both an overview of the optimal control pilot model, and a description of how independent model parameters are selected for this type of application.

2. PRE-EXPERIMENT ANALYSIS The pre-experiment phase of the study consisted of three major task areas as described below: task definition, preliminary selection of aircraft configurations,'and model analysis.

2.1 Task Definition The flight tasks to be performed by the test pilots were defined, closed-loop criteria were specified for model and analysis. As in the preceding study, the task was that of piloting a simulated large commercial transport aircraft in final approach.

Three subtasks were defined: (1) altitude station keeping prior to glideslope capture, (2) glideslope capture, and (3) post-capture tracking of the glideslope. Flare and landing were not considered (and were not performed in the simulation study). For purposes of pre-experiment model analysis, zero-mean turbulence as defined by assumed, with pitch and yaw rate the Dryden model was components omitted. The turbulence model is discussed further in Section 3.1; a complete mathematical description is given in Appendix D.

To ascertain closed-loop requirements, interviews were held with five potential test pilots to determine what they considered to be maximum acceptable values, or "limits", for important system variables in moderate turbulence. (In general, the pilots interpreted a "maximum" value as an excursion indicative of poor approach performance.) Assessments were'obtained for each of the flight subtasks, and for various altitudes with respect to the glideslope tracking subtask. The responses presented below are those appropriate to the conditions explored in the pre-experiment "frozen-point" (i.e., model analysis, which consisted of a steady-state) analysis of glideslope tracking at a reference airspeed of 140 knots, a 3 degree descent angle, and a reference altitude of 500 ft.

On the average, the following zero-peak acceptable excursions from trim were specified: glideslope: l/2 dot* sinkrate: 250 ft/min airspeed: 7.6 kts pitch: 3.5 degrees stick: 28% maximum excursion 4% aircraft weight thrust: For airspeed and sinkrate excursions, for which the pilots tended to impose asymmetric criteria, the above values reflect l/2 the The limit on thrust distance between upper and lower bounds.

represents a distillation of the pilot responses, which were expressed in different units by different pilots (inches throttle The pilots agreed that there movement, percent Nl, change in EPR).

was also a subjective limit to pitch rate, but as they could not assign a quantitative value to this parameter, it was excluded from the list of performance requirements.

Although the pilots did not provide subjective limitations to rate-of-change of stick and throttle, "limits" for these quantities were defined partly to satisfy certain mathematical requirements of the optimal control model, and partly to satisfy physical constraints. A stick rate limit of 28% maximum slew rate was assumed, and the limit on rate-of-change of thrust was set equal to to reflect low-bandwidth l/2 the limit on thrust deviation operation of this control.

To provide the scalar quadratic performance index needed to obtain model solutions, weighting coefficients were defined as the reciprocals of the squares of maximum acceptable values. Thus, an rms derivation of a given system variable equal to its "limit" contributed one unit to the overall "cost".

2.2 Preliminary Selection of Candidate Configurations The set of vehicle configurations selected for experimental study had to meet two conflicting objectives. First, the set had to be sufficient to allow exploration of a range of handling qualities, including potential problems related to flexible an initialsetof configurations airframe properties. Accordingly, stability, was selected to explore effects of relaxed static control augmentation, and structural modes.

* A "dot" is a calibration marking on the glideslope indicator that .35 degrees of path error for the instrument represented about At an altitude of 500 feet, one dot used in the simulation study.

corresponded to a height error of about 58 feet.

A second objective of the study was to obtain data suitable for statistical analysis, which dictated that a number of replications of the entire test matrix be obtained for a number of test pilots. In order to keep simulation costs within reasonable bounds, therefore, the number of test configurations was limited.

With these goals in mind, a set of eight potential configurations were initially selected; this set was reduced to six following pre-experiment model analysis.

The longitudinal flying qualities levels of the eight initial configurations, as predicted by existing criteria, are shown in :ONFIG wnsp :ONFIG wnsp STATIC STATIC -8785 NO. NO. vs n/u vs n/u STABILITY STABILITY dyldV OVERALL s s cph Or T2 cph Or T2 SP SP ph ph _ .I7 1 1 1 1 STABLE 1 1 1 1 1 1 STABLE 1 1 1 1 WORSE WORSE WORSE WORSE 2 l-1/2 1 1 STABLE THAN THAN 2 l-1/2 1 1 STABLE THAN THAN 1 2-l/2 3 3 3 3 WORSE WORSE WORSE WORSE WORSE WORSE 3 THAN 2 THAN UNSTABLE 1 THAN 3 3 3 THAN 2 THAN UNSTABLE 1 THAN I 3 3 3 3 3 3 WORSE WORSE WORSE WORSE WORSE WORSE 4 THAN 1 THAN UNSTABLE 3 THAN 3 3 4 THAN 1 THAN UNSTABLE 3 THAN 3 3 3 3 3 3 5 2 1 1 STABLE 1 2 5 2 1 1 STABLE 1 2 1 1 6 3-112 2 1 STABLE 1 3-112 6 3-112 2 1 STABLE 1 3-112 1 1 7 3 1 1 STABLE 1 3 7 3 1 1 STABLE 1 3 1 1 WORSE WORSE WORSE WORSE WORSE WORSE 8 THAN 1 THAN UNSTABLE 1 THAN 8 THAN 1 THAN UNSTABLE 1 THAN 2 2 3 3 3 3 3 3 Table 1. Flying Quality Levels of Test Configurations l-8 Table 1. There are columns for five -8785B criteria [l]: (1) short period frequency versus acceleration sensitivity (un versus sP (2) short period damping (csp), (3) phugoid damping (cph) or n/a, (4) static stability, and (Sj flight path stability (&/dV).

‘9 1, ThgP pilot, of course, cannot be asked to rate these individual criteria; his rating of longitudinal flying qualities represents their sum. Since the -8785 provides no guidance on how to combine the pieces; one must use his own judgment. The judgment used here was to represent the "-8785 OVERALL" as the worst of the five preceding columns.

The next column, labeled "BANDWIDTH", is a flying quality prediction using a frequency domain pilot-in-the-loop criterion This criterion has been [41.

demonstrated reliably to predict pilot opinion of longitudinal maneuvering dynamics. As such, it is not sensitive to dy/dV, which is a measure of long-term flight path response. It was shown by Rickard 171 that the combination of the Bandwidth and flight path stability estimates, labeled "BW + dr/dV," yields an estimate of longitudinal flying qualities more accurate than the one labeled -8785 OVERALL. The criteria in Table 1 are the tools used to design a matrix of eight configurations.

While there is no officially accepted relation between flying qualities levels and pilot ratings, most practitioners accept the following: Level 1 2 worse than 3 631,2 l-3 l/2 3 l/2 - 6 l/2 - 9 l/2 9 l/2 - 10 Half levels used in Table 1 indicate that a parameter falls too close to a boundary to allow one to reliably determine which level is correct.

Configuration 1, having the closest correspondence to an operational wide-body commercial transport, is considered the baseline condition. Having been explored in the earlier study by Rickard, it provides a useful point of reference. According to the estimates shown in Table 1 (and to Rickard's simulation study), it has Level 1 longitudinal flying qualities. Configurations 8 and 3 explore a progression of increasing static instability, having times to double of 7.7 and 2.4 seconds, respectively.

Configuration 2 was chosen to explore the issue of flight path stability, with dY/dV=0.34, where 0.24 is the Level 3 limit.

Configurations 4 and 5 explore the issue of control augmentation.

They are the same airplane, an advanced supersonic transport, without (4) and with (5) a full-state feedback flight control system which was designed using implicit model following. The unaugmented airplane has very poor flying qualities, while the augmented version has fair to good flying qualities, depending on the criteria used.

Configurations 6 and 7 explore the effect of structural dynamics on flying qualities. Both have the same rigid-body equations, with Configuration 7 having two additional second-order modes representing structural dynamics. The criteria indicate that these configurations will be rated the same. The -8785B criteria, which cannot estimate the effect of structural modes, predict Level 3 flying qualities. The short period frequencies are too low and the damping ratio unacceptable or too high. The Bandwidth criteria, which should be able to predict this effect as it makes no assumption about model order, predicts Level1 flying qualities.

The linearized equations of motion used in modelling vehicle response characteristics are given in Appendix A, and Appendix B contains additional detail characterizing the response of the various configurations explored in this study.

2.3 Model Analysis Pre-experiment steady-state model analysis was performed for control and display conditions pertaining to glideslope tracking at an altitude of 500 feet, a descent angle of 3 degrees, and a reference airspeed of 140 kts. As recommenced by the military handling qualities specifications [l], analysis was performed with a Dryden gust model having longitudinal and vertical rms gust amplitudes of 10 and 6.6 ft/sec, respectively.

necessary to Before obtaining model predictions, it was specify the following independent model parameters relating to the pilot's information processing limitations: "Cost" (1) coefficients of the quadratic performance index, (2) time delay, (3) motor noise, and (4) observation noise. A brief review of how these parameters were selected is given below. The reader is referred to documentation of the preceding study [41 for additional methodological details.

As described in Section 2.1, cost weighting coefficients were defined as the reciprocals of the squares of maximum acceptable vehicle and control excursions as determined from pilot interviews.

A time delay of 0.29 seconds was adopted to account jointly for pilot and control-actuator delays. The motor noise covariance was set at -50 dB relative to predicted control-rate variance to account for limitations on the pilot's ability to predict perfectly the effects of his control inputs.

Various components of the observation noise were quantified: "threshold" and "residual noise" parameters to account for (1) perceptual resolution limitations, (2) noise/signalparameters were selected to account for attention-sharing effects, and (3) a baseline noise/signal ratio was treated as an independent parameter of the model analysis to sweep out curves of predicted performance versus attentional workload.

On the basis of previous laboratory tracking studies [lo], thresholds of 0.05 degrees visual arc and 0.2 degrees/second visual arc were assumed, respectively, for perception of the displacement and velocity of a given display indicator. Consideration of the eye-to-panel distance, plus the effective display gain (inches of indicator displacement per unit "error"), enabled conversion from visual units to problem units. Additional details on display analysis methodology are contained in 141.

Analysis of the cockpit displays yielded the following perceptual thresholds, in problem units, for an altitude of 500 feet: (a) 4.7 feet height error, (b) 19 ft/sec sinkrate error, (c) 0.43 degrees pitch error, (d) 1.7 degrees/second pitch rate, and (e) 1.9 ft/sec airspeed error. The rather large threshold associated with perception of sinkrate error was a consequence of assuming that the pilot attempts to obtain this information from addition, a the velocity of the glideslope indicator.* In "residual noise" of 0.5 degrees was associated with perception of pitch error to account for the lack of an explicit zero-error reference.

To simplify the analysis, the pilot was assumed to pay equal attention to glideslope, pitch, and airspeed indicators (and was assumed to obtain both displacement and rate information from all but the airspeed indicator). In addition, 34% of the attention was assumed "lost" because of overt scanning requirements. Thus, a relative attention of unity corresponded to relative attentions of 0.22 each to glideslope, pitch, and airspeed variables. As described in the literature, attentional and perceptual factors determined the observation noise variance associated with each perceptual input [61.

Independentmodelparameters are given in Table 2a, and curves of predicted performance versus relative attention, generated by the optimal control model, are shown in Figure 2a.

* As discussed shortly, this effective threshold was considerably reduced by assuming sinkrate information to be obtained from the vertical velocity instrument.

Initial Analysis a) Revised Analysis b) 0.1 0.2 0.4 1 7. 4 0.1 0.2 0.4 1.0 2.0 4.0 RELATIVE ATTENTION RELATIVE ATTENTION Figure 2. Performance/Workload Tradeoffs Predicted Figure 2a shows the following trends: 1. Best achievable performance (i.e., lowest cost) with the baseline aircraft (Configuration 1) and the augmented AST (Configuration 5).

2. Worst performance, and greatest sensitivity to attentional workload, with the unstable configurations 3 and 4.

3. Intermediate performance with the configuration having a mild instability (Configuration 8) and the vehicle having adverse dY/dV (Configuration 2).

4. Negligible effects due to simulation of flexible modes (Configurations 6 and 7).

Table 2 Pilot-Related Model Parameters Appropriate to Glideslope Tracking at a SOO-foot Altitude

I

I

Variable Config- "Limit" cost Perceptual Residual Atten- uration Coefficient Threshold Noise tion

L

a) Initial Analysis I all h 29. 1.2 E-3 4.7 0 .22 all 4.2 5.7 E-2 h 19. 0 I I.

all 3.5 8.2 E-2 0 . 43 0.5 .22 I II -- -- 1.7 I all I '4 all 13. 5.9 E-3 U. 1.9 0 .22 -- -- all 22. 2.1 E-3 6 .ep -- -- all 33 6 9.2 E-4 I I ep -- -- -- 14,000 5.1 E-9 .1,2, AT -- -- -- 7,000 2.0 E-8 3,8 A* ----+- 30,000 1.1 E-9 AT 6,7 AT 15,000 4.4 E-9 AT 18,000 3.1 E-9 4,5 A!i 9,000 1.2 E-8 b) Subsequent Analysis all 0 h 58 3.0 E-4 4.7 .03 h all 4.2 1.8 E-2 0.8 0 .23 8 all 3.5 8.2 E-2 0.43 0.5 ‘1.15 -- all 1.7 0 q I 13 0 U. all 5.9 E-3 1.9 .27 -- -- -- 6 all 22 2.1 E-3 ep -- -- -- 6 all 33 9.2 E-4 -- -- -- Tip 14,000 5.1 E-9 1,2, -- -- -- Ai 2,800 1.3 E-7 3,8, -- -- -- AT 30,000 1.1 E-9 6,7 -- -- -- Ati 6,000 2.8 E-8 - As noted earlier, the scope of the manned simulation study was On the basis of this limited to six experimental configurations.

analysis, Configurations 4 and 5 were dropped from further consideration, as they appeared to be similar in terms of performance/workload tradeoffs to Configurations 3 and 1, respectively.

Application of the rating expression of Equation 1 to the performance/workload predictions shown in Figure la yielded unreasonably large Cooper-Harper ratings (e.g., a rating of 8 for the baseline configuration). Partly for this reason, and partly because the Mil Spec backup document [ill indicates that the initial choice of gust intensities represents a low probability gust intensities were halved for subsequent (1%) of occurrence, analysis and experimentation. The reduced levels represent a 50% probability of occurrence.

In addition to the reduction in gust levels, changes in other independent model parameters were modified prior to reanalysis: (1) the allowable performance "window" for glideslope error was increased to +l "dot" to reflect published Category II specifications,*- (2) the performance window for sinkrate was increased from around 54 ft/sec (240 ft/min) to +7.5 ft/sec (450 ft/min) to allow for the fact that, in actual fiight, the flare maneuver would substantially reduce sinkrate prior to impact; (3) we assumed that the pilot would obtain sinkrate information from the vertical speed indicator, and we decreased the perceptual to reflect assumed visual resolution threshold to 0.8 ft/sec with respect to this instrument: (4) the maximum capabilities acceptable value for rate of change of thrust was reduced to l/5 the corresponding limit on thrust deviation to more strongly reflect the pilot's aversion to frequent changes in throttle setting. Finally, the OCM was used to predict the optimal allocation of attention (i.e., the allocation of attention yielding the smallest predicted performance index).

Model parameters used for this subsequent pre-experiment analysis are given in Table 2b. The attentional allocation shown in this table is the average of the optimal allocations predicted for Configurations 1 (baseline) and 3 (unstable); this average allocation was used for subsequent predictions.

The six configurations retained for the simulation study were reanalyzed as described above, and the resulting * "DC-10 Flight Study Guide", Douglas Aircraft Company, June 1975.

performance/workload tradeoffs are shown in Figure 2b.

As is the case with the initial analysis, the penalty for relatively low attention is greatest for Configuration 3, and inclusion of flexible modes has little predicted influence. The predicted performance/workload tradeoff curves are compressed, however, with little separation among the curves for Configurations 1,6,7, and 8 at all but the lower attentional levels. Application of the rating expression of Equation 1 yields predicted ratings (shown later in this report) that range from Level 1 to Level 3 and are consistent with those observed experimentally for Configurations 1 and 3 in the preceding study [4].

3. DESCRIPTION OF EXPERIMENTS the approach-to-landing task were Manned simulations' of performed on a moving-base transport aircraft simulator located at Four test pilots performed multiple Douglas Aircraft Company.

trials with each of six vehicle configurations. Simulation characteristics and experimental procedures are described below.

3.1 Simulation Characteristics The simulation model used with all configurations was a and lateral-directional airplane. Both longitudinal complete The controls degrees of freedom and controls were provided.

and pedals) were DC-10 hardware. Control feel, (column, wheel, and motion limits were based on the DC-lo. A full force gradients, A flight director display set of DC-10 instruments was provided.

was available but not used as this would affect workload and thus, pilot opinion of flying qualities. Actuator performance and, and engine dynamics typical of wide-body aircraft were simulated.

the equations of motion were used in Standard linearized simulation. Euler integration of the differential equations was performed at 20 hertz. The actuators and other elements with fast dynamics were simulated using difference equations.

research and development motion The simulation is Douglas' The motion platform is shown in Figure 3 and the base simulator.

cab interior in Figure 4. The cab is a DC-10 cockpit with stations flight engineer, and observer.

for captain, first officer, is available but was not used in this Synthetic outside vision experiment. Motion limits for the platform are given in Table 3 The bandwidth for small inputs for a moving mass of 22,000 pounds.

is 1 hertz, which can be boosted to at least 2 hertz by use of The evaluation pre-emphasis filters on the motion drive signals.

pilot sat in the captain's seat and the test engineer in the first officer's seat. The engineer controlled all aspects of the once the computers were started, experiment from this position, using the control box shown in Figure 5. The box has six 3-position toggle switches, six momentary switches and sense lights, and five 16-position thumbwheels with LED readouts above.

The box is on an umbilical so that it can be moved around the cab.

software "reads" these switches, performs the commanded The the appropriate information on the functions, and displays displays.

One of the In this experiment , only a few switches were used.

3-position switches was used to turn turbulence off or on, one thumbwheel was used to select the configuration, and the pilot .gure 3. Six-Axis Motion Base Simulator Figure 4. Simulator Cockpit Table 3.

Motion Limits Axis Excursion Velocity Acceleration Heave + 42 in.

39 in./sec 1.65 g - Sway + 67.5 in.

67 in./sec 2.43 g - Surge + 65 in.

71 in./sec 1.50 g - Roll + 30.7 deg 35.6 deg/sec 447 deg/sec2 Pitch + 33.3 deg 33.6 deg/sec 447 deg/sec2 - Yaw + 38.7 deg 36.3 deg/sec 453 deg/sec2 - number was set using another thumbwheel. Three digits of the LED display showed the run number, another showed the pilot number, and another showed the configuration number currently being used by Another panel contained pushbutton switches to the computer.

control start, stop, reset, and other operational functions.

The task flown was a manual instrument landing system (ILS) approach using status (rather than director) data. Plan and side The Dryden views of the approach geometry are shown in Figure 6.

turbulence model [l] was used,,with 50th percentile intensities and scale lengths for an altitude of 500 feet. This model actually varies with airspeed and altitude, but was "frozen" in the experiment to match the steady-state analysis performance with the OCM. A sum-of-sines implementation was used, which concentrates 6.6 3.3 @ TURBULENCE b,l h=T/SECJ Figure 5.

Simulator Control Box the power at discrete frequencies. Twelve discrete frequencies from 0.0838 to 12.57 radians per second were used. Additional details on the gust simulation are given in Appendix D.

3.2 Experimental' Procedures The evaluations were made by four Douglas test pilots, all of whom had prior experience in motion-base-simulator evaluations of flying qualities. Before the evaluations were begun, a checkout pilot flew the entire test matrix. No discrepancies were found except that, contrary to all predictions, Configurations 6 and 7 were unflyable. The coupling between airspeed and column movement was so tight that loss of control was inevitable if flight-path control was attempted. These were the elastic AST configurations, Figure 6. Approach Geometry which had an unusually high value of X . This had been noticed in but was accept%d when the flying-quality the pretest analysis, estimates turned out to be reasonable. Configurations 9 and 10 were developed to replace 6 and 7, respectively, by reducing X to The flying quapity a small value and increasing X to compensate.

parameters and estimates (see *able 4) for 9 and 10 were virtually identical to those for 6 and 7, yet 9 and 10 flew quite well.

Of the four pilots involved in the evaluations, two made four replications of the test matrix and two made five. Each session began with a briefing in which the test procedure and performance standards were reviewed. The pilot then flew several approaches to warm up, or get used to, the equipment and procedure. He then flew two approaches with each configuration (turbulence off, then on).

The configurations were presented in pseudo-random order, with the order balanced across replications and across pilots. The "turbulence off" runs were flown to allow additional practice, to isolate the turbulence effects, and to gather data for the The pilots could do development of a glideslope capture strategy.

whatever maneuvers and experiments they wanted in these runs after the intercept maneuver. In the "turbulence on" runs, however, they were told to track the ILS to the performance standards at all times. A replication of the test matrix took l-1/2 to 2 hours. A total of 319 approaches were flown for data in the test: 25 more At the end of the test, the were flown by the checkout pilot.

pilots were interrogated again about a number of items, including the performance standards they adopted in the test and how they Table 4. Qualities Levels of Test Configurations l-10 9l ZONFIf SP STATIC -8785 BANO- BW+ vs n/cl or T2 NO. 5 STABILITY d$dV OVERALL WIDTH d$dV SP bh ph 1 1 1 STABLE 1 1 1 1 WORSE WORSE 2 l-112 1 1 STABLE THAN THAN 1 2-l/2 3 3 WORSE WORSE WORSE 3 THAN 2 THAN UNSTABLE THAN 3 3 3 3 WORSE WORSE WORSE 4 THAN 1 THAN UNSTABLE THAN 3 3 3 3 5 2 1 1 STABLE 2 1 1 6 3-l/2 2 1 STABLE 3-l/2 1 1 7 3 1 1 STABLE 3 1 1 WORSE WORSE WORSE 8 THAN 1 THAN UNSTABLE THAN 2 3 3 3 9 3 2 1 STABLE 3 1 1 10 2 1 1 STABLE I- l/2 1 1 ~.

The debriefing form is given in allocated their attention.

Appendix E.

Time A great deal of objective and subjective data was taken.

histories of 50 variables were recorded on digital magnetic tape at Stripchart records of 16 variables 5 hertz for every approach.

were made. The mean, root mean square, maximum and minimum values, and standard deviation of 15 variables were computed on-line and Instantaneous output on a line printer at the end of each run.

values of 10 variables at 10 points along the approach were also printed out. The printer was also used to record bookkeeping and date, run number, information, such as run start time to reduce test engineer's configuration number, pilot number, etc., workload. The subjective data taken included Cooper-Harper pilot ratings, effort ratings for the three subtasks and three aspects of and pilot comments. The engineer made brief handwritten control, notes to supplement the complete record made by the cockpit voice recorder.

Data acquisition procedures are described in greater detail in Appendix F.

4. POST-EXPERIMENT ANALYSIS 4.1 Major Results Statistical analysis was performed on both Cooper-Harper ratings and closed-loop performance metrics. Ratings were first averaged across replications to obtain an average rating per condition per pilot. Population means and across-subject standard deviations were then computed‘from the individual subject means.

Statistics on system "errors" (deviations from trim) were computed for the three steady-state-like segments of the approach (see Appendix F). Results for the final segment of the glideslope tracking task -- corresponding to descent from approximately 700 to 200 feet altitude -- are reported here. The mean and variability components of each error variable were analyzed separately. Only response variability is reported here , as only that error component can be compared with model predictions for the glideslope tracking task. (Recall, the external disturbances were zero-mean processes.) Mean error is primarily reflective of piloting strategy (e.g., carry excess airspeed, "duck under" the glideslope) and is thus not treated directly by the OCM. In general, the variability component was dominant. A variance score was first computed for each error variable of interest within a given replication. The square root of this measure was then treated as the basic error score. Note that this measure reflects within-trial variability, not run-to-run or pilot-to-pilot variability. Error scores computed in this manner were then subjected to the same statistical analysis as described above for the pilot ratings. Results of the analysis of rating scores and error scores are tabulated in Appendix G.

For the reader's convenience, Table 5 provides a brief identification of the six test configurations explored in the experimental study.

Measured and predicted pilot opinion ratings are presented in Figure 7a. standard deviations (designated by Across-subject vertical bars) were generally less than one rating unit. Thus, the experimental technique yielded rating predictions that were reasonably consistent across pilots.

The trend of the experimental ratings agreed well with pre-experimental model predictions: configurations 1,8,9 and 10 were rated similarly, whereas configurations 2 and 3 received ratings that were appreciably more adverse. The major discrepancy between prediction and experiment was the relative compression of Table 5. Identification of Test Configurations No. Identifier 1 Baseline: closest to wide-body transport Backside of power curve: large positive ay/aV 3 Significant longitudinal static instability Slight longitudinal static instability 9 Rigid-body approximation to flexible AST 10 Flexible AST the simulation results, with the "better" configurations receiving In addition, Level 2 rather than the predicted Level 1 ratings.

configurations 2 and 3 were rated nearly the same on the average, whereas the analytic technique predicted a 2-unit spread on the Cooper-Harper scale.

In a previous study, in which lateral-directional characteristics were considered to reflect Level 1 handling qualities, the "baseline" Configuration1 received an average pilot rating in the Level 1 range [4,7].- Lateral characteristics were less favorable for the study reported here ,,receiving rating scores in the Level 2 range. Thus, we suspect that the greater than expected rating scores for Configurations 1,8,9 and 10 reflected, in part, an interaction with the lateral-axis task. (Model predictions were based on the assumption that the lateral-axis task would present no appreciable handling qualities problems.)

Configuration 2 was also explored in the previous study. In that study, as well as in the current one, the rating score obtained in the simulation study was greater than predicted analytically. As discussed shortly, this model/experiment difference may be due in part to a failure of the analytic scheme, as described so far, to consider the adverse effects of requiring loop closures that are not part of the pilot's standard repertoire.

measures of quadratic Predicted and experimental the performance index are compared in Figure 7b. Two sets of model obtained with predic,tions are shown: scores relative (4 attentions corresponding to minimum ratings, as determined from the Pilot Opinion a) b) Performance Index 10.0

P

E

A

u

0 0

q 0 q S A A A A 0 MEASURED 0 PREDICTED: MINIMUM RATING 0.2 A PREDICTED: UNITY ATTENTION s a 1 E : 1 ,Y B

0.1 -2

1 2 3 8 9 10 1 2 3 10 8 9 EXPERIMENTAL CONFIGURATION EXPERIMENTAL CONFIGURATION Pigure 7. Comparison of Measured and Predicted Criteria expression of Equation 1, and (b) scores corresponding to a relative attention of unity.

Although scores measured during simulation were considerably greater than predictions , predicted trends were confirmed. As with the rating scores, performance scores for Configurations 1,8,9, and 10 were similar, whereas substantially greater (less favorable) scores were observed for Configurations 2 and 3.

Comparison of predicted and measured "error" variability scores for selected r,esponse variables is given in Figure 8.

Again, measured scores were greater than analytic predictions, but trends related to effects of vehicle characteristics were generally in agreement. In particular, the analytic ,procedure correctly Height a) Airspeed b) I- l- I- F 1 2 3 8 9 10 1 2 3 8 9 10 EXPERIMENTAL CONFIGURATION EXPERIMENTAL CONFIGURATION Pitch cl

I

a

I I I I I I 1 2 3 8 9 10 EXPERIMENTAL CONFIGURATION Comparison of Predicted and Measured "Error" Figure 8.

Variability Scores Throttle Elevator a) gYLYlll "1

J

P

P

A

P -

P

El I I I I I I I 1 2 3 8 9 10 1 2 3 8 9 10 EXPERIMENTAL CONFIGURATION EXPERIMENTAL CONFIGURATION Comparison of Predicted and Measured Figure 8 (Concluded).

"Error" Variability Scores 5.0 - X P ii 2.0 - x % E l.O- t % 0.5 - 0.2 - 0.1L: 1 2 3 6 9 EXPERIMENTAL CONFIGURATION Figure 9. Effect of Throttle Control on Predicted Performance Index predicted that relatively large elevator deflections would be required for Configuration 3, whereas large thrust changes would be required for Configuration 2. Overall, the two predictions of objective performance scores replicated experimental trends with similar fidelity. The (relatively modest) differences between the two sets of model predictions are discussed in Section 4.2.

Additional model analysis was conducted to determine methods for, first, obtaining a more accurate assessment of the adverse handling qualities associated with Configuration 2, and, second, for predicting the severe controllability problems found experimentally with Configurations 6 and 7. Compared to the baseline configuration, these three configurations required a strategy that relied more heavily on throttle for height control and elevator for speed control: Configuration 2 because of adverse dY/dV characteristics, and Configurations 6 and 7 because of a high pitch-speed coupling., This observation suggested a simple technique for analytically detecting handling qualities problems associated with undesirable throttle activity; specifically, model analysis was performed with and without the throttle control active. To test the discriminability of the procedure, reanalysis was performed also for Configuration 1 (baseline), Configuration 3 (greatest instability), and Configuration 9 (Configuration 6 modified to reduce the pitch-speed coupling). This analysis was performed with the baseline observation noise/signal ratio adjusted to reflect unity relative attention.

Figure 9 shows that this method readily identified handling qualities problems related to throttle activity. The predicted quadratic performance indices for Configurations 1,3, and 9, while different from each other, were relatively unaffected by the exclusion of throttle control. On the other hand, omission of throttle control caused the performance metric to more than double for Configuration 2 and to increase nearly sevenfold for Configuration 6.

Thus, a model comparison of this sort appears to be a simple device for predicting handling qualities difficulties caused by requirements for significant throttle activity.

Results of the pilot debriefing supported the methodology followed in this study.

(A sample form is given in Appendix E.)

The pilots were first asked to rate the relative importance of the three flight subphases (pre-capture altitude hold, glideslope capture, and glideslope tracking) with regard to determining their Cooper-Harper rating score for the approach.

Glideslope tracking was rated most important by three of the four test pilots: the remaining pilot rated GS tracking second in importance. Thus, it seems reasonable to focus on the glideslope tracking phase for both analytic and experimental determinations of aircraft handling qualities in final approach.

The pilots were also asked to estimate the level of attention paid to various cockpit instruments during the approach as either Table 6 shows that, for "high, "moderate", or "little or none".

the glideslope tracking tasks, moderate to high levels of attention were estimated for variables included in the pilot's "display for model analysis, whereas moderate to low levels were vector" estimated for variables not considered during analysis.

Table 6. Subjective Estimates of Attention to Cockpit Instruments for the Glideslope Tracking Task Instrument Level of Attention High Moderate Little or None 4 -- -- Glideslope -- Localizer 4 -- 2 2 -- Artificial Horizon 1 3 -- Airspeed Vertical Speed 1 2 Altimeter -- -- 4 Engine (Nl) -- 1 Entries indicate number or responses.

received with regard to maximum A range of responses was these allowable "error" associated with adequate performance; responses were generally consistent with values used during the model analysis.

4.2 Discussion The model-based prediction scheme was found to be a good trends across the various predictor of handling qualities configurations. Criteria for which trends were predicted included (1) Cooper-Harper ratings, (2) a performance index providing an overall scalar metric for closed-loop system performance, and (3) standard deviation scores for important variables. As the model results were obtained before the manned simulation was performed, these results are honest predictions.

The analytic prediction scheme differs from most published that it is based primarily on a handling qualities criteria in description of the task being modeled, rather than on specific of the method is aircraft response characteristics. "Calibration" specifications of the external characterized largely by disturbances and of the maximum acceptable excursions for important The latter are based on a combination of system variables.

published criteria, pilot preferences (determined from interviews), from laboratory human response limitations (determined , physical limitations of the aircraft control system, experiments) and engineering judgement. Ideally, once calibrated for a specific task, the analytic method is capable of rendering valid predictions of handling qualities trends across a variety of aircraft configurations. Additional application will be required to define its range of validity.

Repeatable Cooper-Harper pilot opinion ratings were obtained by following a strict experimental protocol during the simulation.

Important aspects of the experiment design and procedure were: (1) use of stationary, zero-mean, pseudo-random inputs to simulate gust disturbances, (2) well-defined subtasks and associated performance standards, and (3) pilot familiarization with the various experimental configurations.

A procedure was developed for using the model-based scheme to reveal low-frequency flight path control problems, including the pitch-speed coupling problem that was not revealed (at least to its This fullest extent) by existing handling qualities criteria.

procedure involves a two-step process. First, model predictions Model are made with the throttle included as a continuous control.

predictions are then repeated with only the column considered as an the throttle yields an operating control. If exclusion of appreciable degradation in predicted performance, flight path control problems (and adverse pilot opinion ratings) can be expected.

This particular treatment correctly predicted a substantial improvement in handling qualities resulting from a reduction of the pitch-speed coupling characteristics of test configurations 6 and 7. The published criteria failed to predict this improvement.

The analytic prediction scheme correctly predicted the absence influence of structural simulation of. an mode on handling qualities. Additional data are necessary, of course, to determine whether or not this procedure is generally able to quantify the effects of structural modes.

Figure 7 shows that the scalar quadratic performance index mimics the pilot ratings trends. These results confirm the earlier study by Hess [31, in which a monotonic relationship between pilot rating and predicted performance index was demonstrated.

The primary difference between the method applied in this study and that of Hess is in the method for selecting independent model parameters -- especially weightings for the quadratic performance index. Hess has recently proposed a method for selecting weighting coefficients through a rather complex sensitivity analysis using the pilot model [12], whereas the method discussed here relies on task analysis. We suspect that one not thoroughly experienced in the application of the pilot model would find the latter procedure easier to follow.

As described above, model predictions of objective performance measures were obtained in two ways. One prediction scheme simply used the predicted scores corresponding to unity relative attention. The other scheme used predicted performance/workload tradeoffs, along with the pilot rating expression of Equationl, to find the error score corresponding to minimum predicted pilot rating. Figures 7 and 8 show that for total performance index and for height and pitch excursions, the fixed-attention method predicted lower scores for all configurations except for Configuration 3, in which case the minimum-rating scheme predicted lower scores. These methodological differences are readily explained by noting the "attention" level predicted for minimum pilot rating. In the case of Configuration 3 (which exhibited a significant static instability) , minimum rating was predicted for a relative attention greater than unity. (Notice the steep .

performance/workload relationship predicted for this configuration in Figure 2b). The error score predicted for this configuration by the minimum-rating scheme was therefore less than the score predicted for the same configuration using unity attention.

For all remaining configurations, however, minimum rating was predicted for attentions less than unity (typically around 0.3), and the trend was reversed.

There is some evidence to indicate that pilot opinion ratings were less influenced by attentional demand than was assumed in the application of the rating expression of Equation 1. Figure 7a shows that Configuration 3 is the only configuration for which the rating obtained in simulation Cooper-Harper the study was numerically lower than the predicted rating. Recall that Configuration 3 was also the only Configuration for which a relative attention greater than unity was predicted. Furthermore, Figures 7a and 7b show that Configurations 2 and 3, which yielded nearly identical pilot rating, also yielded nearly the same quadratic performance index. Given the lack of a demonstrated influence of attentional workload, it appears that predictions of relative aircraft handling characteristics can be obtained just as and with less effort, reliably, through model analysis using an assumed constant level attention.

summarized in Table G3 of Appendix The effort ratings scores, replicate the trends of the objective performance scores G, (Figures 7 and 8). The rating scores for the glideslope tracking task as a whole show maximum subjective effort for Configurations 2 and 3 -- the configurations for which the Cooper-Harper rating and scalar performance index were least favorable. The trends of the effort ratings scores for individual control requirements are as follows, with trends of the objective performance measures given in parentheses: 1. Greatest effort for vertical flight path control was required Configurations 2 and 3 (largest error for score by Configuration 2.)

2. Greatest effort for airspeed control was required by Configuration 2 (largest error score for Configuration 2).

3. Greatest effort for pitch control was required by Configuration 3 (largest pitch "error" for Configuration 3).

Thus, the subjective effort rating scores do not appear to contain new information, but tend to confirm the results of the objective performance measurements.

Although performance trends were predicted with reasonable fidelity, actual performance scores were about a factor of two greater than those predicted by the analytic procedure. Within the constraints imposed by the remaining contractual resourcesl a number of hypotheses were explored to account for this discrepancy.

First, it was noted that prediction errors could not be accounted for by differences in the tradeoff between control effort and system error reduction. If, for example, the pilots were to assign relatively more importance to minimizing control effort than assumed in the model analysis, predicted errors would be greater, but predicted control deviations would be less. As both error and control scores were larger than predicted, we must discard this hypothesis.

Both the time delay and baseline noise/signal ratios were individually increased to reflect attention-sharing penalties beyond those already accounted for in the analysis to account, in for the non-negligible workload imposed by the part, lateral-directional control task. These tests were made for Configuration 1. Decreasing the overall attention (to the longitudinal control axis) to 0.1 did increase predicted error scores, but not sufficiently to match experimental results.

Similarly, increasing the time delay parameter from 0.29 seconds (pilot and control-system delay combined) to 1.0 seconds failed to increase predicted scores sufficiently. Thus, reasonable manipulations of these parameters do not appear to account for model mismatch.

A look at time history tracings for the elevator control input (not shown here) for pulse-like control reveals a tendency behavior. Often, the pilots appeared to make one-sided corrections with the elevator, then return the stick to the center position before initiating another corrective response. Such behavior is indicative of non-Gaussian (and nonlinear) response behavior and, as such, violates one of the model assumptions. Further research is needed to determine whether or not this violation is sufficient to explain the large experimental error scores.* * Pulse-like control behavior does not necessarily rule out application of the pilot model (which assumes that pilot is a linear, but noisy , processor of information). To the extent that the principal effect of this type of nonlinear response is to introduce spectral components beyond the passband of the closed-loop system, aircraft response variables will be largely Gaussian, and the model should give reliable predictions over the frequency band of interest.

Other applications of the pilot model to realistic tasks involving path control (e.g., landing approach, hover) have also found model predictions to be optimistic compared to actual pilot performance. Smit and Wewerinke [131 were able to improve the correspondence between model and data by assigning "indifference thresholds", equal to l/6 the maximum acceptable excursions, to various display quantities. The rationale was that the pilot's use of cues perceptual was not limited by visual resolution limitations, but rather by the pilot's unwillingness to reduce errors below what is considered good performance.

The notion of indifference thresholds as the limiting factor does not seem reasonable for the data obtained in this study, especially for the more difficult configurations. For example, height and pitch error scores for Configuration 3 were, about 65 feet (more than l/2 "dot" GS error at an respectively, altitude of 500 feet) and 2.8 degrees. These excursions would appear to be sufficient to avoid complacency. (The subjective impression of both the pilots and the experimenter was that the pilots had to work very hard at this task.)

We cannot rule out entirely the influence of perceptual limitations, however. The values used for visual "thresholds" were based on data obtained in well-controlled laboratory studies using highly trained subjects performing a single-variable, wide-band tracking task. It is possible that effective visual thresholds are greater for the types of instrument cues and signal bandwidths obtained in a more realistic flight situation.

In summary, further methodological development appears necessary to allow more accurate predictions of absolute performance in realistic, high-order flight-control tasks.

Nevertheless, accurate predictions of performance trends across task conditions can be achieved. Given the current state of the art, the analytic scheme is recommended for use in obtaining comparative, rather than absolute, handling qualities predictions.

5. CONCLUDING REMARKS The handling qualities prediction scheme explored in this study has been shown to be a good predictor of handling qualities tr,ends across a variety of longitudinal-axis vehicle response characteristics in the landing approach task. The model correctly predicted the trends of Cooper-Harper pilot ratings, scalar performance indices, error scores for important system response variables, and subjective effort rating scores. Absolute scores, on the other hand, were less well predicted, with experimental error scores being substantially larger than predicted. The prediction scheme is therefore recommended as a tool for comparing handling qualities across candidate configurations.

Because the prediction scheme is based on an analysis of the we feel that it has good task structure and task requirements, potential for identifying handling qualities problems across a wide variety of aircraft configurations, for tasks in which the scheme has been calibrated. At present, calibration is limited to longitudinal-axis control in landing approach.

We conclude with suggestions in the following areas: (1) application of the existing methodology, (2) calibration for additional tasks, and (3) further methodological development.

5.1 Application of the Prediction Scheme We recommend that the analytic prediction scheme be applied largely as demonstrated in this report, with the addition of a calibration phase. Application to the study of longitudinal-axis handling qualities in final approach should proceed as follows: 1.

Obtain linearized mathematical models for the various candidate configurations of interest. These models should include not but also the only basic vehicle response characteristics, dynamical response characteristics of the flight control system, SAS, display dynamics, and any other subsystem that might effect closed-loop control.

2. Analyze the display environment to determine the (1) potentially useful cues available to the pilot for flight control, and. (2) the effective perceptual thresholds to be associated with each potential cue. If thresholds cannot be readily determined, then eliminate from the cue set those variables for which the pilot is expected to have difficulty in obtaining useful information (e.g., useful rate information cannot be provided by the motion of glideslope and localizer needles).

3. Assume parameters for the Dryden gust model appropriate to (a) an altitude of 500 feet, (b) the nominal approach speed, and (c) a 50% probability of occurrence.

Select weighting coefficients for the scalar performance index 4.

based on the following maximum acceptable values: (a) 1 dot glideslope error, (b) 7.5 ft/sec sinkrate error, (c) 3.5 degrees pitch excursion, (d) 7.5 knots airspeed, (e) stick excursion and rate 25% of physical limits, (f) thrust excursion of 4% of aircraft weight, and (g) thrust rate-of-change equal to 0.2 of limit on thrust.

5. Perform preliminary model analysis to determine optimal allocation of attention. This procedure will identify potentially redundant display variables to be eliminated from the task description, and it will allocate attention properly among the useful perceptual cues.

6. Obtain performance/workload tradeoffs for two "calibration" vehicles, one of which is known to have good (Level 1) handling qualities, and one of which is known to have poor (preferably If data for such vehicles are not Level 3) handling qualities.

extant, a handling qualities simulation experiment may have to be performed, using the same gust environment and performance requirements adopted for the model analysis.

7. Obtain performance/workload tradeoffs for the candidate configurations by varying the baseline observation noise/signal ratio from 20 dB (relative attention of unity) to -10 dB (relative attention of 10%). Repeat the analysis with the throttle omitted from the control set to identify potential low-frequency path-control problems.

8.

Determine relative handling characteristics of the candidate configurations comparison with the predicted by performance/workload curves of the two calibration configurations.

We have omitted the final step of the analysis procedure as originally proposed; namely, the prediction of Cooper-Harper rating via the rating prediction equation of Equation 1. Because we are suggesting that the scheme be used to provide comparisons, rather than absolute predictions, it is not clear that carrying out this additional step yields useful information beyond that contained in the predicted performance/workload tradeoff curves.

5.2 Calibration of the Methodology for Other Tasks Discussion of the analytic prediction scheme has been limited to longitudi,nal-axis control in landing approach, because that is There is no the task for which validating data have been obtained.

theoretical reason the method should be limited to why longitudinal-axis control, nor to the landing approach task. We feel that the success obtained so far with the method warrants testing in other task situations.

Alog,icalcandidate for application is the lateral-directional axis of control in landing approach.

The analytic methodology is potentially of greater use in this application, as vehicle-centered handling qualities parameters have been less well defined than for longitudinal-axis control.

Calibration of the methodology should not require a structural modification of the analysis scheme. That is, one should be able to follow the procedure outlined in Section 5.1, using existing computer programs. Rather, calibration should consist primarily of defining maximum excursions of system allowable values for variables important to lateral-directional control, again relying on a mixture of published criteria, pilot preferences, physical limitations, and engineering judgment.

5.3 Methodological Development Additional development is suggested to improve predictions of absolute performance levels in realistic flight tasks requiring path control. Further exploration of perceptual thresholds is in order to see whether or not a consistent treatment can be found to explain the results obtained for the various configurations explored in this study. As noted above, considerations of the physical environment, plus limits on the degree to which pilots are willing to reduce errors, may lead to parameter values larger than those usually found in laboratory experiments with wide-band tracking tasks.

The presence of apparent nonlinearities in the pilot's control response suggests additional analysis to determine whether or not this type of behavior can be considered responsible for the discrepancy between prediction and experiment. Spectral analysis of allow the pilot's control response, for example, would comparison of measured "remnant" (non-input-correlated response power) with that predicted by the model. Excessive remnant would suggest readjustment of the noise parameters (observation noise, motor noise, or both) to reflect nonlinear (or intermittent) response behavior.

Appendix A APPENDIX A EQUATIONS OF MOTION motion in Linearized, small perturbation equations of stability axes were used in this study. The equations and the notation follow the standard set by the Bureau of Aeronautics Handbooks and can be found in Reference 14, The work was done using dimensions of feet, seconds, and degrees. The longitudinal equations contain two additional degrees of freedom which represent The the two elastic modes of motion of Configurations 7 and 10.

equations are given below.

gcos Y, ; = Xu(u-ug) + xphg) + XJW-wg) + x 57.296 +x 6 +X. AiH +xAT AT y+ x;\y+x r12 + x* 1 r\2 n2 ii = Z;(,lxg, + z u(u-ug) + z ,(w-wg) + (,:;96 4 zq)q + (zg - ;;;;;;)' + z + z- ;I1 + z,.,2~2 + Z,!,2t2 + Z4,e6e + ZiHAiH + ZATAT n1 n1 n1 4 = M,(u-ug) + M$P;~) + Mw(w-wg) + MS q +M ql+M*; + M* ;I + M 6 + M. AiH + M +M ATAT n 712n2 rl.2 2 'e e lH Y 1 1 ..

= A$.'-bg) + Aw(w-wg) + Aq q % + %lql + A;,;1 + %2r12 + A;,;2 + Adese + AiGAiH ..

= B (&kg) + Bw(W-Wg) + B ‘12 6: qq +j+,q +B;T; +Brl~ +B;I;1 +B66e+B.-AiH =H 11 11 22 e Appendix A The unusual derivatives, and have values only for ze ' xe Configurations 6,7,9, and 10, the elastic airplanes. They are artifacts of the way the elastic equations of motion parameters were computed.

The lateral-directional equations of motion contain only the three rigid body degrees of freedom.

U .

v = yp-vg) + Yp(P-Pg) + .

Ti%E@+y r- r 57.;96 r +y6ada +y6r6r h= Lv (v-v,) + Lp (P-P,) + Lrr -6 + L6a a + Nrr ; = Nv(v-vg) + Np (P-P,) + N6asa +N 6,” The transformations to Euler angles are given below. The equation for 8 is linear and decoupled to be consistent with the pilot modeling work, while the others are nonlinear for realism.

i=q 6 = p + q sin$tanf3 + r cos$tan0 $ = (q sin@ + r cos~$)/cos0 The engine model used was that of a CF6-50A, the engine used in the DC-lo-lo.

To account for differences in airplane weight, levels were factor of 1.286 for the thrust scaled by a Configurations 4 and 5, and a factor of 2.143 for Configurations 6,7,9 and 10. The complex dynamics of engine thrust response to power lever movements were represented by a simple lag as shown below. This approximation is adequate for the purpose of this experiment.

2.5 AL= s+2.5 ATC Appendix A The responses of control surfaces to pilot force inputs are _ functions of the feel systems, which were physically present in the simulator, and the actuators, which were simulated.

Transfer functions for the combination of feel systems and actuators are given below.

(850) (5O)2

Ke

&e -= = 0.26 %- F Ke .

cc (s2+45s+850) (S+50)2 .'a s (lo)(50)2 deg.

-= = 6.6 r KW .

Fw (s+lO) (s+50)2 KR(lO) (5U2 &R deg.

PC = 0.164 Ib.

KR FRP (s+lO) (s+50)2 Appendix B APPENDIX B CHARACTERISTICS OF THE CONFIGURATIONS The longitudinal dimensional stability derivatives of the ten Table configurations studied are given in Tables Bl and B2 below.

Bl contains the usual "rigid-body" derivatives, while Table B2 contains the terms due to the elastic degrees of freedom.

These data are the coefficients in the equations of motions of Appendix A. There are ten values (ten columns) in Table Bl for There is each derivative: these define the ten configurations.

only one value for each derivative in Table B2; the same values for the elastic terms are used in both Configurations 7 and 10.

The other configurations have no values for these terms: they Table B2 is read should be set to zero in the equations of motion.

The as follows. The rows are the degrees of freedom (X,.Z,M,A,B).

For columns are the variables of differentiation (~1.~2 ,etc..).

Its example, the variable in row 3(M) and column 4(n2) is Mn .

value is -0.07588 and its units are degrees per second. 2 Configuration 5 is an augmented variation of configuration 4.

which was designed using implicit model The augmentation system, The following and linear optimal control, is shown in Figure Bl.

configurations which explore the effects of structural modes on flying qualities (6 vs. 7 and 9 vs.. 10) were developed as follows.

Let the elastic airplane be represented by a high order, state The model or its equivalent in the frequency domain.

space they vary in elements of this "state" matrix are not constants: A linear low order some unspecified fashion with frequency.

nonlinear system is proposed approximation to this high order, The which has three "rigid" and six elastic degrees of freedom.

low order state matrix has a term for each degree of freedom and its first two derivatives. The elements of the approximate system the fit for That is, are computed using a sliding frequency fit.

each element is done at a frequency appropriate to that element.

The low order systems developed by this procedure duplicated th,e dynamics of the high order systems with vanishingly small error from static conditions (w=O) up to frequencies beyond the highest natural frequency in the low order system. It should be noted that the\ coefficients'of the so-called rigid degrees of freedom contain the low-frequency effects of elasticity.

The two mode (7 and 10) and zero mode (6 and 9) configurations six mode are simply lower order approximations to the The simplest way to do order reduction is modal configuration.

Table Bl.

Longitudinal Dimensional Stability Derivatives for Rigid Degrees of Freedom DERIVATIVE UNITS 1 2 3 4 3 6 7 8 9 10 -___ -- -- l/SSC -.05221 -.05221 -.05332 -.05254 -.05254 -.002838 -.002838 -.05327 x -.02129 -.02838 u l/set .05479 -.2055 .05712 x -.04897 -.04897 .005731 .005731 .05506 .005731 .005731 w 1 .O .O .O X. .O .O -.03193 -.03193 .O -.03193 -.03193 w ft/sec.2deg.

.O .O .O .O .O .0129 .0129 .O .0129 .0129 % ft/sec.deg. .O .O .O .O .O -5.821 -5.821 .O -.05821 -.05821 X9 ft/sec.2deg. .O .O -.07566 .O .O .O .O .O -.07566 .O x&e x. ft/sec. deg. .O .O .O -.07566 -.07566 .O .O .O .O .O 'h l/slug -5 4.29x10 -5 4.29x1o-5 9.14x1o-5 9.14x1o-5 9.14x1o-5 7.15x1o-5 7.15x1o-5 4.29x10 4.29x1o-5 9.14x1o-5 xAT -.05489 z l/set -.2533 -.2533 -.2610 -. 2695 -.2695 -.05489 -.05489 -.2607 -.05489 " 2. 1 .O .O .O .O -.03256 -.03256 .O -.03256 ' -.03256 .O " l/set -.5818 -.5818 -.5664 -.4041 -.4041 -.4129 -.5802 -.4129 -.4129 -.4129 =w ?ii .O 2. 1 .O .O .O -.00031 -.00031 .o .O .O .O w P* .O .O .O .O -.15059 -.15059 .O ft/sec.2deg. .O -.15059 -.15059 x I..508 1.508 -.2551 -.2350 -.3321 -.3321 1.508 1.508 -.2417 ft/sec.deg. -.2551 td ft/sec.2deg.

-.2403 -.2403 -.2403 -.1804 -.1804 -.1229 -.1229 -.2403 -.1229 -.1229 -.4121 -.4121 ft/sec.2deg. -.4121 -.1804 -.1804 -.2458 -.2458 -.4121 -.2458 -.2458 -6 -6 -6 -6 2.5x10+ 2.5x10+ 1.5x10+ l/slug 1.5x10 -2.29x10 -2.29x10 -2.29x10+ -2.29x10 1.5x10+ 1.5x10-6 'AT li -.001127 -.001127 .O deg/ft.sec. .02136 .02136 .000207 .O .000965 .O .O u .00172 .00172 -.025 deg/ft.sec. -.1237 -.1237 .09225 -.025 .02194 -.025 -.025 M. deg/ft -.i)OO5!.04 -.0005104 -.1308 -.01471 -.01471 -.01056 -.1308 -.01176 -.1308 -.1308 w M -.3109 -.3109 -.5589 l/set -.3989 -.3989 -.2475 -.5589 -. 2960 -. 5589 -. 5589 q -.2899 -.2899 l/sec2 -.6442 -.5275 -1.208 -.6442 -1.208 -.5664 -1.208 -1.208 M6e M. l/sec2 -1.039 -.2899 -. 2899 -2.416 -1.039 -1.039 -2.416 -1.039 -2.416 -2.416 =H deg/s.ft. 1.o1x1o-6 1.o1x1o-6 1.o1x1o-6 2.7~10-~ 2.7x10+ 1.6xlO+j 1.6xlO+j 1.o1x1o-6 1.6x10+ 1.6~10-~ "AT Appendix B Table B2.

Longitudinal Dimensional Derivatives for Elastic Degree of Freedom .- .

w '12 ‘e =H I *

* * * l

ft/sec

-.0007971 I

l * * * ft/sec .01529 -~_

I

* deq/sec -.07588 --- ~-- - ~._. ..___ l/WC l/ft.sec -.06763 -. 3922 l/set l/ft.sec l/ft. l/deg.sec.

-1.233 2.581 -.02344 -.4458 -- ___- * See Table Bl + dH, i A$ dtrim d cd NON-LINEAR EOUATIONS

q

=I

Appendix B the equations which define the truncation: one simply deletes modes being dropped. This was tried and found to work in the present case. When the higher order modes were dropped, the response of the system changed relatively little across the frequency range of interest.

The dimensional stability derivatives for the lateral-directional rigid-body degrees of freedom are given in Table B3. There are no additional derivatives in the lateral-directional equations due to elastic degrees of freedom.

This table is read as was Table B.2. the variable in -for example, row 2, column 3, is Lr=0.2313 second .

Table B3. Lateral-Directional Dimensional Stability Derivatives V r P 'r 'a - Y units l/set.

ft/sec.deg. ft/sec.deg.

ft/sec. deg. ft/sec2deg value -.1132 . 5130 -4.018 . 1102 -. 00985 - L units deg/ft.sec. l/set. l/set.

l/set. l/sec2 value -0.3101 -1.034 . 2313 .2434 . 2696 - N units deg/ft.sec. l/set.

l/set. l/set. l/sec2 value .04724 -.1517 -0.1636 -.2300 . 04087 - A number of longitudinal flying qualities parameters were computed to permit estimation of flying qualities by criteria other than the optimal control pilot model. The results are shown in Table B4. Note that there is data for the two structural degrees of freedom for the configurations 7 and 10 only. The data are mostly frequencies, damping ratios, and steady state response ratios which can be compared to the criteria of Reference 1. The last two rows, however, are the criteria parameters for the Bandwidth Model (Ref. 7). This is a frequency-domain, pilot-model-in-the-loop criterion. It estimates pilot opinion of Appendix B Longitudinal Flying Qualities Parameters Table B4.

CONFIGURATION DERIVATIVE UNITS 1 2 3 4' 5 6 7 8 9 10 -P--------P rad/sec 0.846 w 0.811 (-1.061) (-1.057) 0.720 (0.612) (0.706) (-0.811) (0.604) (0.707) "SP 0.628 0.662 (+0.291) (+0.285) 0.716 !1.40) (1.21). (+0.090) (1.42) ( 1.21) SP wn rad/sec 0.186 0.194 0.210 0.204 0.162 0.045 0.053 0.200 0.043 0.057 ph ‘ph 0.074 0.041 0.331 0.392 0.902 0.213 0.206 0.636 0.188 0.170 l/T rad/sec -0.084 +0.041 -0.082 -0.0092 -0.229 -0.004 -0.004 -0.082 -0.022 -0.029 e1 rad/sec -0.506 -0.631 -0.583 -0.498 -0.497 -0.414 -0.410 -0.564 -0.414 -0.414 l/T4 3.06 n/a g/rad 3.80 4.04 4.35 3.81 3.80 4.36 4.32 4.20 3.08 deg/Kt -0.041 dy/dV -0.040 +o. 339 -0.057 +0.192 -6.512 0.037 0.037 -0.051 -0.018 lb/kt -0.030 -0.058 AFs/AV -1.52 -1.49 -0.896 -0.896 -5.868 -0.132 0.065 0.189 AFs/An 112.4 112.4 37.2 -22.1 60.4 29.0 40.5 14.1 29.0 40.5 lb/g rad/sec 6.04 5.98 w1 0.030 0.021 rad/sec 10.76 10.68 - 2 0.057 0.050 1W3,1 dB -0.3 -0.4 4.9 3.9 -1.25 0.6 0.0 2.1 0.9 0.3 <WC de?! 40.5 41.6 JO.5 62.5 41.5 31.0 28.2 62.3 32.2 29.6 Note: ( ) indicates a root for a first-order mode 4 5 Appendix C APPENDIX C INSTRUMENTPANEL The instrument layout-of the simul,ator cab is shown in Figure Cl. The pilot flies by reference to instruments located directly These are referred to as in front and just to the right of him.

instrument panel and the center main instrument the captain's These panels are shown in more detail in Figures C2 and C3.

panel.

The primary instruments used in performing the longitudinal portion of the task are the airspeed, pitch, glideslope, altitude and rate of climb indicators on the captain's panel, and the Nl (reference RPM) indicators on the center main instrument panel.

..-...-- , , i / )MPARlMENT ACCESS I i \ OBitRVER : - - _. _. ^ . I Layout of Motion Base Simulator Cockpit Figure Cl.

A: Captain's Instrument Panel B: Center Main Instrument Panel Appendix C Figure C2. Captain's Main Instrument Panel Appendix C ---------- --_ -- _ Figure C3. Center Main Instrument Panel Appendix D APPENDIX D TURBULENCE MODELING Zero-mean random turbulence of the Dryden Form [ll was used as a basis both for model analysis and for the manned simulation.

Only four components, u ,v ,w and p were considered; the . .

remaining components, q a4 d 9 ,%ere conadered to be unimportant.

Continuous spectra, cor%espon%ng to an altitude of 500 ft and an airspeed of 140 kts, were of the form 0.643 CT

ug

-= s+O.207 ug NU 0.788(s+O.119 ~

“=

V (~+0.207)~ ?I 1.19(s+O.275) 0

“=

w4 W (~+0.476)~ 1.46 o

Pg=

N s+1.06 P pg where N ,N ,N , having unit and N are white noise processes covarian%e.v yhe simul%tion experiment was performed using the following values for rms gust levels: cl =5 ft/sec ug =3.3 ft/sec aVg =3.3 ft/sec wg =0.58 deg/sec pg It was desired to conduct the simulation in such a manner as to enhance the ability and pilot to estimate power spectra Appendix D describing functions, even though there were no plans to obtain such measures during the course of the present study. Accordingly, the analytical models shown above were used as the basis for constructing sum-of-sinusoids inputs for the four gust components simulated in this study. By confining input power to a few frequencies, rather than spreading the power continuously over a wide. band, sum-of-sines format enables one to readily the distinguish input-correlated from "remnant"-related signal power, the signal-to-noise environment for estimating and it improves pilot describing functions [15]. Derivatives of the turbulence states, which were needed by the simulation program, were calculated via differentiation of the sum-of-sines inputs.

Each gust component was therefore of the form C au cos(ki~ot+~u ) ug (t) =ou gi=l i i The fundamental frequency w was computed as ~IT/T where the T =73 fC?i measurement interval seconds. Values component amplitudes are given in Tale Dl. The initial phase offsets #were chosen randomly from a uniform distribution between 0 and HIT; phase offsets were randomized from run-to-run and from component-to-component ih order togenerate random-appearing inputs having a similar statistic but different time histories.

Table Dl. Turbulence Model Parameters INDEX i VARIABLE UNITS 1 2 3 4 5 6 7 8 9 10 11 12 1 2 5 7 11 17 23 31 47 67 97 131 Ki a .786 .720 .598 .402 .358 -207 .207 -192 ,166 .134 .108 .0872 Y a .601 .689 .662 .472 .428 .326 .251 .233 .202 -163 .131 -106 "i a .309 -474 -589 .532 .555 .459 .367 .348 -304 .246 .200 .161 "i .368 .425 .511 -478 .543 .490 .414 .407 .365 .299 .244 .198 =Pi ._~_-. _ Appendix E APPENDIX E QUESTIONNAIRES USED A number of questionnaires and forms were used in the subject programs. The first briefing with.pilots was held prior to the motion base simulator experiment. An extensive written description of the purposes and methodology of the study was given to the pilots to read prior to the meeting.

This was supplemented by an oral briefing at the meeting, followed by a.question and answer period. At this point, the pilots knew enough about the program to be able (and motivated) to supply quantitative estimates of tracking performance and workload expectations. These data, which are reported in the Task Definition section of this report, were used in the pre-experiment model analysis. Five pilots, in individual interviews, gave quantitative data on the tracking performance and workload levels expected. Specifically, the pilots were asked for their subjective impressions of maximum acceptable excursions for important system variables in moderate turbulence.

Values supplied by the pilots were averaged and then used to set the weighting coefficients in the performance index of the optimal control model and as performance standards for the pilots.

After the experiment was finished, the pilots were asked to fill out the questionnaire which is shown as Table El below.

This was done for two reasons. First, we felt that they could do a better job of giving quantitative answers after doing a test in which they were constantly reminded to perform to a consistent performance standard. Second, we wanted them to quantify the standards to which they actually worked.

Appendix E Table El. Pilot Debriefing Form You have just completed a simulator study of transport longitudinal-axis handling qualities.

You are now asked to fill out this form to,provide the experimenters with certain informa- tion relating to your piloting strategy and your formulation of Cobper-Harper ratings.

Attention Allocation 1.

Check below the relative attentions you paid to the various flight instruments during the runs with turbulence.

If you used different scan strategies for different configurations, indicate the strategy most commonly employed during the study. Indicate separately for each of the three flight subphases.

(Instruments are listed below alphabetically, not by implied order of importance.)

Relative Attention: H=high, M=moderate, L=little or none Altitude Glideslope Glideslope Instrument Capture Tracking Stationkeeping H M L H M L H M L Altimeter - - - Attitude-Director Indicator Artificial Horizon - _ - Glideslope - - - - - Localizer - - - - - - Engine (Nl) - - - - Horizontal Situation Indicator Compass - - - Glideslope - - - - - - Localizer - - - - - - Indicated Airspeed - - - - Vertical Speed - - - Appendix E Pilot Debriefing Form (cont.)

Table El.

Was your attention strategy for any configuration significantly different from that shown above?

Yes No . If so, state which configuration(s), and indicate how the attention strategy differed.

2. Relative Importance of Subphases Indicate below the rank ordering of the three flight subphases in terms of their importance to your longitudinal-axis Cooper-Harper rating. Give the rank ordering that applies to most of the configura- tions. (1 = most important, 3 = least important.)

Altitude Stationkeeping Glideslope Capture Glideslope Tracking Would you order the subphases differently for any configurations?

state which configuration(s), and which Yes No . If so, order.

3. Performance Requirements The Cooper-Harper rating scale is defined in terms of performance For example, a rating of 6 and workload (i.e., "compensation").

"Adequate performance requires extensive is defined by the statement pilot compensation". The objective of this question is to find out what you considered to be "adequate performance" when you assigned Of interest C-H ratings to the various experimental configurations.

Appendix E Pilot Debriefing Form (concl.)

Table El.

here are ratings assigned to longitudinal-axis handling qualities in turbulence. Because the subsequent modeling effort will concentrate on the glide-slope tracking phase, please state performance requirements for that task.

One way to state a performance requirement is to indicate upper and lower bounds, in terms of deviation from trim, and to state the percentage time you feel the airplane should be between these bounds. For example, if we were interested in lateral-axis performance, one might state a lateral-path requirement as being within plus or minus one dot on the localizer 90% of the time.

In general, performance requirements should be specified for attitude, and control variables.

path, Please indicate your definitions of "adequate performance" for as many of the following variables as you can. Indicate NA for any variable for which a performance, requirement is not applicable or for which you cannot give a quantitative answer. Please specify upper and lower bounds in terms of deviation from trim.

GLIDE SLOPE TRACKING Variable Lower Percent Time Upper Bound Bound Within Bounds Glide slope (dots) - - - Sinkrate (ft/min) - - - Airspeed (kts) - - - Pitch (degrees) - - - Pitch rate (deg/sec) - - - Column (fraction of % % available control authority) - - - Throttle (%Nl) (Assume trim _ is 75%) ii’ e Appendix F APPENDIX F DATA RECORDED A great deal of subjective and objective data were recorded during the MBS experiment for analysis in this program and in the This appendix lists the data recorded. The analysis done future.

on this data is described elsewhere.

Subjective data were recorded in three formats: pilot comments, pilot ratings, and cockpit voice recordings. The pilot comments were recorded by the test engineer during and following each run on the pilot Comment Card, Figure Fl. The pilots were encouraged to keep up a running commentary of what they were doing, how the airplane was responding, and their reactions to or impressions of the airplane, the task, and their performance. At the end of each run, the engineer went down the list of eight topics, prompting the pilot for comments and making hand-written notes of the responses. The pilots were then asked to select a Cooper-Harper pilot rating for the longitudinal degrees of freedom and another for the lateral-directional by use of the Cooper-Harper Scale (Figure F2). These numbers were recorded by the engineer on the Pilot Rating Form, Figure F3. The pilot was.then asked to rate the mental effort required to perform each of the three subphases and each of three specific control tasks. The pilots did not verbalize responses: they made marks on the eleven point scales on Figure F2 to indicate the required effort. The final type of subjective data recording was made by the cockpit voice recorder.

This was not a true cockpit voice recorder, as it was wired into the intercom and thus recorded the sounds in the cockpit plus the motion base voices of the other personnel (computer operator, operator, etc). This was not a problem, as these individuals were only involved in the test when starting and stopping a session or when a problem occurred.

A great deal of objective data were recorded during every run.

Fifty channels of time history data were recorded on digital magnetic tape. This was generally done at 5 hertz, with occasional The fifty runs recorded at 20 hertz, the simulation update rate.

variables recorded, the units in which they were recorded, and the word number for each variable in the 50-word block are given in Table Fl. The format for the remaining objective data, recorded is shown in Figure F4. It consists of two pages of for every run, information computed or stored during each run, then output by the MBS Computer on its line printer. There are three kinds of data: run documentation, instantaneous, and statistical. The first block the page documents the run number, of data at the top of configuration number, pilot number, time and date, and run starting or reference information. The second block records instantaneous values of nine variables at ten points along the approach. The values are recorded at the instant the airplane passes through the ten range values in the third column. The first two columns show glideslope and actual altitude at the given range. Glideslope altitude at a given range is always the same: it does not vary The actual altitude does vary, as do the other from run to run.

eight variables (glideslope deviation, localizer deviation, theta, phi, psi, airspeed, wheel force, and column force). The data labeled "Freeze Point" is meaningless. In prior tests, instantaneous values at touchdown were printed here. In this test, however, there is no touchdown, so there are no data.

The remainder of the page lists statistical data for the following three measurement segments: (1) altitude tracking prior to glideslope capture, (2) glideslope tracking between altitudes of 1200 and 700 feet, and (3) glideslope tracking between altitudes of 700 and 200 feet. The five data columns show mean, root mean square, maximum, minimum, and standard deviation for 15 variables.

The data from segment 3 were used in validation of the model.

Appendix F PILOT COMMENT CARD OCPM MBS 81 Please comment on the following items. PILOT CONF# Attention: Indicate number of each item. DATE RUN# 1. Controller force- and displacement characteristics; control harmony 2. Tendency toward pilot induced oscillation 3. Response to turbulence 4. What is the most positive feature of the configuration 5. What is the most objectionable feature of the configuration 6. Ability to control airspeed Ability to control flight path 7.

8. General comments Pilot Comment Card Figure Fl.

Cooper-Harper Pilot Rating Scale Figure F2.

HANDLING QUALITIES RATING SCALE

DEtiNDS 6N THE PILOT Minor but annoying Desired performance requites moderate‘ deficiencies pilot compensation Moderately objectionable Adequate performance requires deficiencies considerable pilot comoensatior?

Very objectionable but Adequate performance requires tolerable deficiencies pilot compensation Adequate performance not attainable maximum tolerable pilot compensation.

Major deficiencies Controllability not in question Considerable pilot compensation Major deficiencies for control Intense pilot compensation is required Major deficiencies retain control Cqntrol will be lost during some Major deficiencies reauired operation )F Definition of required operation involves Pilot decisions I I Cooper-Harper Ref. NASA TND-5153 subphases with accompanying conditions.

Appendix F PILOT RATINGFORM CONFIGf PILOT RUN# DATE Longitudinal Cooper-Harper Hand1 ing Qua1 i ty Rating Lateral-Directional Please indicate level of mental effort and concentration for the following items: Extreme Negligible Mental Mental Effort Effort Specific Flight Phases 1. Altitude stationkeeping !Li 1 1 1 J ’ ’ ’ ’ J 0 5 10 2. Glideslope Capture 1 I 1 1 ' 1 1 ' ' ' J 0 10 Glideslope Tracking 3.

11' l'l""l 0 5 10 Specific Control Activities Vertical Path Control 1.

2. Airspeed Control Pitch Control 3.

Pilot Rating Form Figure F3; Appendix F Table Fl. Magnetic Tape Parameter List -- ---_ Word DESCRIPTION UNITS # ,l Frame Count (Des) 2 e Euler Angles (Deg) Commanded Cab Pilot 3 9 4 * v 2g azCMD ' aZ a i Structural Mode Coords -____.-.- .--. -- -_._ 9 s - 41 Center Throttle Lever 16 r Deflection t / Wheel Deflection Stab Axis Gust Velocites (Ft/sec) 42 (Deg) , 17 % ._ _-i~~-_~- ia vg 43 Rudder Pedal Deflection (in) 44 6= Elevator Position (D=g) "9 (Deg/sec) 45 b Elevator Rate (Deg/sec) e pg -- Inertial Wind Angles 46 AT Thrust Incr. (lb) 21 a (Des) (Volts) 47 LOC Error (.075 volts=ldot) 22 6 Flight Path Angle Stabilizer Position 48 iH (Deg) 23 Y + -_-_ Aileron Position Airplane Pilot Station 24 a XPS Accel .

Rudder Position aYPs -- ‘ Appendix F BCPM n0s a1 STABILITY AhD CehTReL TECHkOLGGY DBUGLAS AIRCRAFT CaRPANY DATES lll29 MAR 24r’a1 RUN rvUl 138 CBNFIG* 9 PILBT#l 1 V(HEF): 14C KT ALPHA (REF): 6 DEQ THETA (REF) 8 613 DEG STARTIhG PEtIhil XI n -SCCOC, FT YI 9 Ca FT ZI c -1SGCa FT TURBULEFiCt SIGw m 383 FT/SEC SIGL . 5-C FT/SEC ‘T THETA AIRCRAF G/S LGC PHI PSI AIR WHEEL C0LURN T ALTI GS AL HANGE DEV DEV SPEED FORCE FeRCE (FT) (DOTS) (DLG) t (DtG) (KTI (LB) (LB1 (FT) (FT) (DOTS) 92 794 15240 5r2 *l 14c92 -2390 iacc -34335r 113 1463* l Y -1.3 13j.3 -23*8 -3*6 165C -31478, *l 5.t 1436* 135t7 -2398 -2r5 15cc l 28613* r3 -ru 6t3 135c -25755s -92 l 23ra 138-7 110 147.2 -12rP 12cc -22897 a r2 l 23ra 13214 -597 see -17172e -91 -9*2 *3 142-6 tee -11442* -1593 14515 -23r8 -1*c -16r 1 l 5mc 191-l -23e8 269. l 3a0.7 I -m6 95 2e6 -6*6 491 ECC r6 134rC -2318 FREEZE P’BINT THE n .C PHI n VU PSl l l C (DEG) vx I rc VY . 8CVZ n l C lFPS EARTH) AX . rC AY L rGAZ 8 *C (FT/SECZ) AVER THE 1 RANGE SEGMEhT: DATE I 11129 WAH 24r'al RUN her 13a 1500 SAHPLES MEAh RAX ST DkV- HP’S MIN 2.824 AIRSPEtC (KT) 139* 12C 139*149 144971c 13C*314 -4LC25-93C; ~31(;14@695 4C3599P52 l -489948352 5181.935 RANGE (FT) -239434 4C*ri63 32,$95 VERT. UFFSLT (FT) rsr070 -62,993 -m4za 31746 7.945 -9*941 3,722 SINK RATS IFT/SEC 1 58952 48603 ,666 THtTA 1Dk.G) 5r915 79189 r23C -es50 PITCH RAlE (UEG/SECl .ac -*Cl2 1*041 m42c -2r717 ,425 ELEVATUR IDEG) *ccc 29567 ELtVATGR RATk (D/S) 11775 -11*415 '1.775 *ccc llD448 CeLUMh FURLE (I.61 22.778 7*592 l 2la731 7.4?8 1.G 599c4*375 52326rQ59 i?WP.PbS THRUST 616819223 59a459 77.3 (LB) Figure F4.

MBS Line Printer Page Appendix F V GUST (FT/SEC J @CCC 4,999 Elm320 -14*4x 4,999 w GUST IFT/SECJ *CCC -12*134 3#3VC r2g24a P GUST (DtG/sEC J 8CCC 1577 BVER THE 2 RAtthE SEGMEhrT: DATE I ii:29 MAR 24rf81 RUN NBe 138 75~ sAnpLf2 ST DEV MEAh -- RMS PAX MIk AIRSPEEc (KT J 139m422 139*5G5 i49aia2 127eaa6 4*ai8 .

~16491*535 -229941727 HANGE (FTI 186688488 8ircac~i09 2564,306 VERT. UFFS~T (FT) 27rC41 3CIL;lE 599882 -8r625 22,246 SINK RAft (FT/SEC J l 1EISC9 13eU66 l 5*492 -189085 4rL;YP 4e485 THETA (DEGJ 3r429 3*49C 1,965 ,650 8C54 PITCH RAlE (DEG/SECJ ,293 *a59 -1.054 ,288 -1,732 ELtVATtJR IDEGJ --ii8 r493 1*686 n479 ELEVATUR RATE (O/S) *GE1 117cs 7t702 l 7r354 1.798 CBLuMh FURLE (LB) -Seaal 89985 2c872a -199134 a.364 42346.836 42536,633 48725.863 379g3*273 4013,632 THRUST (LB) THRUST RATt (L’j/SECJ 348566 1c49a434 5246,734 939169384 1048r665 U GUST (FT/SEC J *CCC Crl34 EC*141 -1Ce426 5,134 V GUST IFT/SEC J accc 5*135 21~320 -119020 5,135 (FT/SECJ *CCC 3.3713 15.262 -a*492 3.37G w GUST P GUST . ccc * $89 21739 -1*481 (CtG/SEC 1 a309 t3VEH THE 3 RAhibE SEGMEhiT: DATE : 11:29 MAR 24r’81 RUN he* 138 750 smus WEAh nAX RMS MIh ST DEV AIRSPEtE (KTJ 141,395 14ll421 146,564 336*343 Ee723 RANGE (FTJ -9C13*34G 9376,504 -45779781 l 13498*652 25U4.294 VERT m UFFS!iT (FT J 7m785 27r aa7 52t6CI -229 107 26rC49 -1lr327 SIhK RATt (FT/SEC) 12,224 -3a9a.s l 2la531 4a596 3*5a7 THETA (DEGJ 3*777 99435 I@416 1.191 PITCH HA rE (DEG/SEC J 95C8 1,892 -1,674 *cc4 (598 kLEVATBR (DtGJ -9C37 ,843 29995 l 3e567 EiClVATW? RAlt-(D/S) .ICl 20531 8.585 -1le586 2@631 -9332 CeLUMh FURCE (LB) 9,785 2C.683 -261377 91780 THRUST 43175mE73 436790453 54C69r39U 37077r437 6617.465 (L0 J THRUST RATE (LU/SEc) l i37ra32 1C43r 257 2C24r027 -6657~355 SQJI,Cll IFT/SEC J *CCC 4@164 6*982 l 1EI941 u GUST 4.164 IFT/SECJ 48244 7e227 V GUST *CCC l 14@439 4e244 W GUST (FT/SEC J *ccc 3*(;19 6.067 -121134 3.9a9 P GUST (CkG/SEC J *ccc 9538 1~124 -2*248 ,638 (cont.)

Figure F4. MBS Line Printer Page Appendix G APPENDIX G TABULATION OF PERFORMAFKE MEASURES The following tables contain statistical analysis of pilot opinion ratings and objective performance measures. Average Cooper-Harper ratings for each pilot are given in Table Gl, along with means and standard deviations of the individual subject means.

Results are given for the longitudinal axis (turbulence on and off) and the lateral axis (turbulence on).

Results of t-tests performed on paired-difference pilot ratings are given in Table G2. With the longitudinal axis, turbulence-on condition taken as a baseline, Table G2a indicates the significance of differences associated with turbulence on or off, and G2b indicates the significance of longitudinal-lateral Entries in this table show the axis differences in pilot ratings.

probability that the measured differences can be explained by the null hypothesis (i.e., that apparent differences are due to randomness in the data and not to the experimental conditions).

Differences considered statistically insignificant for are probabilities greater than 0.05.

Tables G2c through G2e indicate the significance of rating differences across the vehicle configurations. SinceConfiguration 1 is taken as the baseline for all comparisons, these tables show no entry in column 1.

Table G3 contains average effort rating scores for each configuration. These scores were obtained from the effort rating form shown in Table F3.

for eight Table G4 contains means and variances recorded during the longitudinal-axis system variables turbulence-on trials. are shown for the three Results quasi-steady-state tracking segments: pre-capture stationkeeping, and post-capture glideslope tracking between 1200 and 700 feet and between 700 and 200 feet. (Measures from the third segment are discussed in the main text.)

First, a Analysis was performed for each variable as follows.

within-trial mean and standard deviation was computed from each experimental trial.

Within-pilot averaging was performed to yield an average mean score and an average standard-deviation score for Finally, across-pilot each pilot performing each condition.

analysis yielded means and standard deviations of these derived measures. Four measures were thereby obtained: Appendix G 1. the overall mean level of the variable; across-pilot standard deviation of the mean score 2.

(rnndicating the reliability of the population mean); 3. an average standard deviation score (indicating the average variational component for the four pilots); and 4 : an across-pilot standard deviation of the average within-pilot standard deviation scores (indicating the reliability of the averaged standard deviation score).

These four sets of statistical results are presented in order, from top to bottom, in the following tables. The latter two metrics are shown for selected variables in Figure 8 of the main text.

Appendkx G Table Gl. Average Cooper-Harper Pilot Opinion Ratings Configuration Number Pilot - I 1 I 2 13 18 1 9 1 10 Longitudinal Axis, Turbulence Off

a)

1 3.75 16.00 1

6.50 14.00 14.25 14.00

I 3.00 14.25

( 6.00 (3.50

12.75 13.00

I 2.25 I 4.50

I 5.75 I 3.88 I 3.50

12.75

4.63 ) 5.75 1

7.50 (5.00 4.50 5.75 Mean 3.41 5.13 6.44 '4.~0 3.75 3.88 Std. Dev.

1.02 0.88 0.77 0.64 0.79 1.36 b) Longitudinal Axis, Turbulence ON 1 5.80 8.40 7.00 4.20 5.00 5.05 4.00 6.00 7.00 4.88 3.75 3.50 3 4.25 6.63 6.75 5.38 4.00 3.75 5.60 8.00 8.00 5.70 5.00 5.00 Mean 4.91 7.26 7.19 5.04 4.44 4.31 Std. Dev. 0.92 1.13 0.55 0.65 0.66 0.80 cl Lateral Axis, Turbulence On 5.00 6.25 5.50 4.25 4.75 5.00 2 5.00 5.38 5.25 5.25 4.63 4.63 3 4.00 4.00 4.00 4.00 4.00 4.25 4 4.75 6.25 5.00 6.75 5.50 5.63 Mean 4.69 5.47 4.94 5.06 4.72 4.88 Std. Dev.

-47 1.06 0.66 1.25 0.67 0.59 Appendix G Tabll, 62. Analysis of Pilot Rating Scores: T-Tests of Paired ,Differences Configuration I 1 2 3 8 9 10 I Effect of Turbulence, Longitudinal Axis a) I <. 02 <. 001 c.02 NS c.02 NS I I Effect of Axis, Turbulence On b) NS <. 05 C.01 NS ' NS NS - Effect of Configuration, Long. Axis with Turb.

cl * c.001 c.02 NS <. 05 <. 01 I - Effect of Confiquration, Long. Axis without Turb.

d) * <. 02 Cool, NS NS t NS I I I I I I -I '- Effect of Configuration, Lateral Axis with Turb.

e)

*

NS NS ’ NS NS NS I I

I

not significant (a level of significance >.05) NS = * Reference condition.

Appendix G Table G3.

Averaged Effort Rating Scores Task 1 2 I 3 8' 9 10 - Rating by Subtask a) Altitude Stationkeeping 3.8 5.4 5.6 4.2 3.7 3.6 I Glideslope Capture 4.2 5.3 5.6 4.2 3.9 3.7 Glideslope Tracking 4.8 6.3 6.5 4.9 1 4.2 4.0 Rating by Control Requirement b) Vertical Path Control 4.5 6.8 ' 6.9 ' 4.8 4.5 4.4 Airspeed Control 4.3 7.0 5.4 4.6 4.5 4.4 Pitch Control 4.9 6.6 7.5 5.3 4.7 4.5 Average of 4 pilots, 4-5 trials/pilot Appendix G Table G4. Objective Performance Scores Height Error a) OCPM H&S81 TRACKING TASK OATA ENSEH8LE STATISTICS (STD. DEV. AVERAGING) 4-22-8 1 LEVEL 11: AVERAGES ACROSS PILOTS (ACROSS REPLICATIONS) VARIABLE 2: VERT OFFSET (FT) IXPIL) MEAN VERT OFFSET tFT) - TURBULENCE ON 1 2 CONFfGURATION 8 9 SEG:ENT -4.268 -3.296 17.131 -1.646 -3.612 -13.759 2 -35.b86 -73.392 6.155 -9.519 5.720 24.653 3 38.428 -73.441 5.900 17.674 7.959 7.106 (SGXPIL) STANDARD OEVIATIW OF XREP ACROSS PILOTS TWBULENCE ON (A FUNCTION OF XPIL AND XREP**Z) 1 2 CONFjGURATION 8 9 SEGTNT 28.030 54.019 19.530 11.759 7.947 23.479 2 17.086 56.154 lb.533 19.939 18.018 16.356 3 15.837 42.106 9.806 6.697 7.944 7.931 (SIGPILI AVERAGk STANDARD DEVIATION - TURBULENCE OH (AVERAGE OF SIGREP UALUES ACROSS PILOTS) 2 CONFJGURATION 8 9 SEGYENT 43.051 61.245 39.428 29.1% 28. b43 29.709 2 41.244 59.448 29.414 27.842 31.956 31.771 3 34.700 64.28 8 30.508 23.976 23.771 24.096 (SGSGPLB STANDARD DEVIATION OF SIGREP ACROSS PIUITS - TURBULENCE ON IA FUNCTION OF SIGPIA MD SIGREP+rZ) 2 CONF;GURATION 8 9 SEGfENT 8.939 17.151 5.413 6.187 6.766 5.907 2 16.241 11.851 4.211 lb.757 6.965 10.499 .3 8.867 18.866 14.173 3.917 8.321 6.367 Appendix G Table G4.

Objective Performance Scores (cont.)

Vertical Velocity b) OCPM H8S81 TRACKING TASK DATA ENSE?WLE STATISTICS (STD. DEV. AVERAGING) 4-22-81 LEVEL II: AVERAGES ACROSS PILOTS (ACROSS REPLICATIONS1 VARIABLE 3: SIMRATE IFT/SEC) (XPIL) MEAN SINKRATE IFT/SECl - TUIBULENCE ON 1 2 CONFJGURATION 8 9 10 SEG?ENT 0.051 -0.284 -0.614 0.049 -0.029 0.070 2 -15.306 -12.868 -12.647 -13.521 -13.204 -12.586 3 -12.b82 -12.311 -13.574 -12.459 -12.391 -12.209 ISGXPIL) STANDARD DEVIATION OF XREP ACROSS PILOTS - TURBULENCE ON (A FUNCTION OF XPIL AND XREP**Zb 1 2 CONF$GURATION 8 9 10 SEGFNT 0.823 1.227 0.695 0.420 0.387 0.454 2 1.059 1.859 0.814 1.193 1.536 0.947 3 0.690 4.358 1.201 0.943 0.867 0.414 (SIGPIL) AVERAGE STANDARD DEVIATION - TUlBULENCE ON LAVERAGE OF SIGREP VALUES ACROSS PILOTS) 1 2 wNFJGURAT*oN 8 9 10 SEG:ENT 5.518 6.556 8.242 5.125 4.330 3.171 2 5.539 7.816 7.813 5.123 5.064 5.223 3 6.127 9.685 9.581 6.567 5.599 5.980 (SGSGPL) STANDARD DEVIATION OF IGREP ACRO S PILDTS - TURBULENCE ON (A FUNCTIPI OF 5 GPIL AND S GREP-2) f 4 1 2 COWFJGUR*T1oN 8 9 10 SEGTENT 0.771 1.638 1.081 0.9% 1.034 0.581 1.108 2.201 2.428 1.398 0.861 1.699 3 0.640 2.392 5.196 1.870 2.293 l&09 Appendix G Table G4.

Objective Performance Scores (cont.)

Pitch Angle cl OCPH HBS81 TRACKING TASK DATA ENSEMBLE STATISTICS (STD. DEV. AVERAGING) i-22-61 LEVEL II: AVERAGES ACROSS PILOTS (ACROSS REPLICATIONS) VARIABLE 41 PITCli ANGLEdDEGI (XPIL) MEAN PITCH ANGLElOEGI - TUlSULENCE DN 1 2 CONFJGURATIDN 8 9 10 sEGYNT 6.160 5.403 5.965 6.081 6.013 6.068 2 1.861 2.520 2.880 2.519 3.018 3.209 3 2.706 3.010 2.732 3.459 3.402 3.313 (SGXPIL) STANDARD DEVIATION OF XREP ACROSS PILDTS - TlRBlJLWCE ON 1A FUNCTION OF XPIL AND XREP**2) CONF 1 2 $GURATION 8 9 10 SEG:ENT 0.165 1.576 0.387 O.lB 0.104 0.074 2 0.542 0.761 0.349 0.577 0.493 0.305 3 1.120 1.190 0.739 0.4% 0.211 0.097 (SIGPI ) AVERAGE STANDARD DEVIATION - TUlB L NCE W AVERAGE OF SIGREP VALUES ACROSS ‘i PIL 8s T # 1 2 CDNF iGUMTION 8 9 10 SEG:ENT 1.151 1.401 2.324 1.264 0.985 0.859 2 1.177 1.773 2.210 1.191 I.286 1.254 3 1.372 2.250 2.728 1.811 1.484 1.491 (SGSGPL) STANDARD DEVIATION OF SIGREP ACROSS PILOTS - TURBULENCE ON (A FUNCTION OF SIGPIL AND SIGREP-2) 1 2 CONFJGURATION 8 9 10 SEGfENT 0.119 0.377 0.367 0.300 0.354 0.138 0.658 2 0.240 0.767 0.255 0.445 0*5$3 3 0.392 0.714 1.482 0.726 0.600 0.399 Appendix G Objective Performance Scores (cont.)

Table G4.

Pitch Rate d) DCPH MS81 TRACKING TASK DATA ENSEMLE STATISTICS [STD. DEV. AVERAGING) 4-22-81 LEVEL II: AVERAGES ACROSS PILOTS (ACROSS REPLICATIONSb VARIABLE 5: PITCH RATE lDEG/SEC) 1XPIL) MEAN PITCH RATE - TURBULENCE ON I DEG/SEC ) CONFJGURAT1oN 8 9 10 1 2 -0.001 -0.oQ6 -0.008 -0.001 SEGTENT -0.009 -0.021 0.029 0.016 0.006 0.010 2 -0.005 0.008 0.021 0.039 -0.014 -0.024 3 0.012 0.015 1SGXPIL) STANDARD OEVIATIW OF XREP ACROSS PILOTS - TlRBULENCE ON (A FUNCTION OF XPIL AND XREP**Z) 1 2

wNF:GUR*T1oN 8 9 10

sEGFNT 0.020 0.017 0.025 0.008 0.015 0.010 2 0.012 0.041 0.019 0.036 0.016 0.019 3 0.012 0.039 O.DSb 0.045 0.045 0.091 (SIGPIL) AVERAGE STANDARD DEVIATION - TURBULENCE ON (AVERAGE OF SIGREP VALUES ACROSS PILOTS) 1 2 CONFJGURATION 8 10 SEGYENT 0.5b2 0.615 1.288 0.619 0.365 0.341 2 o.eo 0.933 1.328 0.672 0.474 0.609 3 0.882 1.249 0.121 1.638 1.030 0.743 (SGSCPLJ STANCM~ ;W&l ON OF SIGREP ACROSS PILDTS TlnIBlJLWCE ON l&d OF SIGPIL AND SIGREP++t) 1 2 wNFJGURAT1oN 8 9 10 SEGTNT 0.050 0.087 0.255 0.135 0.120 0.027 2 0.046 0.1a5 0.310 0.078 0.077 0.189 3 0.249 0.199 0.522 0.302 0.250 0.135 Appendix G Table G4.

Objective Performance Scores (cont.)

Airspeed d OCPH MS81 TRACKING TASK DATA ENSEMBLE STATISTICS (STD. DEV. AVERAGING) 4-22-81 LEVEL II: AVERAGES ACROSS PILOTS (ACROSS REPLICATIONS) VARIABLE 1: AIRSPEEO IKT 1 - TURBULENCE ON (XPIL) MEAN AIRSPEED (KT) CONF;GURATION 1 2 SEG?ENT 139.264 138.958 139.689 139.711 139.385 138.383 2 143.733 144.24 1 142.499 143.076 144.137 142.423 3 140.902 142.787 143.080 140.215 140.247 140.362 i SGXPIL) STANDA;: DEVIATION OF XREP ACROSS PILOTS - TURBULENCE ON FUNCTION OF XPIL AND XREP**Z) CONF;GURATION 1 2 8 SEGI;IENT 0.707 0.816 0.722 0.456 1.362 1.429 2 1.568 4.274 1.904 2.170 2.041 4.051 3 1.041 2.093 2.784 '1.486 0.777 0.433 (SIGPIL) AVERAGE STANDARD DEVIATION - TCRBULENCE ON (AVERAGE OF SIGREP VALUES ACROSS PILOTS) 1 2 CONF;GURATION 8 9 10 3.782 3.846 3.788 SEG?ENT 3.372 4.277 4.166 2 3.484 5.45b 4.154 3.584 4.285 5.381 3 3.765 6.214 4.357 3.955 4.406 4.550 (SGSGPLI STANDARD OEVIATION OF SIGREP ACROSS PILOTS - TURBULENCE ON (A FUNCTIW OF SIGPIL AN0 SIGREP-2) 1 2 CONF;GURATION 8 9 10 SEGI;IENT 0.200 0.679 0.303 0.258 0.506 0.289 2 0.365 0.996 0.741 0.513 0.649 1.029 3 0.222 1.289 0.738 0.181 0.751 1.161 Appendix G Table G4.

Objective Performance Scores (cont.)

Elevator Deflection Angle f) OCFM MBSBl TRACKING TASK DATA ENSEMBLE STATISTICS (STD. DEV. AVERAGING) 4-22-81 LEVEL II: AVERAGES ACROSS PILOTS (ACROSS REPLICATIOWS) VARIABLE 6s ELEVATDR DEFLECTIW ANGLE tDEG1 (XPIL) MEAN ELEVATOR DEFLECTION ANGLE tDEG1 - TURBULENCE ON 1 2 CONFJGURATIDN 8 9 LO SEGfENT -0.016 -0.155 0.123 0.053 0.027 0.065 0.151 0.330 0.038 0.093 0.102 0.129 3 -0.348 -0.27 1 -0.019 0.230 0.13B 0.169 ISGXPIL) STANDARD DEVIATICM OF XREP ACROSS PILOTS - TURBULENCE ON (A FUNCTION OF XPIL AND XREP+*ZI 1 2 wNFJGURAT1oN 8 9 10 SEGfENT 0.130 0.273 0.100 0.101 0.036 0.096 2 0.284 0.619 0.436 0.204 0.169 0.180 3 0.374 0.207 0.358 0.298 0.177 0.213 (SIGPI b AVERAGE STAMDARD DEVIATION - TUIBUL NCE ON AVERAGE OF SIGREP VALUES ACROSS PILOT k E b 1 2 CONFJGURATIDN 8 9 10 SEGTNT 1.541 1.372 2.887 1.129 0.555 0.550 2 1.357 1.919 3.356 1.405 0.736 0.813 3 l.!%b 2.477 3.927 2.001 1.140 1.143 1 2 CDNFJCURATION 8 9 10 SEGfENT 0.B67 0.092 0.562 O.laa 0.165 0.030 2 0.165 0.229 0.585 0.220 0.109 0.091 0.438 0.121 0.757 0.2B6 0.201 0.141 Appendix G Table G4. Objective Performance Scores (concl.)

Elevator Deflection Rate 4) M8S81 TRACKING TASK DATA ENSEMBLE STATISTICS (STD. DEV. AVERAGING) 4-22-81 LEVEL II: AVERAGES ACROSS PILOTS (ACROSS REPLICATIWSI VARIABLE 9: THRUST 11000 LB.)

(XPIL) MEAN TI-IRUST (1000 LB) - TURBULENCE ON CONF:GURATION 1 2 8 9 SEG:ENT bO.568 61.604 59.886 60.604 60.009 59.851 2 38.424 38.604 43.583 39.853 42.411 41.843 3 37.877 40.609 40.767 41.151 42.359 43.875 (SGXPIL) STANDAIl: DEVIATION OF XREP ACROSS PILOTS TIRBULENCE ON FUNCTION OF XPIL AND XREP**2)

1 2 CWF4GURAT10N 8 9 10

SEGYENT 1.109 2.977 0.816 0.746 0.495 0.897 2 2.408 7.888 2.043 2.529 1.619 1.330 2.243 8.862 2.220 1.324 0.848 1.720 (SIGPIL) AVERAGE STANDARD DEViATION - TUlBULENCE DN 1 AVERAGE OF SIGREP VALUES ACROSS PILOTS) 1 2 wNFJGURATIDN 8 9 10 SEGlflENT 4.135 8.603 3.574 4.289 4.45D 3.661 2 4.460 11.788 3.442 4.179 4.118 5.773 5.765 12.293 3.742 4.397 4.361 4.843 (SGSGPL) STANDARD DEVIATION OF SIGREP ACROSS PILDTS - TURBULENCE DN (A FUNCTIDN OF SIGPIL AND SIGREP*+2) 1 2 CONFIGURATION 8 9 10 SEGTENT 0.372 2.497 0.749 0.887 1.004 1.358 2 1.257 1.734 0.549 1.061 1.520 0.993 3 2.317 2.206 1.093 0.767 1.508 2.617 Qualities of Piloted Airplanes", Military 1. "Flying Specification, MIL-F-8785B (ASG), 1969.

2. Cooper, G.E., and Harper, R.P., Jr., "The Use of Pilot Rating in the Evaluation of Aircraft Handling Qualities", NASA TN D-5153, April 1969.

3. Hess, R.A., "Prediction of Pilot Opinion Ratings Using an Optimal Pilot Model", Human Factors, 1977, 19(S), pp. 459-475.

4. Levison, W.H., "A Model-Based Technique for Predicting Pilot OpinionRatings for Large Commercial Transports", NASACR-3257, April 1980.

5. Levison, W.H., "A Model-Based Technique for Predicting Pilot Opinion Ratings for Large Commercial Transports", Proc. of the 16th Annual Conference on Manual Control, Cambridge, MA, May 1980, pp. 216-243.

6. Baron, S., and Levison, W.H., 'The Optimal Control Model: Status and Future Directions", Proc. of the 1980 International Conf. on Cybernetics and Society, Cambridge, MA, October 1980.

7. Rickard, W.W., "Longitudinal Flying Qualities in the Landing Approach", Proc. of the 12th Annual Conf. on Manual Control, NASA TMX-73, 170, May 1976.

8. Levison, W.H., Elkind, J.I., and Ward, J.L., "Studies of Multi-Variable Manual Control Systems: A Model for Task Interference", NASA CR-1746, May 1971.

9. Levison, W.H., "A Model for Mental Workload in Tasks Requiring Continuous Information Processing", Mental Workload Its Theory and Measurement, Plenum Press, New York and London, 1979.

10. Levison, W.H., "The Effects of Display Gain and Signal Bandwidth on Human Controller Remnant", Wright-Patterson Air Force Base, OH, AMRL-TR-70-93, March 1971.

11. Chalk, C.R., Neal, T.P., Harris, T.M., and Pritchard, F.E., "Background Tnformation and User Guide for MIL-F-8785B (ASG), "Military Specification-Flying Qualities of Piloted Airplanes", AFFDL-TR-69-72, August 1969.

12. Hess, R.A., "A Pilot Modeling Technique for Handling-Qualities of Cybernetics and Information Science, Vol. 3, Research", J.

No. l-4, 1980, pp. 57-97.

13. Smit, J., and Wewerinke, P.H., "An Analysis of Helicopter Pilot Control Behavior and Workload During Instrument Flying Tasks", National Aerospace Laboratory NLR, PublicationNo. NLRMP 78003 U, May 1978.

14. McRuer, D., Ashkinas, I., and Graham, D., Aircraft Dynamics and Automatic Control, Princeton University Press, 1973.

15. Junker, A.M., and Levison, W.H., "Some Empirical Techniques for Human Operator Performance Measurement", Proceedings of the International Conf. on Cybernetics and Society, CambridgcMA, October 8-10, 1980.

1. Report No. 2. Government Accession No.

3. Recipient’s Catalog No.

NASA CR-3572 4. Title and Subtitle 5. Report Date June 1982 ANALYTICAL AND SIMULATOR STUDY OF ADVANCED TRANSPORT HANDLING QUALITIES 6. Performing Organization Code 7. Author(s) 8. Performing Organization Report No, William H. Levison and William W. Rickard 10. Work Unit No.

9. Performing Organization Name and Address Bolt Beranek and Newman Inc.

50 Moulton Street Cambridge, MA 02238 and 11. Contract or Grant No.

Douglas Aircraft Company NASl-16410 Long Beach, CA 90846 13. Type of Report and Period Covered 2. Sponsoring Agency Name and Address Contractor Report National Aeronautics and Space Administration 14. Sponsoring Agency Code Washington, D.C.

20546 I 5. Supplementary Notes Martin T.

Langley Technical Monitors: Moul and David B. Middleton Final Report 6. Abstract An analytic methodology, based on the optimal-control pilot model, is demonstrated for assessing longitudinal-axis handling qualities of Calibration of the methodology is transport aircraft in final approach.

largely in terms of closed-loop performance requirements, rather than specific vehicle response characteristics, and is based on a combination of published criteria, pilot preferences, physical limitations, and engineering judgment.

Six longitudinal-axis approach configurations were studied covering including the presence of a range of handling qualities problems, flexible aircraft modes. The analytical procedure was used to obtain predictions of (a) Cooper-Harper ratings, (b) a scalar quadratic and (c) rms excursions of important system variables.

performance index, A subsequent manned simulation study yielded objective and subjective performance measures that varied across vehicle configurations in the In particular, flexible modes for manner predicted by model analysis.

the specific configurations explored in this simulation study were correctly predicted to have no significant effect on handling qualities.

r. Key Words (Suggested by Author(s)) 18. Distribution Statement Handling Qualities Unclassified - Unlimited Flexible Modes Pilot Modeling Human Operator Technoldgy Subject Category 08 20. Security Classif. iof this page) Unclassified Unclassified For Sate by the National Technical Information Service, Springfield, Virginia 22161 NASA-Langley, 1982