Flight Experiment Investigation of General Aviation Self-Separation and Sequencing Tasks

APPENDIX A. Time-Coded Data Recorded by the Data Acquisition System

(DAS)

State variables and lateral flight path deviation were the primary measurement types; however, supporting measurements including rates, accelerations, and control positions were also recorded. All data analyses were performed on parameters sampled at five samples per second.

• Manifold Pressure (in inches of mercury) • Filtered Y Acceleration (in Gs) • Side Stick (pitch in inches; roll in degrees) • Filtered Z Acceleration (in Gs) • Throttle (in inches) • Indicated Airspeed (in knots) • Rudder Position (in degrees) • Calibrated Airspeed (in knots) • Flaps (in degrees) • True Airspeed (in knots) • Global Positioning System (GPS) Time • Filtered Indicated Airspeed (in knots) • Latitude (in degrees, minutes, and seconds) • Filtered True Airspeed (in knots) • Longitude (in degrees, minutes, and • Vertical Speed (in feet per minute) seconds) • Filtered Vertical Speed (in feet per minute) • Magnetic Heading (in degrees) • Ground Speed (in knots) • True Heading (in degrees) • Pressure Altitude (in feet) • Pitch Attitude (in degrees) • Filtered Pressure Altitude (in feet) • Roll Attitude (in degrees) • True Altitude (in feet) • Roll Rate (in degrees per second) • Filtered True Altitude (in feet) • Pitch Rate (in degrees per second) • Outside Air Temperature (in degrees • Yaw Rate (in degrees per second) Celsius) • Heading Rate (in degrees per second) • Track Angle (in degrees) • Filtered Roll Rate (in degrees per second) • Elevator (in degrees) • Filtered Pitch Rate (in degrees per second) • Destination Waypoint • Filtered Yaw Rate (in degrees per second) • Distance to Waypoint • Filtered Heading Rate (in degrees per • Lateral Path Deviation (in nautical miles) second) • Track (in degrees) • Turn Coordinator Rate (in degrees per • Desired Track (in degrees) second) • Groundspeed (in knots) • X Acceleration (in Gs) • Bearing to Destination (in degrees) • Y Acceleration (in Gs) • Discrete signals from each event marker in • Z Acceleration (in Gs) the cabin • Filtered X Acceleration (in Gs)

APPENDIX B. Correction Equations Used for the Air Data, Attitude and

Heading Reference System (ADAHRS) and Data Acquistion System (DAS)

The Evaluation Pilots (EPs) used standard flight instrumentation to control airspeed, altitude, and vertical speed during the Small Aircraft Transportation System (SATS) Higher Volume Operations (HVO) Self-Separation and Sequencing (SSS) Flight Experiment. This instrumentation included an altimeter, airspeed indicator, and vertical speed indicator, all of which are mechanical, panel-mounted instruments that measure air pressures within the pitot-static system to provide analog outputs. None of these instruments provided digital information that was needed by the research software or for recording in the data acquisition system (DAS). However, this information was available in digital formats from the Air Data, Attitude, and Heading Reference System (ADAHRS) that is part of the baseline research system on the test aircraft.

It was necessary to correct the digital data from the ADAHRS to match similar analog data provided to the EP by the standard flight instrumentation. Data to develop the equations used for these corrections were attained during ground tests by simulating specific pressure changes to the pitot-static system on the aircraft. The aircraft’s instruments were “tapped” before each indication was recorded to account for in- flight vibration. Airspeed measurements were taken in increments of 10 knots (kts), from 80 to 130 kts; altitude measurements were taken in increments of 100 feet (ft.), from 1900 to 6100 ft.; and vertical speed measurements were taken in increments of 100 feet per minute (fpm), from -1200 to 500 fpm. All of these data were recorded in both ascending and descending order.

A linear correlation was made between the ADAHRS data and the standard flight instrumentation data to derive the following correction equations for vertical speed, airspeed, and altitude provided by the ADAHRS. These equations were used by the research software on a real-time basis.

Vertical speed Y = 1.0442X + 1.4555 fpm Airspeed Y = 0.8345X + 28.715 kts.

Altitude Y = 0.9951X – 81.249 ft.

Where: X is uncorrected data Y corrected data

APPENDIX C. Details Regarding Flight Path Parameter Deviation Data

Extraction and Reduction

The flight path deviation data that were analyzed to assess the Evaluation Pilot’s (EP) ability to fly the global positioning system (GPS) approaches with the Small Aircraft Transportation System (SATS) Higher Volume Operations (HVO) procedures were flight parameters that were to remain constant or nulled during flight. The EPs were instructed to null lateral path deviation and maintain constant airspeed during all segments of the approach path and maintain constant altitude during the level-flight segments.

Each of the test conditions contained flight paths that had multiple level-flight segments followed by descent segments. A methodology was developed to consistently determine when the EP had transitioned between level-flight and descending path segments with enough time for the altitude and vertical speeds to become stable for their appropriate segments. Once the start and stop times for each path segment for each test run was established, the data could be further sectioned for the calculation of root mean squared error (RMSE) values.

The following exponential moving average (EMA) of altitude was applied to establish a consistent method of determining flight segments: (C1) Exponential Moving Average = (Current Data Value × (Exponential Percentage)) + (previous exponential moving average value × (1- (Exponential Percentage)) The relationship of the moving average calculation of the altitude parameter was compared to the value of the parameter itself. In an EMA, more weight is given to more recent data. The greater weight helped determine (within a shorter period) an intent by the EP to either maintain altitude, initiate a descent, or level off. The flight path parameter deviation data were requested at a rate of five samples per second (sps). EMA was calculated over a period of 5 seconds, or 25 samples.

2 2 (C2) 077 . 0 Percentage l Exponentia = = = 1 25 1 Samples + + Substituting the above value, the EMA formula is as follows: (C3) Exponential Moving Average = ( ) ( ) ( ) 077 . 0 1 value average moving l exponentia previous 0.077 Value Data Current − × + × For the first calculation of the EMA, the current data value of altitude was used as input for the “previous EMA value.” Subsequently, this first calculation of EMA became the value for the “previous EMA value,” and the EMA calculation was performed on all subsequent altitude data. Ideally, when the EMA minus the current altitude is less than zero (i.e., EMA – current altitude < 0), this would signal the beginning of a descent. However, “whiplash” was observed where the difference between the EMA and the current altitude data value became negative and shortly thereafter became positive again (i.e., representing a false indication). To reduce the false indications, a value of -0.5 was used. When the value of “EMA - current altitude” approached -0.5 from above, false indications were greatly reduced.

When the value of “EMA - current altitude” approached the value -0.5 from below, the EP was leveling off from a descent. Using the EMA method of segmenting data, the flight segments for the beginning of descent and the beginning of level off could be consistently determined with minimum subjective judgment required by the experimenters.

To eliminate each EP’s transition from level-flight to a descent and vice versa, only the middle 30 seconds of descent between the start of descent and level off were analyzed. This helped ensure that the data that were analyzed corresponded to the point at which the EP reached, and was attempting to maintain, a constant rate of descent.

For each flight segment, root mean squared error (RMSE) values were determined by using the corrected data and measuring their deviation from a “target value.” Target values were defined from instrument approach procedures and included altitudes, airspeeds, and lateral path deviations. Given the deviation from the targeted value, the RMSE values associated with altitude and airspeed could be calculated for every straight-and-level segment; airspeed RMSE values could be calculated for all descent segments; and lateral path deviation RMSE values could be calculated for all segments except those involving holding patterns. The equation used to calculated RMSE values was as follows: N ) arget ParameterT ctual ParameterA ( −

∑

1 = i (C4) RMSE = N In this equation, “parameter actual” is represented by the corrected raw data, and “N” is the number of data points used in the flight segment RMSE calculation. Target values were depicted by the instrument approach charts provided to the EPs prior to the experiment flight. The target values for each parameter depended upon the flight segment definition such that: • The airspeed target was 120 knots (kts) prior to the Final Approach Fix (FAF) and was applied to all flight segments; and • Altitude targets varied from 5,500 feet (ft.) to 2,420 ft. and were applied to only the straight-and- level defined flight segments in each test condition.

Once the RMSE value was calculated for each applicable parameter in each flight segment, a total average value was calculated for each parameter recorded during a given replicate of a given test condition for a given EP. Table C1 shows a sample test condition RMSE table.

Table C1. Root Mean Squared Error (RMSE) Values Collected from an Evaluation Pilot during the First Replicate of the Straight In, In-Trail Approach using the Method of Approach Separation 2 Display Format Flight Segment Airspeed Altitude Lateral Path deviation (in knots) (in feet) (in nautical miles) Level 5500 1.01 42.97 0.09 Descent to 4500 4.37 Not Applicable 0.13 Level 4500 3.21 25.48 0.09 Descent to 3700 0.92 Not Applicable 0.17 Level 3700 3.00 15.75 0.16 Average RMSE: 2.50 28.07 0.13 The average RMSE values (Table C1) were analyzed by way of 2 (Approach Type) x 3 (Display Type) x 2 (Replicate) Analysis of Variance (ANOVA) tests.

Adherence to the Federal Aviation Administration’s (FAA) Instrument Rating Practical Test Standards (PTS) was determined using the same target values used in calculating the RMSE values for each flight path parameter [C1]. These target values represented the “correct” instrumentation reading that the EP was instructed to “fly to” during the test conditions’ different flight segments. The number and duration of each failure to adhere to the PTS associated with each flight path parameter was determined for each flight segment completed by each EP during each replicate of each test condition.

Then, for a given flight path parameter, the total time associated with failures to fly within the PTS performance criteria was divided by the total time flying to obtain a percentage of time spent flying out of PTS conformance. These percentage values were analyzed by way of 2 (Approach Type) x 3 (Display Type) x 2 (Replicate) Analysis of Variance (ANOVA) tests.

Reference [C1] Federal Aviation Administration: Instrument Rating Practical Test Standards for Airplane. FAA- S-8081-4C with Changes 1 & 2, U.S. Department of Transportation (Washington, D.C.), 1998.

APPENDIX D. Detailed Results of Flight Path Parameter Deviation Data

Analyses

Deviations from assigned altitude, airspeed, and the lateral path deviation from the assigned route on the instrument approach were used to quantify Evaluation Pilots’ (EPs) flight performance. Root mean squared error (RMSE) values were calculated for these parameters for each test run. Then, a series of 2 (Approach Type) x 3 (Display Type) x 2 (Replicate) Analysis of Variance (ANOVA) tests was conducted on the RMSE values of altitude, airspeed, and lateral path deviations to determine if significant differences existed in these values when a given display type was used during a given approach type [D1, D2, D3].

The main effects of approach type and display type and the Approach Type x Display Type interaction effect were of primary interest. Means and standard deviations associated with the main effect of replicate and interaction effects involving the replicate factor are not presented unless they were found to be significant. Since the main effect of subjects is usually significant in these types of analyses, it is not specifically mentioned unless it was found to be not significant. For all statistical tests, a 5-percent significance level was set a priori.

Altitude Deviation The 2 x 3 x 2 ANOVA conducted on the RMSE values of altitude revealed that: • No significant difference was found to exist between the altitude RMSE values associated with each approach type (F [1, 5] = 0.004; p = 0.954).

• No significant difference was found to exist among the altitude RMSE values associated with the use of different display types (F [2, 10] = 3.30; p = 0.079).

• A significant difference was found to exist between the altitude RMSE values associated with each replicate (F [1, 5] = 7.44; p = 0.041). The occurrence of smaller altitude deviations when EPs completed the second replicate is most likely a result of a practice or learning effect since EPs always performed replicate #2 after completing replicate #1.

• No significant Approach Type x Display Type interaction was found to exist (F [2, 10] = 0.56; p = 0.591).

• No significant Approach Type x Replicate interaction was found to exist (F [1, 5] = 0.17; p = 0.698).

• No significant Display Type x Replicate interaction was found to exist (F [2, 10] = 0.23; p = 0.798).

• No significant Approach Type x Display Type x Replicate interaction was found to exist (F [2, 10] = 1.07; p = 0.378).

Approach Type Table D1 contains the means and standard deviations associated with the altitude deviation data [RMSE values in feet (ft.)] attained for straight-in, in-trail approaches and simultaneous arrival approaches. Sample size equals 36 since altitude deviations were averaged across the three display types.

Table D1. Root Mean Squared Error Altitude Deviations Associated with Approach Type Approach Type Mean (ft.) Standard Deviation (ft.) Sample Size Straight In, In-trail 39.40 13.99 36 Simultaneous Arrival 39.52 12.46 36 EPs maintained assigned altitudes equally well when they performed straight-in, in-trail approaches and when they performed simultaneous arrival approaches.

Display Type Table D2 contains the means and standard deviations associated with the altitude deviation data (RMSE values in ft.) attained for the Baseline display format, the MAS 1 display format, and the MAS 2 display format. Sample size equals 24 since altitude deviations were averaged across the two approach types.

Table D2. Root Mean Squared Error Altitude Deviations Associated with Display Type Display Type Mean (ft.) Standard Deviation (ft.) Sample Size Baseline 37.53 11.76 24 Method of Approach Separation 1 37.88 8.95 24 Method of Approach Separation 2 42.98 10.25 24 EPs maintained assigned altitudes equally well when they performed approaches using the Baseline display format, the MAS 1 display format, and the MAS 2 display format.

Replicate Table D3 contains the means and standard deviations associated with the altitude deviation data (RMSE values in ft.) attained for replicate #1 and replicate #2. Sample size equals 36 since altitude deviations were collected from six EPs, each of whom completed a series of six test conditions twice.

Table D3. Root Mean Squared Error Altitude Deviations Associated with Display Type Replicate Mean (ft.) Standard Deviation (ft.) Sample Size #1 43.57 13.32 36 #2 35.35 11.78 36 As noted earlier, the ANOVA revealed that the main effect of replicate was statistically significant.

An examination of the mean altitude deviations associated with each replicate shows that smaller altitude deviations occurred when EPs completed the second replicate. This finding is most likely a result of a practice or learning effect since EPs always performed replicate #2 after completing replicate #1.

Approach Type x Display Type Table D4 and Figure D1 contain the means and standard deviations associated with the altitude deviation data (RMSE values in ft.) attained when straight-in, in-trail approaches and simultaneous arrival approaches were performed using different display types. Sample size equals 12 since altitude deviations were not averaged across approach type or display type.

Table D4. Root Mean Squared Error Altitude Deviations Associated with Approach Type x Display Type Test Condition Mean (ft.) Standard Deviation (ft.) Sample Size Straight In, In-trail Approach / 36.82 13.69 12 Baseline Display Format Simultaneous Arrival Approach / 38.24 14.41 12 Baseline Display Format Straight In, In-trail Approach / 36.64 11.91 12 Method of Approach Separation 1 Display Format Simultaneous Arrival Approach / 39.11 11.13 12 Method of Approach Separation 1 Display Format Straight In, In-trail Approach / 44.75 15.70 12 Method of Approach Separation 2 Display Format Simultaneous Arrival Approach / 41.20 12.54 12 Method of Approach Separation 2 Display Format Altitude Deviation Root Mean Squared Error with 95% Confidence Interval, feet 36.82 38.24 36.64 39.11 44.75 41.2 Straight In Sim. Arrival Straight In Sim. Arrival Straight In Sim. Arrival Baseline Baseline MAS 1 MAS 1 MAS 2 MAS 2 Approach Type x Display Type Figure D1. Mean altitude deviations associated with Approach Type x Display Type.

EPs maintained assigned altitudes equally well when they performed different approach types using different types of displays.

Airspeed Deviation The 2 x 3 x 2 ANOVA conducted on the RMSE values of airspeed revealed that: • A significant difference was found to exist between the airspeed RMSE values associated with each approach type (F [1, 5] = 6.80; p = 0.048).

• No significant difference was found to exist among the airspeed RMSE values associated with the use of different display types (F [2, 10] = 0.49; p = 0.627).

• No significant difference was found to exist between the airspeed RMSE values associated with each replicate (F [1, 5] = 0.90; p = 0.386).

• No significant Approach Type x Display Type interaction was found to exist (F [2, 10] = 0.55; p = 0.595).

• No significant Approach Type x Replicate interaction was found to exist (F [1, 5] = 0.01; p = 0.939).

• No significant Display Type x Replicate interaction was found to exist (F [2, 10] = 0.13; p = 0.883).

• No significant Approach Type x Display Type x Replicate interaction was found to exist (F [2, 10] = 0.28; p = 0.763).

Approach Type Table D5 contains the means and standard deviations associated with the airspeed deviation data [RMSE values in knots (kts)] attained for straight-in, in-trail approaches and simultaneous arrival approaches. Sample size equals 36 since airspeed deviations were averaged across the three display types.

Table D5. Root Mean Squared Error Airspeed Deviations Associated with Approach Type Approach Type Mean (kts) Standard Deviation (kts) Sample Size Straight In, In-trail 3.11 1.34 36 Simultaneous Arrival 2.85 0.99 36 Since the ANOVA revealed that the main effect of approach type was statistically significant, a simple examination of the mean airspeed deviations associated with each approach type was used to determine if greater airspeed deviations occurred during the straight-in, in-trail approaches or during the simultaneous arrival approaches. When airspeed RMSE values were averaged across the three display types, the mean airspeed deviation was 2.85 kts during the simultaneous arrival approaches, and the mean airspeed deviation was 3.11 kts during the straight-in, in-trail approaches. Statistically, EPs were able to maintain assigned airspeeds more accurately during the simultaneous arrival approaches than during the straight-in, in-trail approaches. However, a mean airspeed difference of just one-quarter of a knot is operationally insignificant.

Display Type Table D6 contains the means and standard deviations associated with the airspeed deviation data (RMSE values in kts) attained for the Baseline display format, the MAS 1 display format, and the MAS 2 display format. Sample size equals 24 since airspeed deviations were averaged across the two approach types.

Table D6. Root Mean Squared Error Airspeed Deviations Associated with Display Type Display Type Mean (kts) Standard Deviation (kts) Sample Size Baseline 2.92 1.20 24 Method of Approach Separation 1 3.01 1.13 24 Method of Approach Separation 2 3.01 1.24 24 EPs maintained assigned airspeed equally well when they performed approaches using the Baseline display format, the MAS 1 display format, and the MAS 2 display format.

Approach Type x Display Type Table D7 and Figure D2 contain the means and standard deviations associated with the airspeed deviation data (RMSE values in kts) attained when straight-in, in-trail approaches and simultaneous arrival approaches were performed using different display types. Sample size equals 12 since airspeed deviations were not averaged across approach type or display type.

Table D7. Airspeed Deviations (Root Mean Squared Error values in knots) Associated with Approach Type x Display Type Test Condition Mean (kts) Standard Deviation (kts) Sample Size Straight In, In-trail Approach / 2.83 1.09 12 Baseline Display Format Simultaneous Arrival Approach / 3.02 1.35 12 Baseline Display Format Straight In, In-trail Approach / 3.28 1.48 12 Method of Approach Separation 1 Display Format Simultaneous Arrival Approach / 2.74 0.57 12 Method of Approach Separation 1 Display Format Straight In, In-trail Approach / 2.81 0.95 12 Method of Approach Separation 2 Display Format Simultaneous Arrival Approach / 3.22 1.48 12 Method of Approach Separation 2 Display Format 4.5 3.5 2.5 Airspeed Deviation 1.5 Root Mean Squared Error with 95% Confidence Interval, knots 0.5 2.83 3.02 3.28 2.74 3.22 2.81 Straight In Sim. Arrival Straight In Sim. Arrival Straight In Sim. Arrival Baseline Baseline MAS 1 MAS 1 MAS 2 MAS 2 Approach Type x Display Type Figure D2. Mean airspeed deviations associated with Approach Type x Display Type.

EPs maintained assigned airspeeds equally well when they performed different approach types using different types of displays.

Lateral Path Deviation The 2 x 3 x 2 ANOVA conducted on the RMSE values of lateral path deviation revealed that: • No significant difference was found to exist between the lateral path deviation RMSE values associated with each approach type (F [1, 5] = 0.94; p = 0.376).

• No significant difference was found to exist among the lateral path deviation RMSE values associated with the use of different display types (F [2, 10] = 1.43; p = 0.284).

• No significant difference was found to exist between the lateral path deviation RMSE values associated with each replicate (F [1, 5] = 1.49; p = 0.276).

• No significant Approach Type x Display Type interaction was found to exist (F [2, 10] = 2.08; p = 0.176).

• No significant Approach Type x Replicate interaction was found to exist (F [1, 5] = 1.08; p = 0.345).

• No significant Display Type x Replicate interaction was found to exist (F [2, 10] = 0.90; p = 0.435).

• No significant Approach Type x Display Type x Replicate interaction was found to exist (F [2, 10] = 0.55; p = 0.591).

Approach Type Table D8 contains the means and standard deviations associated with the lateral path deviation data [RMSE values in nautical miles (n. m.)] attained for straight-in, in-trail approaches and simultaneous arrival approaches. Sample size equals 36 since lateral path deviations were averaged across the three display types.

Table D8. Root Mean Squared Error Lateral Path Deviations Associated with Approach Type Approach Type Mean (n. m.) Standard Deviation (n. m.) Sample Size Straight In, In-trail 0.08 0.04 36 Simultaneous Arrival 0.07 0.05 36 EPs maintained the assigned course equally well when they performed straight-in, in-trail approaches and when they performed simultaneous arrival approaches.

Display Type Table D9 contains the means and standard deviations associated with the lateral path deviation data (RMSE values in n. m.) attained for the Baseline display format, the MAS 1 display format, and the MAS 2 display format. Sample size equals 24 since lateral path deviations were averaged across the two approach types.

Table D9. Root Mean Squared Error Lateral Path Deviations Associated with Display Type Display Type Mean (n. m.) Standard Deviation (n. m.) Sample Size Baseline 0.07 0.05 24 Method of Approach Separation 1 0.09 0.05 24 Method of Approach Separation 2 0.07 0.03 24 EPs maintained assigned course equally well when they performed approaches using the Baseline display format, the MAS 1 display format, and the MAS 2 display format.

Approach Type x Display Type Table D10 and Figure D3 contain the means and standard deviations associated with the lateral path deviation data (RMSE values in n. m.) attained when straight-in, in-trail approaches and simultaneous arrival approaches were performed using different display types. Sample size equals 12 since lateral path deviations were not averaged across approach type or display type.

Table D10. Root Mean Squared Error Lateral Path Deviations Associated with Approach Type x Display Type Test Condition Mean (n. m.) Standard Deviation (n. m.) Sample Size Straight In, In-trail Approach / 0.07 0.03 12 Baseline Display Format Simultaneous Arrival Approach / 0.08 0.06 12 Baseline Display Format Straight In, In-trail Approach / 0.10 0.05 12 Method of Approach Separation 1 Display Format Simultaneous Arrival Approach / 0.07 0.05 12 Method of Approach Separation 1 Display Format Straight In, In-trail Approach / 0.07 0.03 12 Method of Approach Separation 2 Display Format Simultaneous Arrival Approach / 0.07 0.03 12 Method of Approach Separation 2 Display Format 0.16 0.14 0.12 0.1 0.08 0.06 Lateral Path Deviation Root Mean Squared Error 0.04 0.02 with 95% Confidence Interval, nautical miles 0.07 0.08 0.1 0.07 0.07 0.07 Straight In Sim. Arrival Straight In Sim. Arrival Straight In Sim. Arrival Baseline Baseline MAS 1 MAS 1 MAS 2 MAS 2 Approach Type x Display Type Figure D3. Mean lateral path deviations associated with Approach Type x Display Type.

EPs maintained assigned course equally well when they performed different approach types using different types of displays.

References [D1] Keppel, G.: Design and Analysis: A Researcher’s Handbook. Third ed., Prentice-Hall, Inc., 1991.

[D2] Norusis, M. J.: SPSS® for Windows™: Base System User’s Guide, Release 6.0 . SPSS Inc. (Chicago, IL), 1993.

[D3] Winer, B. J.; Brown, D. R.; and Michels, K. M.: Statistical Principles in Experimental Design. Third ed., McGraw-Hill, Inc., 1991.

APPENDIX E. Detailed Results of Adherence to PTS for the Instrument

Rating Data Analyses

Percentages of time that Evaluation Pilots (EPs) failed to adhere to the Practical Test Standards (PTS) for the Instrument Rating were used to quantify flight performance. The duration (i.e., elapsed time in seconds) of each failure to fly within the PTS was the difference between the point in time when a PTS parameter was breached and the point in time when adherence to that PTS parameter was regained. For a given flight path parameter, the total time associated with failures to fly within PTS performance criteria was divided by the total time flying to obtain a percentage of time spent flying out of PTS conformance.

Percentages of time that EPs failed to adhere to the PTS were calculated for altitude, airspeed, and lateral path deviation. Data used to assess PTS conformance were obtained from 30-second segments during descents where the parameters would be stabilized. Airspeed and lateral path deviation data were assessed during both level-flight and descents. Altitude data were assessed only during level-flight segments. All EPs successfully adhered to the PTS performance criteria for lateral tracking for the Instrument Rating (i.e., ± 1.0 n. m. prior to the FAF, and ± 0.225 n. m. after the FAF) 100-percent of the time. Therefore, 2 (Approach Type) x 3 (Display Type) x 2 (Replicate) ANOVAs were conducted on the percentages of time that EPs failed to adhere to the PTS for altitude and airspeed to determine if significant differences existed in these values when a given display type was used during a given approach type [E1, E2, E3].

The main effects of approach type and display type and the Approach Type x Display Type interaction effect were of primary interest. Means and standard deviations associated with the main effect of replicate and interaction effects involving the replicate factor are not presented unless they were found to be significant. Since the main effect of subjects is usually significant in these types of analyses, it is not specifically mentioned unless it was found to be not significant. For all statistical tests, a 5-percent significance level was set a priori.

Adherence to Altitude PTS The 2 x 3 x 2 ANOVA conducted on the percentages of time that EPs failed to adhere to the PTS for altitude revealed that: • No significant difference was found to exist among the percentages of altitude PTS nonconformance associated with subjects (F [1, 5] = 6.18; p = 0.055).

• No significant difference was found to exist between the percentages of altitude PTS nonconformance associated with each approach type (F [1, 5] = 3.46; p = 0.122).

• No significant difference was found to exist among the percentages of altitude PTS nonconformance associated with the use of different display types (F [2, 10] = 0.09; p = 0.918).

• No significant difference was found to exist between the percentages of altitude PTS nonconformance associated with each replicate (F [1, 5] = 2.44; p = 0.179).

• No significant Approach Type x Display Type interaction was found to exist (F [2, 10] = 1.29; p = 0.317).

• No significant Approach Type x Replicate interaction was found to exist (F [1, 5] = 0.18; p = 0.687).

• No significant Display Type x Replicate interaction was found to exist (F [2, 10] = 0.55; p = 0.592).

• No significant Approach Type x Display Type x Replicate interaction was found to exist (F [2, 10] = 1.17; p = 0.350).

Subjects Across EPs, there was very little variability among the percentages of altitude PTS nonconformance.

EPs adhered to the altitude PTS in a very consistent manner with each other across the test conditions.

Approach Type Table E1 contains the means and standard deviations associated with the percentages of altitude PTS nonconformance attained for straight-in, in-trail approaches and simultaneous arrival approaches. Sample size equals 36 since percentages were averaged across the three display types.

Table E1. Percentages of Altitude PTS Nonconformance Associated with Approach Type Approach Type Mean Standard Deviation Sample Size Straight In, In-trail 1.08 2.71 36 Simultaneous Arrival 2.01 3.23 36 EPs adhered to the altitude PTS equally well when they performed straight-in, in-trail approaches and when they performed simultaneous arrival approaches.

Display Type Table E2 contains the means and standard deviations associated with the percentages of altitude PTS nonconformance attained for the Baseline display format, the MAS 1 display format, and the MAS 2 display format. Sample size equals 24 since percentages were averaged across the two approach types.

Table E2. Percentages of Altitude PTS Nonconformance Associated with Display Type Display Type Mean Standard Deviation Sample Size Baseline 1.52 3.48 24 Method of Approach Separation 1 1.68 2.28 24 Method of Approach Separation 2 1.43 3.23 24 EPs adhered to the altitude PTS equally well when they performed approaches using the Baseline display format, the MAS 1 display format, and the MAS 2 display format.

Approach Type x Display Type Table E3 and Figure E1 contain the means and standard deviations associated with the percentages of altitude PTS nonconformance attained when straight-in, in-trail approaches and simultaneous arrival approaches were performed using different display types. Sample size equals 12 since percentages were not averaged across approach type or display type.

Table E3. Percentages of Altitude PTS Nonconformance Associated with Approach Type x Display Type Test Condition Mean Standard Sample Size Deviation Straight In, In-trail Approach / 0.75 1.86 12 Baseline Display Format Simultaneous Arrival Approach / 2.30 4.53 12 Baseline Display Format Straight In, In-trail Approach / 0.71 1.47 12 Method of Approach Separation 1 Display Format Simultaneous Arrival Approach / 2.65 2.57 12 Method of Approach Separation 1 Display Format Straight In, In-trail Approach / 1.77 4.11 12 Method of Approach Separation 2 Display Format Simultaneous Arrival Approach / 1.09 2.16 12 Method of Approach Separation 2 Display Format PTS Noncomformance Mean Percentage of Altitude with 95% Confidence Interval 0.75 2.3 0.71 2.65 1.77 1.09 Straight In Sim. Arrival Straight In Sim. Arrival Straight In Sim. Arrival Baseline Baseline MAS 1 MAS 1 MAS 2 MAS 2 Approach Type x Display Type Figure E1. Mean percentages of altitude Practical Test Standard (PTS) nonconformance associated with Approach Type x Display Type.

EPs adhered to the altitude PTS equally well when they performed different approach types using different types of displays.

Adherence to Airspeed PTS The 2 x 3 x 2 ANOVA conducted on the percentages of time that EPs failed to adhere to the PTS for airspeed revealed that: • No significant difference was found to exist among the percentages of airspeed PTS nonconformance associated with subjects (F [1, 5] = 2.38; p = 0.183).

• No significant difference was found to exist between the percentages of airspeed PTS nonconformance associated with each approach type (F [1, 5] = 2.35; p = 0.186).

• No significant difference was found to exist among the percentages of airspeed PTS nonconformance associated with the use of different display types (F [2, 10] = 2.38; p = 0.142).

• No significant difference was found to exist between the percentages of airspeed PTS nonconformance associated with each replicate (F [1, 5] = 0.35; p = 0.580).

• No significant Approach Type x Display Type interaction was found to exist (F [2, 10] = 1.72; p = 0.228).

• No significant Approach Type x Replicate interaction was found to exist (F [1, 5] = 0.001; p = 0.980).

• No significant Display Type x Replicate interaction was found to exist (F [2, 10] = 0.17; p = 0.849).

• No significant Approach Type x Display Type x Replicate interaction was found to exist (F [2, 10] = 0.15; p = 0.861).

Subjects Across EPs, there was very little variability among the percentages of airspeed PTS nonconformance.

EPs adhered to the airspeed PTS in a very consistent manner with each other across the test conditions.

Approach Type Table E4 contains the means and standard deviations associated with the percentages of airspeed PTS nonconformance attained for straight-in, in-trail approaches and simultaneous arrival approaches. Sample size equals 36 since percentages were averaged across the three display types.

Table E4. Percentages of Airspeed PTS Nonconformance Associated with Approach Type Approach Type Mean Standard Deviation Sample Size Straight In, In-trail 1.11 3.72 36 Simultaneous Arrival 0.30 1.37 36 EPs adhered to the airspeed PTS equally well when they performed straight-in, in-trail approaches and when they performed simultaneous arrival approaches.

Display Type Table E5 contains the means and standard deviations associated with the percentages of airspeed PTS nonconformance attained for the Baseline display format, the MAS 1 display format, and the MAS 2 display format. Sample size equals 24 since percentages were averaged across the two approach types.

Table E5. Percentages of Airspeed PTS Nonconformance Associated with Display Type Display Type Mean Standard Deviation Sample Size Baseline 0.48 1.66 24 Method of Approach Separation 1 0.25 0.90 24 Method of Approach Separation 2 1.37 4.49 24 EPs adhered to the airspeed PTS equally well when they performed approaches using the Baseline display format, the MAS 1 display format, and the MAS 2 display format.

Approach Type x Display Type Table E6 and Figure E2 contain the means and standard deviations associated with the percentages of airspeed PTS nonconformance attained when straight-in, in-trail approaches and simultaneous arrival approaches were performed using different display types. Sample size equals 12 since percentages were not averaged across approach type or display type.

Table E6. Percentages of Airspeed PTS Nonconformance Associated with Approach Type x Display Type Test Condition Mean Standard Sample Size Deviation Straight In, In-trail Approach / 0.25 0.62 12 Baseline Display Format Simultaneous Arrival Approach / 0.72 2.30 12 Baseline Display Format Straight In, In-trail Approach / 0.50 1.24 12 Method of Approach Separation 1 Display Format Simultaneous Arrival Approach / 0.00* 0.00* 12 Method of Approach Separation 1 Display Format Straight In, In-trail Approach / 2.58 6.22 12 Method of Approach Separation 2 Display Format Simultaneous Arrival Approach / 0.17 0.58 12 Method of Approach Separation 2 Display Format * NOTE: A value of zero indicates conformance 100-percent of the time.

PTS Nonconformance with 95% Confidence Interval Mean Percentage of Airspeed 0.25 0.72 0.5 0 2.58 0.17 Straight In Sim. Arrival Straight In Sim. Arrival Straight In Sim. Arrival Baseline Baseline MAS 1 MAS 1 MAS 2 MAS 2 Approach Type x Display Type Figure E2. Mean percentages of airspeed Practical Test Standards (PTS) nonconformance associated with Approach Type x Display Type.

EPs adhered to the airspeed PTS equally well when they performed different approach types using different types of displays.

Adherence to the PTS Performance Criteria for Lateral Tracking All EPs successfully adhered to the CDI PTS for the Instrument Rating 100-percent of the time during all six of the experiment’s test conditions. It was concluded that EPs were able to maintain their course with an appropriate level of precision while performing straight-in, in-trail approaches and simultaneous arrival approaches using the Baseline display format, the MAS 1 display format, and the MAS 2 display format.

References [E1] Keppel, G.: Design and Analysis: A Researcher’s Handbook. Third ed., Prentice-Hall, Inc., 1991.

[E2] Norusis, M. J.: SPSS® for Windows™: Base System User’s Guide, Release 6.0 . SPSS Inc. (Chicago, IL), 1993.

[E3] Winer, B. J.; Brown, D. R.; and Michels, K. M.: Statistical Principles in Experimental Design. Third ed., McGraw-Hill, Inc., 1991.

APPENDIX F. Detailed Results of Subjective Workload Data Analyses

Evaluation Pilots (EPs) used the Air Force Flight Test Center’s (AFFTC) Seven-Point Subjective Workload Estimate Scale and the Modified Cooper-Harper (MCH) Rating Scale to rate the level of workload that they experienced during each of the experiment’s six test conditions [G1, G2]. Since each test condition was performed twice during the course of the experiment, each EP provided 12 AFFTC workload ratings and 12 MCH workload ratings. For each EP, the two AFFTC workload ratings associated with a given test condition were averaged together to yield a set of six mean workload ratings, and the two MCH workload ratings associated with a given test condition were averaged together to yield a set of six mean workload ratings. Nonparametric tests were employed as a conservative method for analyzing workload ratings associated with discrete rating scale items. For all statistical tests, a 5-percent significance level was set a priori.

Approach Type AFFTC Workload Ratings Table F1 contains the means and standard deviations for the AFFTC workload ratings associated with performing straight-in, in-trail approaches and simultaneous arrival approaches. Sample size equals 18 since workload ratings were averaged across the two replicates and the three display types.

Table F1. Air Force Flight Test Center Workload Estimate Scale Ratings Associated with Approach Type Approach Type Mean Standard Deviation Sample Size Straight In, In-trail 2.72 0.52 18 Simultaneous Arrival 2.61 0.53 18 A Wilcoxon Test (i.e., a nonparametric within-subject test appropriate for analyzing two related samples of ordinal data) performed on these means revealed that EPs reported experiencing equivalent levels of workload during the straight-in, in-trail approaches and the simultaneous arrival approaches (p = 0.6546) [G3].

MCH Workload Ratings Table F2 contains the means and standard deviations of with the MCH workload ratings associated with performing straight-in, in-trail approaches and simultaneous arrival approaches. Sample size equals 18 since workload ratings were averaged across the two replicates and the three display types.

Table F2. Modified Cooper-Harper Workload Ratings Associated with Approach Type Approach Type Mean Standard Deviation Sample Size Straight In, In-trail 2.42 0.97 18 Simultaneous Arrival 2.14 1.03 18 A Wilcoxon Test revealed that EPs reported experiencing equivalent levels of workload during the straight-in, in-trail approaches and the simultaneous arrival approaches (p = 0.0833).

Display Type AFFTC Workload Ratings Table F3 contains the means and standard deviations for the AFFTC workload ratings associated with performing approaches using the Baseline display format, the MAS 1 display format, and the MAS 2 display format. Sample size equals 12 since workload ratings were averaged across the two replicates and the two approach types.

Table F3. Air Force Flight Test Center Workload Estimate Scale Ratings Associated with Display Type Display Type Mean Standard Deviation Sample Size Baseline 2.63 0.53 12 Method of Approach Separation 1 2.63 0.57 12 Method of Approach Separation 2 2.75 0.50 12 A Friedman Test (i.e., a nonparametric within-subject test appropriate for analyzing three or more related samples of ordinal data) performed on these means revealed that EPs reported experiencing equivalent AFFTC workload ratings when different display types were used (X [2] = 2.2353; p = 0.3270) [G3, G4].

MCH Workload Ratings Table F4 contains the means and standard deviations for the MCH workload ratings associated with performing approaches using the Baseline display format, the MAS 1 display format, and the MAS 2 display format. Sample size equals 12 since workload ratings were averaged across the two replicates and the two approach types.

Table F4. Modified Cooper-Harper Workload Ratings Associated with Display Type Display Type Mean Standard Deviation Sample Size Baseline 2.13 0.98 12 Method of Approach Separation 1 2.38 1.11 12 Method of Approach Separation 2 2.33 0.96 12 A Friedman Test performed on these means revealed that EPs reported experiencing equivalent MCH workload ratings when different display types were used (X [2] = 0.7778; p = 0.6778).

Approach Type x Display Type AFFTC Workload Ratings Table F5 and Figure F1 contain the means and standard deviations for the AFFTC workload ratings associated with performing straight-in, in-trail approaches and simultaneous arrival approaches using different display types. Sample size equals 6 since workload ratings were averaged across replicates.

Table F5. Air Force Flight Test Center Workload Estimate Scale Ratings Associated with Approach Type x Display Type Test Condition Mean Standard Deviation Sample Size Straight In, In-trail Approach / 2.58 0.58 6 Baseline Display Format Simultaneous Arrival Approach / 2.67 0.52 6 Baseline Display Format Straight In, In-trail Approach / 2.75 0.52 6 Method of Approach Separation 1 Display Format Simultaneous Arrival Approach / 2.50 0.63 6 Method of Approach Separation 1 Display Format Straight In, In-trail Approach / 2.83 0.52 6 Method of Approach Separation 2 Display Format Simultaneous Arrival Approach / 2.67 0.52 6 Method of Approach Separation 2 Display Format with 95% Confidence Interval Mean AFFTC Workload Rating 2.58 2.67 2.75 2.5 2.83 2.67 Straight In Sim. Arrival Straight In Sim. Arrival Straight In Sim. Arrival Baseline Baseline MAS 1 MAS 1 MAS 2 MAS 2 Approach Type x Display Type Figure F1. Mean Air Force Flight Test Center (AFFTC) Subjective Workload Estimate Scale ratings associated with Approach Type x Display Type .

A Friedman Test performed on these means revealed that equivalent workload ratings were reported to be experienced when different display types were used during the straight-in, in-trail approaches and the simultaneous arrival approaches (X [5] = 2.2519; p = 0.8133).

MCH Workload Ratings Table F6 and Figure F2 contain the means and standard deviations for the MCH workload ratings associated with performing straight-in, in-trail approaches and simultaneous arrival approaches using different display types. Sample size equals 6 since workload ratings were averaged across replicates.

Table F6. Modified Cooper-Harper Workload Ratings Associated with Approach Type x Display Type Test Condition Mean Standard Deviation Sample Size Straight In, In-trail Approach / 2.08 1.07 6 Baseline Display Format Simultaneous Arrival Approach / 2.17 0.98 6 Baseline Display Format Straight In, In-trail Approach / 2.58 0.97 6 Method of Approach Separation 1 Display Format Simultaneous Arrival Approach / 2.17 1.29 6 Method of Approach Separation 1 Display Format Straight In, In-trail Approach / 2.58 0.97 6 Method of Approach Separation 2 Display Format Simultaneous Arrival Approach / 2.08 0.97 6 Method of Approach Separation 2 Display Format with 95% Confidence Interval 2.08 2.17 2.58 2.17 2.58 2.08 Mean Modified Cooper-Harper Workload Rating Straight In Sim. Arrival Straight In Sim. Arrival Straight In Sim. Arrival Baseline Baseline MAS 1 MAS 1 MAS 2 MAS 2 Approach Type x Display Type Figure F2. Mean Modified Cooper-Harper (MCH) workload ratings associated with Approach Type x Display Type.

A Friedman Test performed on these means revealed that equivalent workload ratings were reported to be experienced when different display types were used during the straight-in, in-trail approaches and the simultaneous arrival approaches (X [5] = 5.0000; p = 0.4159).

References [F1] Ames, L. L.; and George, E. J.: Revision and Verification of a Seven-Point Workload Estimate Scale (AFFTC-TIM093-01). Air Force Flight Test Center (Edwards Air Force Base, CA), 1993.

[F2] Wierwille, W. W.; and Casali, J. G.: A Valid Rating Scale for Global Mental Workload th Measurement. Proceedings of the Human Factors Society 27 Annual Meeting, 1983, pp. 129-133.

[F3] Norusis, M. J.: SPSS® for Windows™: Base System User’s Guide, Release 6.0 . SPSS Inc. (Chicago, IL), 1993.

[F4] Winer, B. J.; Brown, D. R.; and Michels, K. M.: Statistical Principles in Experimental Design. Third ed., McGraw-Hill, Inc., 1991.

Form Approved REPORT DOCUMENTATION PAGE OMB No. 0704-0188 The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY) 2. REPORT TYPE 3. DATES COVERED (From - To) 05 - 2005 01- Technical Publication 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER Flight Experiment Investigation of General Aviation Self-Separation and Sequencing Tasks 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER Murdoch, Jennifer L.; Ramiscal, Ermin R.; McNabb, Jennifer L.; and Bussink, Frank J. L.

5e. TASK NUMBER 5f. WORK UNIT NUMBER 23-786-10-10 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT NUMBER NASA Langley Research Center Hampton, VA 23681-2199 L-19101 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S) National Aeronautics and Space Administration NASA Washington, DC 20546-0001 11. SPONSOR/MONITOR'S REPORT NUMBER(S) NASA/TP-2005-213539 12. DISTRIBUTION/AVAILABILITY STATEMENT Unclassified - Unlimited Subject Category 04 Availability: NASA CASI (301) 621-0390 13. SUPPLEMENTARY NOTES An electronic version can be found at http://ntrs.nasa.gov 14. ABSTRACT A new flight operations concept called Small Aircraft Transportation System (SATS) Higher Volume Operations (HVO) was developed to increase capacity during Instrument Meteorological Conditions (IMC) at non-towered, non-radar airports by enabling concurrent operations of multiple aircraft. One aspect of this concept involves having pilots safely self-separate from other aircraft during approaches into these airports using appropriate SATS HVO procedures. A flight experiment was conducted to determine if instrument-rated general aviation (GA) pilots could self-separate and sequence their ownship aircraft, while following a simulated aircraft, into a non-towered, non-radar airport during simulated IMC. Six GA pilots' workload levels and abilities to perform self-separation and sequencing procedures while flying a global positioning system (GPS) instrument approach procedure were examined. The results showed that the evaluation pilots maintained at least the minimum specified separation between their ownship aircraft and simulated traffic and maintained their assigned landing sequence 100-percent of the time. Neither flight path deviations nor subjective workload assessments were negatively impacted by the additional tasks of self-separating and sequencing during these instrument approaches.

15. SUBJECT TERMS SATS; Self-separation; Sequencing; Instrument approaches; Workload 19a. NAME OF RESPONSIBLE PERSON 18. NUMBER 17. LIMITATION OF 16. SECURITY CLASSIFICATION OF: OF ABSTRACT STI Help Desk (email: help@sti.nasa.gov) a. REPORT c. THIS PAGE b. ABSTRACT PAGES 19b. TELEPHONE NUMBER (Include area code) (301) 621-0390 U U U UU Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std. Z39.18