Small UAV Automatic Ground Collision Avoidance System Design Considerations and Flight Test Results

In order to accommodate the extra throughput that is required, some portion of the standard set of data recorded throughout the test mission may need to be sacrificed. Some of those “always recorded” parameters may need to be deleted, or some may need to be recorded at a slower rate. The standard set of parameters as recorded throughout each test flight for the SUAV Auto GCAS project is given in appendix A.

Test Results One participant in the SUAV Auto GCAS test project described the results by stating, “It worked surprisingly well. ” That somewhat cryptic description is actually a good overall summary, because the system did work better than expected given that the entire project was conceived as a low-budget demonstration effort.

There was never the intent that this demonstration effort directly apply to a production implementation.

However, there was always a vision that the results might be good enough to form a basis for follow-on efforts. The results validated the potential for the new technology concepts and opened the door to several follow-on efforts. The overall assessment of SUAV Auto GCAS is divided into the following areas:  Excellent CFIT protection,  Adequate nuisance potential, and  Outstanding modular technologies.

Controlled Flight Into Terrain Protection There were a total of 61 Auto GCAS events that were included in the post-flight analysis process. This includes almost all test runs on which an avoidance maneuver was initiated by Auto GCAS, but does not include PARS-type avoidance maneuvers. These events do not include another three dozen runs from flights 7, 8, 9, and 10, when the telemetry reception was especially poor (before the directional ground antenna was developed).

Valid initiations occurred on 52 of the 61 runs. The nine invalid initiations were induced by residual telemetry problems caused by incorrect setup or aiming of the directional antenna assembly.

The primary measure of CFIT protection was, “How close did the aircraft get to the rocks?” This question was addressed using the minimum AGL value reached during a recovery. All AGL values, including minimum AGL, were obtained by comparing to the NED truth source as described in the “Comparison to Digital Terrain Truth Data” section of appendix B.

Of the 52 valid initiations, 42 resulted in a usable minimum AGL value. Eight cases did not obtain a usable minimum AGL value because the safety pilot diverted the flightpath before reaching the minimum AGL location. In those cases, the safety pilot took control because the ability to judge terrain clearance was degrading (due to a combination of distance and other terrain in the background), not because of a problem with the trajectory of the Auto GCAS avoidance maneuver. Two of the unusable minimum AGL cases were due to FAIL conditions that were modified in later software updates.

Figure 44 shows “ how close the aircraft got to the rocks ” using a histogram for the 42 runs that resulted in a usable minimum AGL value.

Figure 44. Minimum AGL (mountainous and smooth terrain).

Because the test runs were accomplished in a buildup manner, they tended to progress through TCB settings of 200, 100, and 0 ft. In order to show all of the data on the same plot, the TCB was subtracted from the actual minimum AGL value obtained from each run. This effectively shows “ how close the aircraft would have come to the rocks if TCB had been set to zero. ” Most of the runs in figure 44 show that the aircraft cleared the buffered “ rocks ” by 100 ft or more, with a mean of 142 ft. None of the 42 runs penetrated the TCB. This shows that the combination of the scanning techniques and the built-in buffer of 40 ft worked quite well to minimize the potential for terrain impact.

One run would have been within 25 ft of the “ rocks ” if the TCB had been set to zero (the test run had a TCB setting of 100 ft). That particular run was one during which the actual trajectory during the avoidance maneuver went far outside the scan pattern at initiation. It is suspected that this deviation was caused by a significant change in winds after the avoidance was initiated. The effects of non-steady winds were discussed above, and warrant additional consideration on future Auto GCAS systems. This case shows the way in which non-steady wind effects could reduce CFIT protection if future Auto GCAS algorithms do not account for those effects.

The data in figure 44 represent all of the usable minimum AGL data regardless of the type of terrain along the flightpath. Almost all of those runs were over mountainous terrain. Only a few runs were conducted over smooth terrain (a dry lakebed) because that terrain was not considered sufficiently challenging to the system. The two runs over smooth terrain that provided usable minimum AGL data are shown in figure 45. As expected, these runs recovered at a minimum AGL that was just above the built-in buffer of 40 ft.

Figure 45. Minimum AGL (smooth terrain).

The data in figures 44 and 45 indicate that the SUAV Auto GCAS demonstrated excellent CFIT protection given the basic design that was implemented for this demonstration effort.

Nuisance Potential Providing a measure of nuisance potential is more problematic than providing a measure for CFIT protection. Nuisance potential of a generic UAV is a more subjective measure that will almost always depend on the perspective of the individual UAV operator as well as on the mission of a particular UAV.

Most Auto GCAS nuisance evaluations have been accomplished by conducting a series of operationally relevant tasks and asking the pilot to qualitatively evaluate the nuisance potential of any Auto GCAS avoidance maneuvers encountered. That type of subjective evaluation provides a valid overall indication of nuisance potential but does not establish a quantitative boundary for when an avoidance maneuver would be considered a nuisance.

Available Reaction Time As discussed above, nuisance criterion flight tests were conducted on an F-16 in the 1990s. This study was the only known analytical flight-test study specifically planned to develop a criterion for Auto GCAS nuisance potential. The F-16 study was accomplished by allowing pilots to initiate avoidance maneuvers at progressively lower altitudes until each pilot reached a comfort threshold for that maneuver. The end result was a quantitative nuisance criterion for typical low-altitude maneuvering for F-16 missions.

The F-16 nuisance criterion indicated that any Auto GCAS avoidance maneuver initiated more than 1.5 s earlier than necessary to avoid hitting the ground, would be considered a nuisance by an F-16 pilot aware of ground proximity. That time-based metric was considered much more applicable than a distance-based metric. The term “ available reaction t ime” (ART) has been used as one way to describe that time-based metric.

A similar analytical study to develop an Auto GCAS nuisance criterion for UAVs has not been conducted. It is suspected that the operator of a UAV would accept avoidance maneuver activations at higher ART values than the 1.5-s threshold for F-16 pilots. An Auto GCAS nuisance criterion for a UAV would probably depend on a number of additional relevant factors, such as:  The size of the UAV;  The maneuverability of the UAV;  Whether the UAV is remotely piloted or autonomous; and  Whether direct line-of-sight control or a satellite link were being used (which determines the amount of delay in the two-way control link).

Since many of these factors will be platform-dependent, it is probably not appropriate to define a generic nuisance criterion that will work for all UAVs. However, it should be practical and necessary to develop a nuisance criterion that makes sense for each platform by using simulators combined with in-flight pilot experience. It will be essential for Auto GCAS designers on future UAV projects to have some kind of guidance regarding nuisance potential so that design tradeoffs can be made.

Recommendation 13 (R13): Future Auto GCAS projects on UAVs should develop a nuisance criterion specific to that project.

An indicator of the nuisance potential as experienced on the SUAV Auto GCAS project can be obtained by inspection of ART. Available reaction time was defined as the amount of time after initiation of the Auto GCAS maneuver that the same maneuver could have been delayed and barely avoided terrain. The premise of the ART calculation was to determine the amount of time available for a pilot to react if an Auto GCAS maneuver were not initiated. A negative ART indicates that the avoidance maneuver would not prevent the aircraft from flying into the terrain, or, in this case, the terrain plus TCB. A formal study has not yet been accomplished to quantify the ART nuisance boundary for UAVs similar to the test aircraft. The ART was not determined from a qualitative pilot opinion but was based on an extrapolation of the actual aircraft states combined with the trajectory of the actual avoidance maneuver for a given test run. The method for determining ART values on the SUAV Auto GCAS project is described in appendix B.

A total of 43 runs provided usable ART values. A number of runs did not provide usable ART values, usually because the safety pilot took control and ART could not be determined from the limited data available for those runs.

The ART values obtained during SUAV Auto GCAS tests are shown in figure 46. Even though these values cannot be compared to a UAV nuisance criterion, they still provide some helpful insight into the overall conservatism of the SUAV Auto GCAS algorithm and the consistency from run to run.

Figure 46. Available reaction time.

As expected, the ART values shown in figure 46 tend to be higher than the 1.5-s criterion of the F-16.

The mean ART value of 3.5 s might still be quite adequate for a UAV. The highest ART value (approximately 7 s) might be considered a nuisance for UAVs.

UAV Mission Segments Since the DROID was not a production platform having a well-defined mission, operationally relevant tasks could not be selected from an existing list. However, as previously discussed, ridge crossings and valley patrols were the two mission segments identified with some operational relevance to UAV missions.

Only a few of each task type were conducted near the end of flight-testing because these were considered lower-priority tests for this project.

All of the ridge crossing and valley patrol tests were conducted with the TCB set to zero. Higher TCB settings were not used because the intent was to accomplish these runs as if they were portions of an operational mission in which no flight-test buffers would be added. A buildup approach was used by starting new runs approximately 200 ft above the test card minimum safe altitude that would be used on the final run. This approach provided the pilot in the ground cockpit the chance to become familiarized with the visual cues available on the cockpit video display and to get an idea of the overall setup for each run.

Seven ridge crossings were accomplished on flight 20, including two buildup runs that were 200 ft higher than the final runs. The ridge was crossed in both directions (roughly west to east and then east to west). Five runs were accomplished with the Multi-Trajectory option set to ON, providing the Auto GCAS algorithm with the option to use left, straight, or right avoidance maneuvers. All five of those runs crossed the ridge without any indication of the need for an avoidance maneuver. The aircraft crossed the ridge as close as 196 ft AGL.

Two additional runs were accomplished with the Multi-Trajectory option set to OFF. This setting forced the Auto GCAS algorithm to only use the straight trajectory for avoidance maneuvers. One of these runs crossed the ridge at approximately 175 ft AGL, but without an avoidance maneuver.

The second run with the Multi-Trajectory option set to OFF showed an indication of a straight avoidance maneuver, but the aircraft did not react. This miss was quickly traced to a setup problem with ® the Gumstix personal computer. This setup problem did not invalidate the other ridge crossing runs because the main goal was to determine whether nuisance activations would occur. However, this last run with the Multi-Trajectory option set to OFF, did indicate a possible nuisance activation. A pilot assessment would be needed to determine whether that particular avoidance would have been considered a nuisance.

If the aircraft had reacted to the initiation, it is likely that the maneuver would have been very short, and the pilot may not have considered it a nuisance given the low altitude.

An example visualization aid for the ridge crossing with the lowest AGL is shown in figure 47. This figure shows the flightpath followed by the DROID, with the superimposed wingspan of an MQ-9 (the white lines between the red and green “wingtips”) to represent a medium-to-large UAV. This figure provides a good perspective of how closely an MQ-9 could have crossed that ridge without triggering an avoidance maneuver.

Figure 47. Ridge crossing test.

A few valley patrol tasks were attempted on flight 21. Part of flight 21 had been dedicated to the final collision avoidance runs with the smartphone installed on the test aircraft. Since it was already late in the day, the decision was made to only attempt valley patrol tasks with the Multi-Trajectory option set to OFF.

The previous flight had provided confidence that tasks executed with the Multi-Trajectory option set to ON would be unlikely to show any avoidance maneuvers at the planned altitudes (at least 150 ft AGL with no ridge crossings).

The first valley patrol pass was conducted in a counterclockwise direction as a buildup run at 200 ft above the minimum safe altitude shown on the test cards. There were no avoidance maneuver indications on this pass. While attempting to set up for a lower-altitude pass (at the minimum safe altitude), a straight avoidance maneuver occurred. This avoidance maneuver occurred well away from the safety pilot and terrain clearance could not be properly judged from that distance. In this situation the safety pilot took control and maneuvered the aircraft back toward the holding pattern.

The third valley patrol pass was intended to be a clockwise pattern at 200 ft above the minimum safe altitude. However, during the setup another straight avoidance maneuver occurred. This maneuver occurred even further away from the safety pilot than had the avoidance on the first valley patrol pass, so the test team decided not to attempt any more nuisance evaluations on that flight.

The overall observations made during the valley patrol tests confirmed the general expectations. When the Auto GCAS algorithm was constrained to only use the straight trajectory, the occurrence of avoidance maneuvers was not intuitive and could happen at surprising locations. This occurrence was partially because of the very low climb performance that was being modeled (in order to replicate a medium-to-large UAV).

The low climb performance led to avoidance maneuver initiations even when terrain features were several thousand feet away from the test aircraft. The occurrence was made even less intuitive because of the way the P-factor effect was modeled within the Auto GCAS algorithm. As modeled, the P-factor effect caused the “ s traight” trajectory prediction to be a long, curving flightpath to the left. The pilot could not simply look straight ahead to see possible terrain conflicts because the trajectory as adjusted for the P-factor effect could trigger from terrain well to the side and was not easy to visualize.

Another factor that diluted these nuisance evaluations was the setup of the ground cockpit. In an ideal setup, control would be returned directly to the pilot immediately after an avoidance maneuver had completed, but because of the way in which the ground cockpit was set up during these tests, when an avoidance maneuver was complete, control reverted to the GCO. The pilot could resume control by hitting a button on the ground cockpit, but it was not always obvious when that button was active. This setup could have been easily improved, but with cost and schedule impacts to the ground control station that were determined to be out of scope.

Avoidance Maneuver Termination An Auto GCAS nuisance evaluation can also be influenced by the timely termination of avoidance maneuvers. If an avoidance maneuver lasts too long, it will undoubtedly be considered a nuisance.

Conversely, even if a pilot feels an avoidance maneuver may not have been necessary, it may not be viewed as a nuisance as long as the maneuver was very short in duration (pilots have described this occurrence as a “speedbump”) .

During the F-16 Auto GCAS FRRP (ref. 2), a quantitative criterion was used to assess flyup maneuver duration. That criterion was originally developed during F-16 Auto GCAS flight-testing in the 1990s. The resulting criterion specified that a flyup must terminate when the flightpath had cleared the terrain immediately ahead of the aircraft by less than 5 deg of overshoot. This flightpath-based criterion was applicable to the F-16 since it only used straight flyup maneuvers, and the aircraft was usually near wings-level when it cleared significant terrain features. It now appears that the production F-16 version of Auto GCAS will be able to meet that criterion, and related F-16 pilot comments have been very favorable.

However, a similar criterion does not yet exist for the timely termination of UAV avoidance maneuvers.

Recommendation 14 (R14): Future Auto GCAS projects on UAVs should develop a termination timeliness criterion specific to that project.

The SUAV Auto GCAS mechanization added another dimension to a termination assessment because of the options for turning avoidance maneuvers. Since most of the SUAV avoidances were turning maneuvers, the most relevant angular measure was heading change. In addition, UAV pilots may be able to tolerate more angular termination overshoot in terms of heading change than F-16 pilots in terms of flightpath angle.

A termination analysis was accomplished for every run that continued all the way through normal termination (without the safety pilot taking control). Of the 52 valid initiations, 28 continued all the way through normal termination without being interrupted by the safety pilot or experiencing telemetry dropout.

On each run, an ideal termination heading was calculated using a simple extrapolation of the tangent line at each point along the turning avoidance maneuver. The ideal heading was defined as the point at which that tangent line first became clear of terrain. That ideal heading calculation included three frames of persistence, for consistency with the on-board termination calculation. The ideal heading was compared with the actual heading at termination to obtain a delta heading.

The specific methodology is described in appendix B and an example is shown in figure 48. The semi-horizontal black and red lines show the trajectory of the test aircraft prior to and after the avoidance maneuver initiation, respectively. The vertical black and red lines show the update rate at which the algorithm was calculated in that segment. The green lines show tangential extrapolations at key points along the trajectory. Those extrapolations included a vertical component that was based on the climb rate at that point. This example shows how straight tangent lines were projected from the turning maneuver trajectory until all terrain within approximately 3 nm was cleared for three sequential frames. The distance of 3 nm was selected to be consistent with the normal Auto GCAS algorithm, but could be tuned for specific platforms. The projected tangent lines slope up in order to include the climb rate established at that point in the avoidance maneuver.

Figure 48. Ideal termination heading.

Despite the lack of a formally-developed termination criterion for UAVs, a reference of some kind was still needed to help interpret the test results. It was assumed that a UAV pilot would tolerate quite a bit more angular overshoot than the 5-deg flightpath overshoot that worked well for the F-16 pilots. Therefore a “ Good ” termination was defined to be no sooner than the ideal heading, but less than 15 deg past that ideal heading. The results are shown in figure 49.

Figure 49. Termination timeliness: heading.

The data in figure 49 show that there was only one avoidance maneuver that would have been considered “ Good ” using the 0- to 15-deg delta heading criteria. The remaining maneuvers were considerably late or early. This undesirable result can be attributed to the way in which the P-factor effect was used as part of the termination logic.

When the DROID P-factor effect was initially quantified, the decision was made to incorporate that characteristic as part of the trajectory prediction algorithm instead of taking the extra time to try to design a set of autopilot commands to compensate for the P-factor effect. This approach also influenced the termination logic because the straight trajectory (which is a curving trajectory due to the P-factor effect) was used to determine when the aircraft was clear of terrain.

An example of how the P-factor effect induced late terminations is shown in figure 50. The semi-horizontal black and red lines show the trajectory of the test aircraft prior to and after the initiation of the avoidance maneuver (as in figure 48). The green line shows the tangential extrapolation of the ideal termination (three sequential frames clear of terrain). The transition from the red line to the blue line shows the point at which the flight-test maneuver was terminated. The blue line represents the straight trajectory (as influenced by the P-factor effect) that was used by the Auto GCAS algorithm to determine when the avoidance was terminated. Even though the ideal green tangent line for this maneuver was clear of both the near ridgeline and the more distant peaks, that maneuver did not terminate until the blue straight trajectory (as influenced by the P-factor effect) was clear of the terrain. The specific features of the terrain in the test area tended to amplify this effect, but the overall result tended to be a significant delay in termination that would most likely be considered a nuisance, the maneuver lasting much longer than necessary.

Figure 50. Late termination example.

An example of how the P-factor effect induced early terminations is shown in figure 51. The semi-horizontal red line shows the trajectory of the test aircraft during the avoidance maneuver.

The transition from the red line to the blue line shows the point at which the flight-test maneuver was terminated. The orange line shows the tangential extrapolation of the trajectory at that point. The green line shows the tangential extrapolation of the ideal termination (three sequential frames clear of terrain). In this case, the blue curving trajectory (as influenced by the P-factor effect) caused the termination logic to see the gap in terrain before the aircraft was pointed toward that gap, causing the avoidance maneuver to terminate while the aircraft was still pointed at terrain (orange line). In several cases the aircraft did not continue in the turn, which triggered the almost-immediate initiation of additional avoidance maneuvers.

Although not a direct safety hazard, when additional avoidance maneuvers occur immediately after a previous avoidance maneuver has terminated, the result creates an impression of reduced confidence in the Auto GCAS algorithm, and would probably be considered a nuisance as compared to a single, longer avoidance maneuver.

Figure 51. Early termination example.

Given the overall undesirable results, it may not be warranted to include the P-factor effect in the termination logic even if the P-factor effect is considered in the development of the trajectory prediction.

This rationale is based on the principle that an aircraft should be considered clear of terrain when it can roll out of the turn and continue in a straight climb without intersecting the digital terrain.

Recommendation 15 (R15): Future Auto GCAS projects on propeller aircraft should determine if the use of the P-factor effect in the termination logic is warranted on that platform.

Direct pilot evaluations of the termination characteristics were not obtained because nuisance evaluations were only accomplished on two flights and no avoidance maneuvers were initiated when the Multi-Trajectory option was set to ON. In addition, the ground control van mechanization did not return control directly to the pilot, so any qualitative termination evaluation would have been skewed when compared to a more representative mechanization. Pilots flew some of the collision avoidance test points, but the focus was on getting the system to function as desired, and not on a thorough evaluation of termination timeliness.

Termination timeliness was analyzed in terms of delta time in addition to the delta heading analysis.

A “Good” termination was defined to be no earlier than the time at which the ideal heading was achieved, and less than 1.5 s beyond that time (1.5 s correlates to the 15-deg delta heading based on a nominal turn rate of 10 deg/s during these avoidance maneuvers). The results are shown in figure 52.

Figure 52. Termination timeliness: time.

Using the delta time criterion also results in a single maneuver that would have been considered “Good . ” The remaining maneuvers reflect the undesirable effects of the P-factor. The data depicted in figure 52 indicate that termination timeliness was generally between 1-5 s early for left turns, and 1-5 s late for right turns. Two outliers are shown terminating early by approximately 8 and 10 s. These two outliers show that the method for calculating the ideal heading may be susceptible to using instantaneous values for the extrapolated flightpath. Future projects implementing similar analysis techniques should consider using a five-point moving average instead of the instantaneous values (which may be more susceptible to gusting winds).

The overall conclusion from this termination analysis is that the designers for each platform will need to carefully consider the logic to be used for when an avoidance maneuver should be terminated and control returned to the pilot.

Recommendation 16 (R16): Future Auto GCAS projects should consider implementing termination logic that returns control to the pilot when well clear of terrain in the immediate vicinity, but should not be overly conservative for distant peaks.

Trajectory Prediction Accuracy The accuracy of the trajectory prediction can be assessed by answering two fundamental questions:  How well did the trajectory prediction at initiation compare to the flightpath of the actual avoidance maneuver?

 Did the flightpath of the actual avoidance maneuver stay within the scan pattern used at initiation?

These questions are addressed by utilizing the analyzed data from individual test runs as in the example in figure 53 to create the summary data that are depicted in figure 54. A more complete description of the technique that was used is presented in appendix B.

Figure 53. Worst-case trajectory example.

Figure 54. Trajectory prediction accuracy.

Figure 53 shows the worst-case example from all test runs. On this run (flight 18, event 6) the actual flightpath went the furthest outside of the scan pattern on the side closest to terrain. The blue line shows the trajectory prediction at initiation. The green circles show the scan pattern at initiation, with the dashed white line representing the outer edges of that scan pattern. The red line shows the actual flightpath during the avoidance maneuver. The yellow dot identifies the point where the actual flightpath went the furthest outside the scan pattern.

The situation shown in figure 53 is undesirable because the aircraft could have flown close to terrain features that might not have been detected by the scan pattern. In this case, the resulting minimum AGL was 125 ft, but that test point had been executed with the TCB set to 100 ft. If the TCB had been set to zero, terrain clearance would have been approximately 25 ft. This was the lowest minimum AGL for any test run, and it was considerably lower than the more typical clearance at 100 ft or greater.

In a production Auto GCAS implementation this characteristic could degrade CFIT protection. Limited analysis indicates that this worst-case example was probably induced by changing winds during the avoidance maneuver. The onboard wind calculations from the Piccolo II autopilot showed a significant shift in wind magnitude and direction during the maneuver. However, those onboard wind calculations become more suspect when the aircraft is not in straight-and-level flight. If confirmed, this characteristic warrants additional effort to account for changing winds through increased buffers, a wider scan pattern, or some other method.

Recommendation 17 (R17): Future Auto GCAS projects should determine if changing winds during the recovery should be addressed in the design for that platform.

A case-by-case analysis of each test run is useful, but it is even more helpful to look at summary data that encompass all of the relevant test results. The individual run analysis technique presented in figure 53 was applied to all relevant test runs to obtain the summary data depicted in figure 54. Each symbol in figure 54 represents a single test run at the point along the actual flightpath that was furthest away from the trajectory prediction.

Although the scan pattern size and shape varied based on airspeed, wind, and other factors, the data were normalized to a generic scan pattern by using a percentage deviation from the centerline trajectory prediction. At any given distance along the centerline, 100 percent was located on the scan pattern line (perpendicular to the centerline) and 50 percent was located halfway between the centerline and the scan pattern line.

The normalized data depicted by figure 54 also include test runs that resulted in right avoidance maneuvers (shown with closed symbols) even though the generic scan pattern shown is for left maneuvers (shown with open symbols). The data for right maneuvers was included by flipping the x-axis for those test runs. This method maintained the relative position of each dot in the sense that any symbols below the centerline in figure 54 were inside the turn, whereas any symbols above the centerline were outside the turn. The specific normalization analysis techniques are described in appendix B.

Figure 54 addresses the two fundamental questions about the accuracy of the trajectory prediction. The summary data given in figure 54 show that most of the test runs flew reasonably close to the centerline of the trajectory prediction, and most of the worst-case locations were within the scan pattern. Another way of showing the same trend is presented in figure 55.

Figure 55. Trajectory prediction histogram: across the scan pattern.

Figure 55 shows the same data from figure 54 represented by a histogram indicating the worst-case position as a percentage of the scan width relative to the centerline. Almost half of the worst-case positions were less than 40 percent of the scan pattern width. Most runs stayed with the scan pattern for the entire avoidance maneuver. Only a few runs strayed outside the scan pattern (as indicated on the histogram for values greater than 100 percent).

Only one case went beyond the scan pattern on the outside of the turn (closer to terrain). That run was at 132 percent relative to the outer scan pattern line and occurred during event 6 of flight 18 (fig. 53).

Three cases went beyond the scan pattern on the inside of the turn (farther from terrain). These three runs ranged from 117 percent to 197 percent relative to the inner scan pattern line. Similar to event 6 of flight 18, it is suspected that variable winds during these maneuvers contributed to flying beyond the scan pattern. These cases do not necessarily indicate degraded CFIT protection, but there could be increased nuisance potential if this situation is not adequately addressed in a production implementation. The only situation in which these three cases might result in degraded CFIT protection would be if the aircraft were descending into terrain that happened to enter the scan pattern inside of the predicted avoidance maneuver (at the same time as when the other avoidance directions were unavailable). That situation may be possible but is probably unlikely.

Figure 56 shows the same data from figure 54, represented by a histogram indicating where the worst-case locations occurred relative to the distance (range from initiation) along the centerline. This histogram shows that most of the worst-case locations were within the first third of the total scan length.

The same trend can be seen in figure 54. This does not imply that the scan length was longer than necessary.

The avoidance maneuvers for these particular test situations were completed relatively quickly, and therefore any trajectory errors did not have time to grow larger. Different test situations could lead to trajectories that last longer and therefore require the longer scan length.

Figure 56. Trajectory prediction histogram: along the centerline.

Scan Pattern Functionality The functionality of the scan pattern can be assessed by answering two fundamental questions:  1. What was the location of the terrain post that triggered the avoidance maneuver (relative to the scan pattern)?

 2. Did the flightpath of the actual avoidance maneuver stay within the scan pattern used at initiation? (Note: this is the same question that was used to assess the trajectory prediction accuracy; the trajectory prediction accuracy and scan pattern functionality modular components are inter-related.)

These questions are addressed by utilizing the analyzed data from individual test runs similar to the example presented in figure 57 to create the summary data that are depicted in figure 58. The individual run in figure 57 is the same run that was shown in figure 53. Additional data is presented in figure 54.

The terrain post that triggered the avoidance maneuver is represented by the orange balloon in figure 57.

The method used to determine the specific location of that trigger post for a particular run is described in appendix B. In the SUAV Auto GCAS algorithm, the terrain posts were not used directly, but rectangular areas around each post were created at the same height as that post. The rectangular area is represented by the orange box in figure 57.

Figure 57. Scan pattern functionality.

An avoidance maneuver was triggered as soon as one of the rectangular terrain polygons was within one of the green scanning circles at a height above the trajectory prediction (including any built-in or flight-test altitude buffers added to the terrain height).

In figure 57 the orange terrain polygon was within three of the green scan circles. In this case, the buffered terrain height was above the trajectory prediction at all three of those scan circles, so the avoidance maneuver was triggered by whichever was first in the internal sequence of checks. One frame earlier (approximately one-fifth of one second), the path of the trajectory prediction was far enough away from that particular terrain polygon/post so that an avoidance maneuver was not triggered.

The main point to be gleaned from figure 57 is that the individual terrain post that triggered this flyup maneuver was outside the scan pattern. As will be seen in figure 58, this result was fairly common, and the terrain scanning approach was implemented as a conscious part of the design process to ensure that terrain posts were not missed.

The summary data presented in figure 58 were constructed using the same type of normalization process as applied to those data depicted in figure 54. As can be seen, almost all of the trigger posts were outside the scan pattern (on the side closer to terrain). A small number of trigger posts were within the scan pattern, and one was outside the scan pattern on the inner side of the turn. These occurrences were generally when the test aircraft was in a dive, so that the scan pattern or trajectory prediction descended down onto the highest terrain post, as compared to the more typical case in which the test aircraft flew in level flight toward the trigger post.

Figure 58. Trigger post locations.

The mean for trigger post locations was at 175 percent relative to the scan width for the outer scan line.

The trigger posts were usually outside the scan pattern because of the scan technique that included any portion of a terrain rectangle within the scan pattern (fig. 28). There was no significant difference in trigger post location for left or right avoidance maneuvers.

The trigger post with the greatest percentage outside the scan pattern was at 366 percent relative to the scan width for the inner scan line. That seemingly extreme case was from event 5 of flight 18, which was a straight activation from a diving maneuver over smooth terrain. That straight activation reflected the general trend expected from diving maneuvers over smooth terrain: the straight trajectory prediction climbs at 1000 fpm whereas the left and right predictions climb at 800 fpm. Over smooth terrain the left and right trajectories will be lower and intersect with smooth digital terrain before the straight trajectory, resulting in a straight trajectory as the last viable option.

In addition, the same trend resulted in trigger posts quite close to the initiation point because the straight trajectory prediction curved upwards quite rapidly compared to the essentially flat digital terrain. When the trigger post occurred close to the initiation, the very narrow scan pattern detected the tiny corner of a terrain rectangle generated from a post that was well outside the scan pattern compared to the scan width.

Figure 59 shows the same data from figure 58, represented by a histogram indicating the trigger post locations as a percentage of the scan width relative to the centerline. The data in figure 59 resulted in a mean value of 175 percent scan width and a standard deviation of 71 percent. The straight activation at 366 percent was an outlier and was not included in this histogram, but the value was included in the calculation of the mean.

Figure 59. Trigger post location histogram: across the scan pattern.

Figure 60. Trigger post location histogram: along the centerline.

Figure 60 shows the same data from figure 58, represented by a histogram indicating the worst-case locations relative to the distance (range from initiation) along the centerline. The data in figure 60 resulted in a mean value of 38 percent range along the centerline and a standard deviation of 17 percent.

Avoidance Maneuver Summary This section describes the general characteristics of the avoidance maneuvers encountered during this project. The discussion includes the flight conditions at initiation, the direction in which the algorithm commanded the maneuver, and the overall duration of the maneuver (in terms of time and heading change).

The basic flight conditions at initiation are shown in figures 61 and 62. Figure 61 shows that the DROID speed range of 40 to 80 KIAS was well-covered by test conditions. There were two main target airspeeds on the test cards: 45 and 70 KIAS. However, the normal variations induced by autopilot control and piloted maneuvering provided the additional spread around the two main airspeeds. Most of the initiations were in relatively level flight, but a few shallow dives and climbs were also accomplished. The mishap cases that were replicated tended to be at the higher end of the DROID speed range, because those mishaps occurred on the MQ-1 and the MQ-9.

Figure 61. Initiation flightpath versus airspeed.

Figure 62. Initiation flightpath versus bank angle.

Figure 62 shows the bank angle variations at initiation. Most of the runs were near wings-level, but a few were initiated at approximately 30-40 deg bank. The three mishap cases targeted 0 deg, 15 deg, and 25 deg of bank. Although one test card sequence targeted negative bank angle, that series was only attempted on flight 17 and the resulting data were not usable due to poor telemetry on that flight.

Almost all of the avoidance maneuvers were to the left or the right, as shown in figure 63. The data in figure 63 include all valid initiations, even if the safety pilot took control later.

Figure 63. Avoidance maneuver direction.

Only two test setups resulted in straight avoidance maneuvers with usable data. Straight avoidance maneuvers were only expected to occur over smooth terrain, or possibly during ridge crossings. A number of smooth-terrain test points were attempted, but most of those occurred on missions with poor telemetry and resulted in unusable data. The Auto GCAS algorithm initiated one straight avoidance maneuver during ® a ridge crossing on flight 20, but the Gumstix personal computer was not set up correctly on that mission and the autopilot did not respond to the commanded maneuver.

The longest avoidance maneuver was approximately 15 s and the shortest approximately 1 s, as shown in figure 64. The duration was a simple calculation that began when the Auto GCAS algorithm initiated the avoidance maneuver and ended when the algorithm determined the maneuver was complete. The data in figure 64 do not include runs in which the safety pilot took control prior to normal termination.

Figure 64. Avoidance maneuver duration.

There was considerable variation in avoidance maneuver duration with no discernible pattern. This variation was not surprising, because the duration was solely driven by the amount of time needed for the aircraft to clear the terrain. Many of the test setups were intentionally targeted at a bowl-shaped canyon in order to stress the system. That bowl- shaped canyon tended to force the aircraft to “go back toward the way it had come, ” resulting in quite a few avoidance maneuvers each lasting longer than 10 s. The duration of avoidance maneuvers was also influenced by the way the P-factor effect was modeled in the termination logic.

The amount of heading change that occurred during avoidance maneuvers is shown in figure 65. The data in figure 65 only include turning maneuvers, and only include runs that continued to normal termination before the safety pilot took control.

Figure 65. Avoidance maneuver heading change.

The effect of the P-factor bias can be seen in figure 65. Left turns had a mean heading change of approximately 80 deg, whereas right turns had a mean heading change of approximately 110 deg. Left turns tended to have less heading change and right turns more heading change. It was not unusual for avoidance maneuvers to continue for 90 deg heading change or more.

Concluding Remarks

The small unmanned aerial vehicle (SUAV) Automatic Ground Collision Avoidance System (GCAS) project successfully demonstrated many important collision avoidance technologies. Foremost among these demonstrations were:  Auto GCAS testing with multiple avoidance maneuvers including turns to either side;  Testing of digital terrain scanning techniques determined directly from the predicted trajectory;  In-flight testing of highly compressed digital elevation models;  In-flight testing of digital elevation models that had been customized to reflect tighter tolerances in some areas and relaxed tolerances in other areas;  In-flight testing of Auto GCAS on an unmanned aerial vehicle; and  Hosting Auto GCAS algorithms on a smartphone during flight tests.

Additional noteworthy accomplishments included:  Design and implementation of a flight-test user interface that enabled ground operators to control the Auto GCAS algorithms on the smartphone (either with the smartphone on the ground or on board the test aircraft);  Rapid design and implementation of a directional antenna system to avoid multi-path noise when in the vicinity of terrain, which greatly improved communications between the test aircraft and the ground control van;  Derivation of Dryden Remotely Operated Integrated Drone trajectory prediction models purely from very limited flight-testing of the specific avoidance maneuvers on the test aircraft, without relying on any type of simulation;  Innovative use of Google Earth to enable pre-flight visualizations of Auto GCAS setups, enhance test-card generation, and greatly improve post-flight data analysis of test maneuvers; and  Auto GCAS algorithms coded into Java ™ (Oracle Corporation, Redwood Shores, California) (the native language for the smartphone).

Although the SUAV Auto GCAS implementation was not intended for production, the test results are positive enough to provide a solid basis for scaling onto many production UAV platforms. The same basic SUAV Auto GCAS concepts should also be adaptable to similar aircraft types having relatively low airspeeds and low maneuverability (typical of general aviation).

Some of the concepts have demonstrated the potential to provide significant capabilities for Auto GCAS implementations on higher-airspeed, higher-maneuverability aircraft such as fighters, transports, business jets, and airliners. These concepts include the use of highly-compressed DEMs, multi-trajectory avoidance options, and terrain scanning techniques.

Auto GCAS follow-on projects could include:  Implementation of GCAS algorithms on a smartphone or tablet for general aviation platforms.

General aviation applications could be automated when a digital autopilot is available, but significant improvements to protection against controlled flight into terrain (CFIT) might also be obtained when a digital autopilot is not available by using the same GCAS algorithms as part of a ground proximity warning system utilizing manual pilot reactions. Any automatic implementation of GCAS using a smartphone or tablet would need to pay particular attention to the redundancy and reliability aspects in order to avoid violation of the “ do no h arm” principle . General aviation smartphone or tablet implementations should be evaluated first in a simulation environment and then in flight-testing.

 Flight-test demonstration of GCAS algorithms for helicopter platforms. It is expected that helicopters could also benefit from automatic GCAS or a GCAS-based ground proximity warning system. Because of the additional avoidance maneuver options available to a helicopter (stop and hover, reverse direction, et cetera) additional development would be required to properly evaluate GCAS tradeoffs.

 Analytical studies and flight-test demonstrations to evaluate the effectiveness of new Auto GCAS concepts for fighter platforms. Even though a very capable production version of Auto GCAS is nearing deployment to the F-16 fleet, variations of Auto GCAS are still in development for the F-22 and F-35. Some of the techniques implemented on the SUAV Auto GCAS could provide significant improvements to the tradeoff between CFIT protection and nuisance potential for any of those airplanes.

 Analytical studies and flight-test demonstrations to evaluate the effectiveness of new Auto GCAS concepts for transport platforms. Transport platforms have limited maneuverability compared to fighter platforms, but also have much higher airspeed envelopes compared to an SUAV. This characteristic might lead to the need for adaptations of existing GCAS technologies that would be appropriate to explore in a flight-test development environment.

Summary of Recommendations

Recommendation 1 (R1): Future Auto GCAS projects should consider applying resources to develop improved integration of in-flight wind estimates with the Auto GCAS algorithm (page 19).

Recommendation 2 (R2): Future Auto GCAS projects should consider implementing a monitor to isolate and protect against corrupted digital terrain data (page 26).

Recommendation 3 (R3): Future Auto GCAS projects should consider implementing a monitor to protect against false requests for avoidance maneuvers (page 26).

Recommendation 4 (R4): Future Auto GCAS projects should pay special attention to the input signal conditioning necessary for that particular implementation (page 28).

Recommendation 5 (R5): Future Auto GCAS projects should consider incorporating multiple trajectory predictions to provide more than one option and to reduce nuisance potential (page 30).

Recommendation 6 (R6): Future Auto GCAS projects for performance-limited aircraft may need to consider including density altitude as an input to the trajectory prediction (page 36).

Recommendation 7 (R7): The developers on any Auto GCAS project should consider the addition of vertical obstructions as part of the algorithm (page 39).

Recommendation 8 (R8): Future Auto GCAS projects should carefully assess the tradeoffs between short-term PVI cost savings against the potential for longer-term impacts on the user (page 49).

Recommendation 9 (R9): Future Auto GCAS projects involving flight-testing of UAVs should pay particular attention to telemetry and control links when operating in close proximity to terrain (page 49).

Recommendation 10 (R10): Future Auto GCAS projects on UAVs should consider a mode state implementation that allows the avoidance to continue even after reaching a FAIL state (page 50).

Recommendation 11 (R11): Future Auto GCAS projects on UAVs should consider a self-recovering mode state implementation to resume CFIT protection as soon as the cause for the FAIL state no longer exists (page 50).

Recommendation 12 (R12): Future Auto GCAS test projects using a smartphone as the data recording device should consider implementing a recording method that provides data throughout the flight, not just when avoidance maneuvers occur (page 67).

Recommendation 13 (R13): Future Auto GCAS projects on UAVs should develop a nuisance criterion specific to that project (page 71).

Recommendation 14 (R14): Future Auto GCAS projects on UAVs should develop a termination timeliness criterion specific to that project (page 74).

Recommendation 15 (R15): Future Auto GCAS projects on propeller aircraft should determine if the use of the P-factor effect in the termination logic is warranted on that platform (page 78).

Recommendation 16 (R16): Future Auto GCAS projects should consider implementing termination logic that returns control to the pilot when well clear of terrain in the immediate vicinity, but should not be overly conservative for distant peaks (page 79).

Recommendation 17 (R17): Future Auto GCAS projects should determine if changing winds during the recovery should be addressed in the design for that platform (page 80).

Appendix A: Parameter List

The main purpose in presenting the parameter list is to provide future project teams with a starting point for consideration when they are developing their own project-specific parameter list. Table A1 provides a complete list of the parameters that were available to the Auto GCAS test team. All of the parameters that were used for post-flight analysis were recorded on the smartphone and downloaded after the flight. The post-flight analysis parameters were recorded on the smartphone regardless of its location (whether in the ground control van or in the DROID). Additional telemetered data were available in the ground control van, but those data were not used for post-flight analyses and so are not included in table A1. Additional data were also recorded on the user interface (UI) laptop computer, but those data were not used for post-flight analyses and so have not been included in table A1.

Three sources provided the data that were recorded on the smartphone: the Piccolo II autopilot; the ® UI laptop computer (by way of uplinked telemetry or the Gumstix personal computer); and the Auto GCAS algorithm on the smartphone. Selected parameters from the Piccolo II autopilot included basic aircraft state information such as latitude, longitude, airspeed, altitude, angular rates, linear accelerations, et cetera, along with autopilot status and target values and the winds as estimated by the standard Piccolo II autopilot software. The UI parameters consisted primarily of state values that could be set by the UI Operator: Terrain Clearance Buffer; horizontal and vertical uncertainties (for the built-in buffers); the flag to switch between multiple avoidance trajectories and the straight-only trajectory, and others. The data from the Auto GCAS algorithm on the smartphone included Auto GCAS modes, time to flyup for each of the three trajectories, and minimum approach to terrain for each of the three trajectories, along with some Auto GCAS status words.

A number of parameters were not directly available from any of the sources but could be calculated using the available parameters. Those calculated parameters are listed near the end of table A1.

The parameter names in table A1 are listed in two columns: “Smartphone parameters,” and “AUTO GCAS maneuver - specific parameters.” Most of the parameter names are the same in both of these columns. In a few c ases the parameter existed in only one form, indicated in table A1 by the entry “N/A.” in the relevant parameter column. The difference only applied to the type of post-flight analysis being conducted.

Table A1. Small Unmanned Aerial Vehicle Automatic Ground Control Avoidance System parameters.

Smartphone parameters AUTO GCAS Units Description Source Data maneuver - specific parameters rate time_system time_system ms Time since start - up Piccolo II 5 Hz flt N/A N/A Flight number Algorithm 5 Hz exe exe N/A Execute flag (1 or 0) Smartphone 5 Hz latitude latitude deg GPS latitude Piccolo II 5 Hz longitude longitude deg GPS longitude Piccolo II 5 Hz GPS altitude above the altitude_gps_wgs84 altitude_gps_wgs84 ft Piccolo II 5 Hz WGS84 ellipsoid ktas ktas kn Calculated true airspeed Piccolo II 5 Hz kias kias kn Indicated airspeed (raw) Piccolo II 5 Hz bankAngle bankAngle deg Bank (roll) angle Piccolo II 5 Hz climbRate climbRate ft/s Climb rate Piccolo II 5 Hz Roll rate, as read from rollRate rollRate deg/s Piccolo II 3 - axis Piccolo II 5 Hz gyroscope heading heading deg True heading Piccolo II 5 Hz Wind velocity, north windVelNorth windVelNorth ft/s (parameter is positive with Piccolo II 5 Hz wind from south) Wind velocity, east windVelEast windVelEast ft/s (parameter is positive with Piccolo II 5 Hz wind from west) horizUncertainty horizUncertainty ft Horizontal uncertainty UI laptop 5 Hz vertUncertainty vertUncertainty ft Vertical uncertainty UI laptop 5 Hz TCB TCB ft Terrain Clearance Buffer UI laptop 5 Hz rollRateLag rollRateLag N/A Roll rate lag UI laptop 5 Hz airspeedLag airspeedLag N/A Airspeed lag UI laptop 5 Hz climbRateLag climbRateLag N/A Climb rate lag UI laptop 5 Hz multiAvoidReq multiAvoidReq N/A Multi - avoid request UI laptop 5 Hz acDataValid acDataValid N/A AC data valid flag UI laptop 5 Hz AGCASmodeReq AGCASmodeReq N/A AGCAS mode request UI laptop 5 Hz AGCAS mode as reported AGCASmode AGCASmode N/A Algorithm 5 Hz by smartphone AGCAS autopilot apCmd apCmd N/A Smartphone 5 Hz command request AGCAS time to flyup, strTime2Flyup strTime2Flyup s Smartphone 5 Hz straight leftTime2Flyup leftTime2Flyup s AGCAS time to flyup, left Smartphone 5 Hz AGCAS time to flyup, rightTime2Flyup rightTime2Flyup s Smartphone 5 Hz right AGCAS Minimum minApprTerrainStr minApprTerrainStr ft Approach to Terrain, Smartphone 5 Hz straight AGCAS Minimum minApprTerrainLeft minApprTerrainLeft ft Smartphone 5 Hz Approach to Terrain, left AGCAS Minimum minApprTerrainRight minApprTerrainRight ft Smartphone 5 Hz Approach to Terrain, right errorCode errorCode N/A AGCAS error code Algorithm 5 Hz warningCode warningCode N/A AGCAS warning code Smartphone 5 Hz infoCode infoCode N/A AGCAS info code Smartphone 5 Hz Local map reference, lmRefLatitude lmRefLatitude deg Smartphone 5 Hz latitude Local map reference, lmRefLongitude lmRefLongitude deg Smartphone 5 Hz longitude time_gps_hours time_gps_hours hr GPS time Piccolo II 5 Hz time_gps_minutes time_gps_minutes min GPS time Piccolo II 5 Hz time_gps_seconds time_gps_seconds s GPS time Piccolo II 5 Hz altitude_baro altitude_baro ft Barometric altitude, MSL Piccolo II 5 Hz laserAlt laserAlt ft Laser altitude, AGL Piccolo II 5 Hz RPM RPM rpm Revolutions per minute Piccolo II 5 Hz mag_hdg_deg mag_hdg_deg deg Magnetic heading Piccolo II 5 Hz pitch_deg pitch_deg deg Pitch Piccolo II 5 Hz yaw_deg yaw_deg deg Yaw Piccolo II 5 Hz pitch_rate_dps pitch_rate_dps deg/s Pitch rate Piccolo II 5 Hz yaw_rate_dps yaw_rate_dps deg/s Yaw rate Piccolo II 5 Hz xaccel_g xaccel_g g Acceleration, x - direction Piccolo II 5 Hz yaccel_g yaccel_g g Acceleration, y - direction Piccolo II 5 Hz zaccel_g zaccel_g g Acceleration, z - direction Piccolo II 5 Hz Indicated airspeed loop Piccolo II LoopTarget0_kts LoopTarget0_kts kn 5 Hz target LoopTarget1_ft LoopTarget1_ft ft Altitude loop target Piccolo II 5 Hz LoopTarget2_deg LoopTarget2_deg deg Bank loop target Piccolo II 5 Hz LoopTarget3_deg LoopTarget3_deg deg Flaps loop target Piccolo II 5 Hz LoopTarget4_deg LoopTarget4_deg deg Heading loop target Piccolo II 5 Hz LoopTarget5_deg LoopTarget5_fpm ft/min VRate loop target Piccolo II 5 Hz Autopilot global on/off Piccolo II flag. Indicates whether or not SUAV was flown by AP_Global AP_Global N/A ground control 5 Hz operator(s), or whether or not a safety pilot had control.

TrackerStatus TrackerStatus N/A Tracker status Piccolo II 5 Hz Indicated airspeed loop Piccolo II LoopStatus0 LoopStatus0 N/A 5 Hz status LoopStatus1 LoopStatus1 N/A Altitude loop status Piccolo II 5 Hz LoopStatus2 LoopStatus2 N/A Bank loop status Piccolo II 5 Hz LoopStatus3 LoopStatus3 N/A Flaps loop status Piccolo II 5 Hz LoopStatus4 LoopStatus4 N/A Heading loop status Piccolo II 5 Hz LoopStatus5 LoopStatus5 N/A VRate loop status Piccolo II 5 Hz 10 s average of parameter windVelNAve windVelNAve ft/s UI laptop 5 Hz windVelNorth 10 s average of parameter windVElEAve windVelEAve ft/s UI laptop 5 Hz windVelEast triggerPressed triggerPressed N/A Trigger pressed UI laptop 5 Hz Pilot Activated Recovery PARSstate PARSstate N/A UI laptop 5 Hz Switch state Pilot Activated Recovery PARSengaged PARSengaged N/A UI laptop 5 Hz Switch engaged flag Flyup active flag. Boolean 1 (flyup active) or 0 (flyup flyupActive flyupActive N/A UI laptop 5 Hz not active). Same as parameter flyup_active.

noTMcount noTMcount N/A Telemetry failure flag Smartphone 5 Hz flyupHold flyupHold N/A Flyup hold flag Smartphone 5 Hz flyupHoldCount flyupHoldCount N/A Flyup hold count Smartphone 5 Hz North velocity (north - east - Piccolo II Vnorth N/A ft/s 5 Hz down frame) East velocity (north - east - Piccolo II Veast N/A ft/s 5 Hz down frame) Down velocity (north - Vdown N/A ft/s Piccolo II 5 Hz east - down frame) Same as parameter Clock_ms Clock_ms ms Calculated N/A time_system Same as parameter Hours Hours hr Calculated N/A time_gps_hours Same as parameter Minutes Minutes min Calculated N/A time_gps_minutes Same as parameter Seconds Seconds s Calculated N/A time_gps_seconds time_sec time_sec s Time, starting at zero Calculated N/A time_sec_since_midnight time_sec_since_midnight s Time since midnight Calculated N/A Time since midnight, time_sec2 time_sec2 s Calculated N/A starting at zero time_irig time_irig s Time, IRIG style Calculated N/A Change in time between time_delta time_delta s Calculated N/A data points Change in time between time_delta2 time_delta2 s data points, starting at Calculated N/A zero LeftRPM LeftRPM rpm Same as parameter RPM Calculated N/A Same as parameter latitude_deg latitude_deg deg Calculated N/A latitude Same as parameter longitude_deg longitude_deg deg Calculated N/A longitude Same as parameter height_ft height_ft ft Calculated N/A altitude_gps_wgs84 Same as parameter direction_deg direction_deg deg Calculated N/A heading Same as parameter alt_ft alt_ft ft Calculated N/A altitude_baro tas_kts tas_kts kn Same as parameter ktas Calculated N/A Same as parameter roll_rate_dps roll_rate_dps deg/s Calculated N/A rollRate Same as parameter roll_deg roll_deg deg Calculated N/A bankAngle Same as parameter agl_ft agl_ft ft Calculated N/A laserAlt Same as parameter windsouth_fps windsouth_fps ft/s Calculated N/A windVelNorth Same as parameter windwest_fps windwest_fps ft/s Calculated N/A windVelEast Same as parameter pars_engage pars_engage N/A Calculated N/A PARSengaged ias_kts ias_kts kn Same as parameter kias Calculated N/A Same as parameter climb_rate_fps climb_rate_fps ft/s Calculated N/A climbRate Parameter climbRate, climb_rate_fpm climb_rate_fpm ft/min Calculated N/A multiplied by 60.0 Flyup active flag as set by UI. Boolean 1 (flyup flyup_active flyup_active N/A active) or 0 (flyup not Calculated N/A active). Same as parameter flyupActive GPS altitude, above mean altitude_gps_msl altitude_gps_msl ft Calculated N/A sea level Text string used in N/A type N/A determining where flyups Calculated N/A occur in data N/A event N/A Event number Calculated N/A Altitude, AGL (used for N/A alt_AGL_ft ft comparison with laser Calculated N/A altimeter data) Altitude, NED, using N/A alt_NED_wgs84_ft ft Calculated N/A WGS84 ellipsoid Altitude, NED, using N/A alt_NED_wgs84_m m Calculated N/A WGS84 ellipsoid N/A GPSVelNorth ft/s GPS velocity, north Calculated N/A N/A GPSVelEast ft/s GPS velocity, east Calculated N/A N/A GPSVelDown ft/s GPS velocity, down Calculated N/A N/A deltaDistN ft Change in distance, north Calculated N/A N/A deltaDistE ft Change in distance, east Calculated N/A N/A deltaDistD ft Change in distance, down Calculated N/A N/A dt_sec s Differential time Calculated N/A N/A GPSVel_fps ft/s GPS velocity, total Calculated N/A N/A GPSVel_kts kn GPS velocity, total Calculated N/A N/A GPSAccelNorth_fpsps ft/s GPS acceleration, north Calculated N/A N/A GPSAccelEast_fpsps ft/s GPS acceleration, east Calculated N/A N/A GPSAccelDown_fpsps ft/s GPS acceleration, down Calculated N/A N/A ART s Available reaction time Calculated N/A N/A FPA deg Flightpath angle Calculated N/A Number of degrees turned N/A deltaTermAz deg Calculated N/A until termination occurs Time until termination N/A deltaTermTime N/A Calculated N/A occurs

Appendix B: Analysis Techniques

Most of the analyses presented in this report consist of straightforward presentations of the data and require no further elaboration. A few of the analysis methods were unique, however, warranting some additional background description to help future project teams adapt those methods to their own needs.

Those unique analysis techniques were:  Google Earth interface;  Available reaction time;  Determination of trigger post locations;  Determination of worst-case mismatch locations;  Trajectory normalization analysis; and  Termination logic cross-checks.

Google Earth Interface One of the tools used by the Small Unmanned Aerial Vehicle (SUAV) team for planning test missions and analyzing flight-test data was Google Earth. Google Earth provided a markup language known as keyhole markup language, or KML. (Keyhole Corp., Mountain View, California, was a technology company purchased by Google in 2005). Keyhole markup language allows users to plot locations, lines, and polygons within Google Earth for visualization purposes. This same markup language enables Google Earth to render images of three-dimensional buildings, text, and other visualization aids. For the purpose of analyzing SUAV flight test data, KML proved to be an extremely useful data visualization tool for several reasons:  Raw flight-test data (SUAV latitude, longitude, and elevation) were plotted using Google Earth, which readily allowed users to see where the Dryden Remotely Operated Integrated Drone (DROID) airplane flew relative to the nearby terrain features.

 Algorithm data were also plotted using Google Earth. Algorithm data include trajectory predictions, scan patterns, and representative digital elevation model (DEM) data. Plotting algorithm data using Google Earth allowed users to determine the effectiveness of the collision avoidance algorithms.

 The Auto GCAS team was able to use Google Earth to visualize Auto GCAS termination predictions. Such visualizations would have been much less useful and far more difficult without Google Earth.

 The team was able to use Google Earth to visualize an available reaction time (ART) by extending the actual avoidance trajectory to where the representative DEM data were located. The ART was useful for determining a degree of “nuisance potential,” as discussed above within the main body of this report.

During the analysis of the SUAV flight-test data, various software routines were implemented to write KML files for use with Google Earth. In order to process KML files for use with Google Earth, the analysis ® team used MATLAB (The MathWorks, Natick, Massachusetts). The MATLAB utility was the tool of choice because it provided a simple interface with which users could easily write scripts which, when executed, would write KML files. Of particular interest was the MATLAB Mapping Toolbox™, which contained several geodetic calculation functions. The geodetic calculation functions were used to calculate trajectories on the WGS84 ellipsoid mathematical model of the surface of the Earth.

® Excel Test Planning Tool ® The SUAV Auto GCAS test point setups and test cards were produced using an Excel (Microsoft Corporation, Redmond, Washington) worksheet, a sample of which is shown in figure B1, which implemented both the collision avoidance algorithms (similar to those used on the smartphone) and the DEM data set as needed for the SUAV Auto GCAS flight-testing. The smartphone contained DEM data for ® the entire Earth, but the Excel worksheet needed DEM data only for the local test areas.

® Figure B1. Excel test planning application.

® The Excel worksheet provided a very useful, basic simulation of the end-to-end Auto GCAS ® algorithm. The Excel methodology had been used successfully to support a number of F-16 airplane ® (Lockheed Martin, Bethesda, Maryland) Auto GCAS projects dating back to the early 1990s, and the Excel implementation was also used successfully in support of this SUAV Auto GCAS project. Future related projects should take into consideration that any Auto GCAS simulation method will need to have ® ® functionality similar to the Excel implementation described herein. The combination of the Excel worksheet simulation and Google Earth was new for the SUAV Auto GCAS project. That combination of the two was particularly effective for evaluating and selecting potential flight-test setups. The ability to feed simulation results into Google Earth should be a consideration for future Auto GCAS projects.

® The user interface of the Excel test planning application is shown in figure B1. The user could enter all of the flight-test point setup parameters (the red numbers shown in figure B1), and then use the graphical user interface buttons (the buttons within the blue fields, also shown in figure B1) to plot collision avoidance algorithm results using Google Earth.

The test planning process typically began by choosing a simple start location (latitude, longitude, and altitude) using Google Earth. A target terrain feature and a desired heading for approaching that terrain ® feature were visually selected. The Excel - Google Earth combination was then used to refine the setup to obtain the desired conditions at collision avoidance initiation. The remaining parameters KCAS (knots calibrated airspeed); dive; bank; TCB (terrain clearance buffer); et cetera were entered and the “Run Time” along the desired heading was varied until the proximity to terrain indicated the need for an avoidance ® maneuver. Given all of the setup parameters, the results from the Excel worksheet indicated the expected location for initiation of the avoidance maneuver and the direction of the avoidance (left, straight, or right).

A considerable amount of additional useful information was also provided, such as the expected flightpath and the closest approach to terrain. All of this information could be visualized using Google Earth as described below.

® Excel - Google Earth Interface To implement the user interface buttons that are shown within the blue fields in figure B1, the analysis team used Microsoft Visual Basic for Applications (VBA). The VBA is a scripting interface that has been ® ® included with all builds of Microsoft Office since version 4.0 (Microsoft Office 2010 contains VBA ® version 14). The VBA allows Excel users to facilitate automatic worksheet features and calculations that if performed manually would require vast amounts of time. In the case of the information shown in figure B1, the user would have to constantly enter a time in the “Run Time” field in order to determine exactly where and when a ground collision avoidance maneuver would occur. The analysis team inserted VBA code to automate the process of determining the location of a ground collision avoidance maneuver.

® The user simply clicked on the button labeled “Simulate,” and Excel ran the calculations until an avoidance, as shown in figure B2, was found.

Figure B2. Avoidance maneuver initiation example.

Figure B3 displays the results of clicking on the “Plot in Google Earth” command button that is shown in figures B1 and B2. In figure B3, the black line is the “pre - avoidance” flightpath (that is, the predicted flightpath until the point of an Auto GCAS maneuver). When the flightpath was expected to be straight (a 0 initial bank value in merged cells A4/A5 as shown in figure B2), the program drew a straight line in Google Earth to represent a straight pre-avoidance flightpath. That straight line began at the initial longitude-latitude-altitude coordinates and along the heading specified by the user, and ended at the location of the avoidance maneuver initiation.

Figure B3. Result from clicking “Plot in Google Earth” button illustrated in figure B2.

If a turning flightpath was expected, the pre-avoidance flightpath displayed as curved in Google Earth.

The length of that curved black line was based on the user-specified heading change, using a radius established by the user-specified bank angle and load factor. For example, in figure B2, merged cells G14/G15 were set to 45, which meant that the curved black line in Google Earth represented a turn through a heading change of 45 deg prior to the Auto GCAS avoidance maneuver initiation. In the example worksheet shown in figure B2, the load factor of 1.2 g entered in merged cells C2/C3 determined the radius of the curved flightpath. Visualizing the results in Google Earth enabled the user to determine where a turning flight-test maneuver needed to begin in order to result in an avoidance initiation at the desired location and flight conditions.

The light green cylinders in figure B3 represent the Auto GCAS algorithm scan width, which is a function of the range along the avoidance prediction path (the scan width circles increase in diameter with range from the initiation point). The scan width near the initiation point was based on the horizontal uncertainty (from merged cells G12/G13 in figure B2). The dark green discs (“wagon wheels”) in figure B3, which lie slightly above the light green cylinders, represent the sum of the terrain clearance buffer (TCB from merged cells A13/A14 in figure B2) and vertical uncertainty (from merged cells G10/G11 in figure B2). Whenever any portion of the buffered terrain (represented by the dark green discs in figure B3) intersected with the trajectory prediction (represented by the blue line in figure B3) the result would indicate the coordinates for the expected avoidance initiation (indicated by the white arrow in figure B3).

The “Plot In Google Earth” buttons shown in figures B1 and B2 allowed the user to display KML paths within Google Earth that represented the ground collision avoidance trajectory predictions. An ® ® ® Excel - MATLAB interface was required in order to use MATLAB functions that were needed to ® ® produce KML files. The MATLAB application and Excel communicated through the Microsoft ® ® Component Object Model (COM). By implementing a COM interface, MATLAB and Excel were able to easily exchange data in real time.

® In addition to pre-flight predictions using the Excel tool, the analysis team also plotted flight-test data from the DROID within Google Earth. Figure B4 represents an actual avoidance maneuver taken from flight-test data. The black extrusion represents the actual DROID flightpath before and after the avoidance maneuver. The red extrusion represents the actual DROID flightpath during the avoidance maneuver.

Terminology for the green cylinders and dark green “wagon wheels” remains the same as that used for figure B3. The orange cubes represent the DEM data for the GCAS valley test area. It can be seen in figure B4 that neither the left nor straight trajectory was selected (because the trajectory predictions were well below the buffered terrain within the scan patterns). Figure B4 also shows that the actual flightpath for the selected right avoidance maneuver closely followed the trajectory prediction for this example. Additional examples for Google Earth plots of flight-test data are shown in the main body of this report as well as below within this appendix.

Figure B4. Example flight-test data.

Available Reaction Time The premise of the “Available Reaction Time (ART)” calculation was to determine the amount of time within which a pilot would need to react were an Auto GCAS maneuver not initiated. Available reaction time was defined as the amount of time after initiation of the Auto GCAS maneuver within which the same maneuver could have been delayed while still avoiding terrain. The reason for calculating ART was to determine the “nuisance potential” of the system. A formal study has not yet been performed to quantify the ART nuisance boundary for UAVs similar to the DROID. For the purpose of this demonstration project, ARTs in excess of six seconds were considered possible nuisances, while those less than four seconds were considered probable non-nuisances. A negative ART indicates that the avoidance maneuver would not prevent the DROID from flying into the terrain, or in this case, the terrain plus TCB.

To calculate ART, the trajectory of the DROID after avoidance maneuver initiation was extrapolated as though the avoidance maneuver had been delayed. Figure B5 shows a setup maneuver, avoidance activation, and avoidance maneuver but without an extrapolated trajectory. The extrapolated trajectory consisted of three segments: the delay segment, the delayed avoidance maneuver segment, and the post-termination maneuver segment. The delay segment, shown in green in figures B6, B7, and B8, estimated the DROID’s path had no avoidance maneuver been initiated . The delayed avoidance maneuver segment, shown in dark red in the same three figures, used the same avoidance maneuver path that occurred in flight-testing, delayed by the selected time increment. The post-termination maneuver segment, shown in dark blue in these figures, attempted to estimate the trajectory as though the avoidance maneuver had continued after termination of the flight-test maneuver. The delay segment was progressively increased until any of the three segments intersected with the digital terrain. The ART was then defined at the value of the delay segment one frame before the extrapolation intersected with the digital terrain.

Figure B5. Actual trajectory.

Figure B6. Three-segment extrapolation at intersection with terrain.

Figure B7. Three-segment extrapolation one frame before terrain intersection.

Figure B8. Three-segment extrapolation from above.

Figures B5, B6, and B7 show flight 19 event 16. Figure B5 shows the actual trajectory of the DROID, including the path during the setup maneuver prior to the avoidance activation and the actual avoidance maneuver. Figure B6 shows the three segments of extrapolation delayed 2.2 s, resulting in part of the trajectory intersecting with the digital terrain. Fig. B7 shows the three segments one frame earlier, when the extrapolation was delayed 2.0 s and the trajectory just missed the digital terrain. For this example, this technique resulted in an ART value of 2.0 s.

The delay segment treated the DROID as a point mass and used basic particle kinematics to extrapolate using the aircraft states at the original activation, ignoring acceleration in the z-axis. The delayed avoidance maneuver was a copy of the actual avoidance maneuver, rotated about the z-axis to align with the delay segment, so that the flightpath continued smoothly into the delayed avoidance maneuver. The flightpath after the delayed avoidance maneuver was calculated differently, depending on whether the avoidance maneuver was over smooth terrain or not.

Over smooth terrain, the important factor in determining whether the extrapolated flightpath would intersect with the digital terrain was the movement of the DROID in the vertical plane, because the digital terrain was of approximately the same height throughout the surrounding area. In mountainous areas, however, the movement of the DROID in the horizontal plane dominated the consideration of whether the extrapolated flightpath would intersect the digital terrain. For this reason, over smooth terrain, the actual flightpath of the DROID after avoidance maneuver termination was used. Over mountainous terrain the trajectory prediction (as calculated at activation) was used to estimate the flightpath after avoidance maneuver termination. The trajectory prediction was used for mountainous terrain instead of the actual flightpath because the safety pilot often took control shortly after normal termination (as planned for each test point). The trajectory prediction was considered the best estimate for the flightpath that would have occurred had the safety pilot not taken control. It was not necessary to use the trajectory prediction for smooth terrain because there were few of those events and the safety pilot did not take control until after the point at which the extrapolated trajectory intersected with digital terrain.

The best available digital terrain model was used to determine the intersection with the extrapolated flightpath. For these ART analyses, the one-third-arc-second National Elevation Dataset (NED) was used.

The delay was incremented in time steps that were the same as the recorded data on the smartphone (roughly 0.2 s). The specific magnitude of each time step was dependent on the smartphone data time steps because the smartphone calculations were somewhat asynchronous and did not calculate at precise, regular intervals.

Three Segment Extrapolation Methods As discussed above, the DROID trajectory was extrapolated in three segments: the delay segment, the delayed avoidance maneuver segment, and the post-termination maneuver segment. A top view of these three segments at the point where the ART value was determined for flight 19 event 16 (as viewed from above) is shown in figure B8. This figure shows that the three-segment extrapolation method creates a continuous trajectory.

Delay Segment For the first segment of the extrapolated flightpath for ART calculations, the SUAV was treated, for simplicity, as a point mass. In general, ART calculations were based on particle kinematics, with the exception that vertical accelerations were ignored, and the DROID was assumed to experience a constant rate of climb or descent.

In the case of the ART analysis, vertical accelerations were not taken into account for two reasons. The first reason was the extremely low signal-to-noise ratio in the z-axis Piccolo II autopilot accelerometer data (see figure B9). The second reason was that the DROID often flew in near-equilibrium flight (lift equaling weight, and thrust equaling drag) as commanded by the autopilot. In addition, the small size of the DROID made it extremely susceptible to even slight changes in wind direction, wind speed, turbulence, and other atmospheric conditions. In calculating the ART for the F- 16, the maneuvers were far “larger” in the sense of the scale of the F-16; therefore all accelerations (north/east/down) had to be taken into account.

Figure B9. Impact of atmospheric turbulence on measured vertical acceleration.

In the case of the ART analysis as applied to the SUAV project, the noisy z-axis accelerometer data induced inappropriate and unlikely estimated flightpaths. The Piccolo II autopilot kinematics data (north and east velocities and accelerations) also failed to record; therefore, the analysis team generated the estimated flightpaths from known position and velocity data (latitudes, longitudes, and climb rate).

Velocities were obtained from changes in latitude and longitude, and accelerations were further differentiated from changes in velocity. The technique of differentiation in order to obtain velocities and accelerations was not ideal because differentiation often amplified signal noise.

To counteract the effects of signal noise due to differentiation, the analysis team applied a generic smoothing routine to the north/east velocities and accelerations; vertical acceleration remained too noisy to be useful even after smoothing attempts. Figure B9 shows a typical sample of the measured z-acceleration.

The upper limit of the normal maneuvering envelope of the DROID as configured (1.3 g given the preprogrammed bank limit of 40 deg) is also shown in figure B9. This clearly shows that the noise in the data captured was induced by atmospheric turbulence, not the normal maneuvering of the DROID.

Delayed Avoidance Maneuver The delayed avoidance maneuver flightpath estimated what the DROID would have done had the same avoidance maneuver been executed after some time delay. Since the heading may have changed during the delay segment, the delayed avoidance maneuver was rotated to align with the heading at the end of that delay segment. A new initial heading and a new position (latitude, longitude, and altitude) were chosen to match the end of the delay segment flightpath, and from this position, the flightpath angles and changes in heading of the original avoidance maneuver were applied. Changing the initial heading but recreating the same change in heading effectively “rotated” the avoidance maneuver about the z -axis and maintained continuity in the horizontal flightpath between the delay segment and the delayed avoidance maneuver.

Rotating the avoidance maneuver about the y-axis was not necessary because vertical acceleration was ignored and the flightpath angle did not change during the propagation.

Figures B6 and B8 show that the method of rotating the flightpath of the original avoidance maneuver and appending it to the end of the delay segment resulted in a continuous propagated flightpath. Ignoring the vertical acceleration did not induce a significant discontinuity in the vertical flightpath at the point where the delay segment met the delayed avoidance maneuver. This method worked because most flight-test maneuvers were near 1 g during the approach to the avoidance maneuver. If the maneuvering of the DROID at the initiation of the avoidance maneuver were more dynamic, a different propagation method would have been required.

Post-Termination: Mountainous Terrain When the avoidance maneuver ended, if neither the safety pilot nor ground control gave instructions to the DROID, the DROID would stay in the climbing turn last commanded during the avoidance maneuver.

In other words, the autopilot made no effort to return the DROID to level flight before terminating. Over mountainous terrain, it was the curvature of this climbing turn that most accurately determined whether the extrapolated flightpath would intersect with digital terrain. The actual post-termination flightpath could not be used because it was normal test procedure for the safety pilot to take control soon after the avoidance maneuver had terminated. Therefore it was determined that the best estimate for propagating the flightpath of the DROID after termination was based on the steady climbing turn calculated as part of the trajectory prediction from when the avoidance maneuver was initiated.

To make the estimation of flightpath as continuous as possible, the point along the trajectory prediction where the heading most closely matched the heading at the end of the delayed avoidance maneuver was used as the beginning of the post-termination segment. This flightpath was then translated in three dimensions to meet the end of the delayed avoidance maneuver. This method may result in some discontinuity in flightpath angle, but since the slope of the flightpath of the DROID was so much less than the slope of the digital terrain, small offsets in extrapolated altitude were less significant than the latitude and longitude of the DROID during this segment. Also, this method guaranteed that the flightpath angle over the entire extrapolation was a flightpath angle sustainable by the DROID.

Flight 19 event 10 having flown over mountainous terrain, it was used to demonstrate this method for the post-termination segment. Figures B6 and B8 show that this method created a smooth transition (that is, revealed no obvious discontinuities) from the delayed avoidance maneuver to the post-termination segment.

Post-Termination: Smooth Terrain Over smooth terrain, the most important factor in determining ART was the vertical flightpath shortly after the avoidance maneuver. In order to obtain the most accurate trajectory propagation, the actual flightpath after termination of the avoidance maneuver was used (instead of the trajectory prediction method that was used for mountainous terrain). This method worked over smooth terrain because there were a few seconds until the safety pilot took control, and that amount of time was sufficient to provide enough data for the propagation. The actual flightpath after termination, up until the safety pilot took control, was appended to the end of the delayed avoidance maneuver. That flightpath was rotated and translated similar to the way in which the delayed avoidance maneuver was appended to the delay segment. This ensured continuity in both heading and flightpath angle.

The post-termination segment of the extrapolated flightpath over smooth terrain was significantly less important to the ART calculation than that segment over mountainous terrain. Most avoidance maneuvers over smooth terrain terminated during a slight climb, which was sufficient to clear smooth terrain.

Comparison to Digital Terrain Truth Data Once the longitude, latitude, and altitude of the DROID were extrapolated for the entire three-segment trajectory, the extrapolated trajectory was compared to “best source” data for the digital terrain. For this ART analysis the best source for truth data was considered to be the one-third-arc-second NED. At every point of longitude-latitude in the extrapolated flightpath (shown by solid colored vertical lines in Google Earth snapshots in figures B6, B7, B10, and B11) the height of the NED was found. The TCB for each event was then added to the NED as a way to normalize across every test run, since the Auto GCAS system interpreted the “ground” to be at digital terrain height + TCB. The NED altitude plus TCB is represented by a tan line in these figures. Because flight 19 event 16 had a TCB of 0 ft and Google Earth also used the one-third-arc-second NED, these tan lines are barely visible above the ground as it is displayed in Google Earth depiction of figures B6 and B7. Therefore, flight 19 event 7, with a TCB of 100 ft, was used to demonstrate this concept in figures B10 and B11. Figure B10 shows the delayed avoidance maneuver at the frame when the propagated trajectory was just clear of the buffered terrain (NED + TCB). That frame in the propagation was used to define the ART value. Figure B11 shows the delayed avoidance maneuver one frame later, when the propagated trajectory intersected with the NED + TCB. The vertical tan line shows the first point along the trajectory where the NED + TCB line was above the trajectory. The intersection in this event happened during the delayed avoidance maneuver segment, but it sometimes happened in the post-termination segment (as in the case of flight 19 event 16).

Figure B10. Extrapolated trajectory above NED + TCB (at the ART).

Figure B11. Extrapolated trajectory below NED + TCB (one frame after the ART).

Determination of Trigger Post Locations As described in the “Generic ‘Sense Terrain’ Module” section in the main body of this report, the DEM was decompressed as a rectangular grid of posts with latitude, longitude, and altitude, and treated as a flat polygon surrounding that post at the same altitude. The trigger post analysis seeks to determine which specific post caused the avoidance maneuver.

The trajectory prediction and related scan circles in the direction of the avoidance were used as the basis for this analysis. In the case of flight 16 event 6, which is depicted in figures B12, B13, and B14, the avoidance was to the right, so the right trajectory prediction was analyzed. Within the Auto GCAS algorithm, the digital terrain was represented by scan circles at the altitude of the highest terrain polygon within the radius of each circle. That altitude was raised further to account for the flight-test buffer (the TCB) and the built-in buffer (vertical uncertainty) since the trajectory prediction was being compared to DEM + TCB + vertical uncertainty altitude.

Figure B12. Trajectory prediction and scan circles viewed from above.

Figure B13. Trajectory prediction and scan circles viewed from side.

Figure B14. Trigger post location relative to scan circle.

In figure B13, the scan circles are shown in light green (at the DEM altitude) and dark green (at the DEM + TCB + vertical uncertainty altitude). The trajectory prediction is shown in blue. When the trajectory prediction was lower than one of the buffered scan circles the avoidance maneuver was triggered.

The trigger post was associated with the polygon that gave this scan circle its height. In figure B13, it can be seen that the 8th light green scan circle (from the left) is paired with the first dark green buffered circle that the trajectory prediction passes underneath. Figure B14 shows this 8th scan circle and the surrounding DEM polygons. In this case the scan circle just touches the corner of the polygon associated with the trigger post. The practical significance of this implementation was that the DEM posts which triggered avoidance maneuvers tended to be outside of the scan circles and therefore created some additional horizontal buffer away from the actual terrain.

Determination of Worst-Case Mismatch Locations In order to evaluate the accuracy of the trajectory prediction, the worst-case mismatch between the actual ground track compared to the predicted ground track was found for each event.

The actual ground track always included the automated avoidance maneuver. In some cases, the actual ground track also included the time after the automatic system terminated in order to determine when the DROID was at its minimum altitude above the ground (as long as the safety pilot did not take control). In figures B15 and B16, the actual ground track is shown by a red dotted line. The predicted ground track and half-scan width, as determined by the algorithm, are shown in blue and purple solid lines, respectively.

Figure B15. Worst-case point: inside of the scan.

Figure B16. Worst-case point: outside of the scan.

In some cases, the ground track was never farther away from the predicted ground track than the half-scan width. The DROID stayed above the terrain included in the scan pattern. In such cases, the worst case was the point at which the actual ground track was farthest from the prediction. Figure B15 shows the worst-case analysis plot for such a case, flight 12 event 1. The red actual ground track is clearly inside the purple scan, and the worst case is the point at which the red actual ground track is farthest from the blue predicted ground track.

In other cases, the actual ground track was farther away from the prediction than the half-scan width.

In these cases, the DROID flew over terrain not necessarily accounted for by the algorithm. In such cases, the worst case was the point at which the flightpath was farthest outside the scan. Figure B16 shows the worst-case analysis plot for such a case, flight 18 event 6. The red actual ground track extrudes past the purple scan, and the worst case is the point at which the red actual ground track is farthest outside the purple scan.

The methods for finding the distance along the trajectory prediction and the half-scan width at each predicted ground track point are described in the following section.

Trajectory Normalization Analysis In order to assess how well the overall Auto GCAS algorithm was working, a normalization analysis was used to compare key points from multiple test runs all on the same plot. This normalization method was used to show key points at the trigger posts and the location of the worst-case mismatch between the actual trajectory compared to the prediction as described in the appendix sections “Determination of Trigger Post Loca tions” and “Determination of Worst - Case Mismatch Locations.” The following discussion assumes a known latitude and longitude for each point in question.

The premise of the normalization analysis was that any desired point could be characterized by a distance along the trajectory prediction and distance perpendicular to the trajectory prediction. These distances could also be expressed as percentages of the total length and percentages of the corresponding width.

Scans for individual flight-test runs had different shapes and lengths depending on factors such as initial turn rate, airspeed, and wind. In Figures B17 through B22, comparisons of the trajectory prediction to the right were generated by changing the initial turn rate (bank), airspeed, and wind, respectively. A similar set of scans could be generated for a trajectory prediction to the left or straight. In each set of comparison figures, the trajectory prediction shown on the left-hand side of the page is based on the DROID initially traveling at 0 deg bank and 75 KTAS with no wind. Figure B17 compares the no-bank trajectory prediction to a trajectory prediction with an initial 40-deg left bank in figure B18. The DROID needs time to adjust from turning left to turning right, so the initial opposite bank makes the turn somewhat wider. Conversely, an initial bank in the same direction of the avoidance makes the turn tighter. Figure B19 compares the 75-KTAS trajectory prediction to a trajectory prediction with an initial speed of 60 KTAS in figure B20.

The slower airspeed results in a tighter turn radius. Figure B21 compares the trajectory prediction with no initial crosswind to a trajectory prediction with a 20-kn crosswind from the right in figure B22. A crosswind of that magnitude from that direction tightens the turn radius considerably, while a crosswind in the opposite direction widens the turn radius.

Figure B17. Trajectory prediction with no initial Figure B18. Trajectory prediction with initial bank . 40 - deg left bank .

Figure B19. Trajectory prediction with 75-kn Figure B20. Trajectory prediction with 60-kn initial speed. initial speed.

Figure B21. Trajectory prediction with no initial Figure B22. Trajectory prediction with 20 - kn wind. initial crosswind.

A single scan shape was selected to use as a reference for showing the key points from all of the flight-test runs. This reference scan shape was selected from flight 12 event 4, initiated at 73 KTAS, 0.5 deg bank, with a negligible crosswind. By normalizing the location of each point in question relative to the reference scan shape, those points could all be placed on one plot even if the original avoidance maneuver was to the left, straight, or right. The results shown in figures 54 and 58 of the main body of this report used the techniques discussed in the following paragraphs. The results of this analysis are also shown on histograms in figures 55, 56, 59, and 60 of the main body of this report, to provide an alternative summary.

Finding Distances The trajectory prediction on the smartphone determined the centerline at multiple bin locations. The spacing between each bin was a function of how dynamic the expected avoidance maneuver would be at that location. The bins were spaced closer together during dynamic portions of the maneuver such as during g-onset or roll-rate onset. The bins were spaced farther apart if the expected avoidance maneuver at that location would be relatively stabilized.

Given the latitude and longitude of individual bin locations along the centerline of a trajectory prediction from flight-testing, the first steps were to find the total length of the centerline and the scan pattern width at each bin location. A visual representation of this method is shown in figure B23. Each distance point is shown as a blue dot. At the beginning of the trajectory (the most dynamic portion of the maneuver), the dots are so close together that they appear to form a solid line. The total distance is the sum of the straight line distances between neighboring distance points.

Figure B23. Method for finding total distance along trajectory prediction.

The scan pattern width could then be found at any distance point along the trajectory prediction using equation (B1). In order to match the values used during SUAV Auto GCAS flight test in equation (B1), 𝐻𝑜𝑟𝑖𝑧𝑜𝑛𝑡𝑎𝑙𝑈𝑛𝑐𝑒𝑟𝑡𝑎𝑖𝑛𝑡𝑦 = 50 𝑓𝑡 , and 𝑈𝑛𝑐𝑒𝑟𝑡𝑎𝑖𝑛𝑡𝑦𝐺𝑟𝑜𝑤𝑡ℎ𝐴𝑛𝑔𝑙𝑒 = 10° for turning scans and 5° for straight scans. The results of this method are shown in figure B24. By plotting the purple half-scan width on each side of the blue trajectory prediction, the entire scan is shown.

𝐻𝑎𝑙𝑓 𝑆𝑐𝑎𝑛 𝑊𝑖𝑑𝑡 ℎ (B1) 2 2 √ ( ) = 𝐻𝑜𝑟𝑖𝑧𝑜𝑛𝑡𝑎𝑙𝑈𝑛𝑐𝑒𝑟𝑡𝑎𝑖𝑛𝑡 𝑦 + ( 𝐷𝑖𝑠𝑡𝑎𝑛𝑐𝑒𝐴𝑙𝑜𝑛𝑔𝑇𝑃𝐴 ∗ sin 𝑈𝑛𝑐𝑒𝑟𝑡𝑎𝑖𝑛𝑡𝑦𝐺𝑟𝑜𝑤𝑡 ℎ 𝐴𝑛𝑔𝑙𝑒 Figure B24. Half-scan width.

Once the total length of the centerline and the scan pattern width at each point along the entire length were known, the next step was to identify where key points of interest from a flight-test maneuver were located relative to the centerline. As mentioned at the beginning of this section, those key points could be trigger posts or they could be the location of the worst-case mismatch between the actual trajectory and the prediction. Given the latitude and longitude of a key point in question, distances along the centerline and perpendicular to the centerline were needed.

The two distance point locations along the centerline closest to the key point in question were found by using a distance function to find the distance from every point location along the centerline to the key point.

The total distance along the centerline from maneuver initiation up until the distance point closest to the key point in question was determined by using the sum of the distances between neighboring distance point locations, as already described in the discussion above of figure B23.

To find the distance perpendicular to the centerline, distance functions were used to determine the perpendicular distance, as shown in figure B25. The two closest bin locations were mathematically connected by a straight line. A perpendicular to that line was determined which also intersected with the key point. The length of this line was used as the distance perpendicular to the centerline.

Figure B25. Method for finding perpendicular distance at a given point along the trajectory prediction.

Nondimensionalizing Distances and Plotting on a Representative Trajectory Prediction Algorithm In order to make the distances nondimensional, the perpendicular distance was divided by the half-scan width at that location to convert into a percentage. Similarly, the distance along the trajectory prediction from maneuver initiation to the key point was converted into a percentage by dividing by the total length of the trajectory prediction. In addition, to transpose a point from a right or straight trajectory prediction onto the intended reference (left trajectory prediction), the key point location inside or outside of the trajectory prediction was used. The right trajectory prediction shown in figure B26 shows a point in orange that is 50 percent along the trajectory prediction length, and 75 percent of the half-scan width inside the curve of the trajectory prediction. The right trajectory prediction shown in figure B26 also shows a point in blue that is 50 percent along the trajectory prediction length, and 75 percent of the half-scan width outside the curve of the trajectory prediction. The left trajectory prediction shown in figure B26 shows how the orange and blue points were transposed onto the reference left trajectory prediction.

Figure B25. Example for non-dimensional key point locations.

Termination Logic Cross-Checks When an SUAV avoidance maneuver was not terminated by the safety pilot, the Auto GCAS algorithm determined when the avoidance maneuver was no longer necessary and returned control to the ground control operator. In order to assess the timeliness of the avoidance maneuver termination, the flight-test results were compared with an ideal termination defined by generating a straight line tangent to the avoidance flightpath. A straight line tangent (including the current climb rate) was used as a way to identify that there was no obstructing terrain directly in front of the DROID. Although other methods could have been used, this was considered the simplest and would most directly correlate with the view of the pilot (in this case, the view provided by the forward-looking video camera). When the straight tangent line was projected to be clear of the DEM terrain for three consecutive time frames (approximately 0.2 s per frame), that third frame was considered the ideal termination.

The purpose for these termination logic cross-checks was to determine whether the Auto GCAS algorithm terminated the maneuver earlier than it should have, at about the right time, or later than necessary. As a general result, when the DROID maneuvered left to avoid terrain, the software logic caused the avoidance maneuver to terminate earlier than it should have. For a right avoidance, the Auto GCAS maneuver tended to terminate later than necessary. The overall results are described in the main body of this report; the methods used are described below.

Figure B27 illustrates how the Auto GCAS algorithm determined when to terminate the avoidance maneuver using an example from flight-testing. The black line in figure B27 represents the actual flightpath of the DROID prior to the avoidance maneuver. The navy blue line in figure B27 represents the trajectory prediction at initiation (in this case, the Auto GCAS algorithm determined that the DROID should avoid terrain by executing a left turn). The red line in figure B27 represents the actual flightpath during the avoidance maneuver. To determine when an avoidance maneuver should be terminated, the Auto GCAS algorithm computed a straight trajectory at every time frame throughout the maneuver. The labeled “1st blue line” in figure B27 represents the straight trajectory prediction one frame before the trajectory prediction was clear of the buffered digital terrain (green rectangular polygons). The labeled “2nd blue line” in figure B27 represents the straight trajectory prediction at the frame when the trajectory prediction was first clear of the buffered digital terrain. The straight trajectory predictions do not appear straight in figure B27 because of the P-factor, described in the main body of this report. The Auto GCAS algorithm terminated the avoidance maneuver when three consecutive frames were clear of terrain.

Figure B27. Example avoidance maneuver.

® To accomplish termination logic cross-checks, the Auto GCAS team used MATLAB to implement the calculations combined with Google Earth as a visualization aid. Using Google Earth KML files, violet straight lines were drawn tangential to the red avoidance maneuver flightpath in figures B28 through B30.

To avoid clutter, these violet lines were only drawn every fifth frame (there is roughly 1 s between each line).

Figure B28. Tangential lines (violet) drawn from the avoidance flight path (red).

Figure B29. Right-turning avoidance terminated late.

Figure B30. Left-turning avoidance terminated early.

Next, three arc-second buffered DEM terrain tiles (represented by the green polygons) were added. The buffered terrain added the TCB value to the DEM altitude but did not include any built-in buffers.

Therefore, the results from any test run could be evaluated in the same manner as if the TCB were set to 0.

The DEM terrain tiles were sized at three arc-seconds to be consistent with the resolution used by the Auto GCAS algorithm and to minimize computational time. This calculation being post-flight, the theoretical accuracy could have been improved using higher resolution DEM tiles (as fine as the one-third-arc-second resolution of the NED source data) but that increased accuracy was not considered necessary for this analysis.

The ideal termination heading was defined to be when the green DEM polygons did not block three consecutive tangential paths (as shown by the three lighter colored violet lines in figures B28 and B29).

The determination of when the tangent lines no longer intersected with the DEM terrain was accomplished as a numerical calculation but is shown using Google Earth to help visualize the concepts.

Figures B28 and B29 illustrate termination calculations for similar right-turning avoidance maneuvers.

Figure B28 shows the avoidance maneuver as viewed from almost directly above. Figure B29 shows the same maneuver from the perspective of a lower viewing angle. The darker violet lines represent the tangential paths that were blocked by terrain (drawn every five frames), and the three right-most lighter violet lines represent the three-frames-of-persistence clear of terrain (drawn every frame). In these cases, the right-turning avoidances illustrated in figures B28 and B29 indicate that the avoidance maneuver terminated approximately 26 to 29 deg later than necessary.

The short vertical lines along each tangential path represent a distance equivalent to one arc-second.

Greater accuracy could have been achieved with smaller intervals for the tangential paths (that is, one-third arc second instead of one arc-second resolution) but computational time would have been increased significantly as a result.

Figure B30 illustrates termination calculations for a left-turning avoidance maneuver. The red flightpath in figure B30 once again represents the actual avoidance maneuver path up to the point at which the Auto GCAS algorithm terminated the maneuver. The violet straight lines (drawn every fifth frame) represent the tangential paths along the red actual avoidance path. The orange semicircle represents an extrapolated avoidance trajectory that the DROID would have taken had it continued the avoidance maneuver. The dark-orange straight lines (drawn every fifth frame) represent the tangential paths along the extrapolated orange semicircle. Finally, the three straight yellow lines (drawn at each frame) represent the consecutive tangential paths which do not intersect with the green DEM polygons.

To determine an extrapolated avoidance trajectory, several new elements were needed. Since the smartphone hosting the Auto GCAS algorithm did not use a consistent time interval for each frame, an average time interval was selected using the last five time increments in the red portion of the actual avoidance maneuver (see figure B31). Next, a constant radius was calculated for the extrapolated avoidance maneuver based on the arc between the last two points of the actual avoidance maneuver. This radius was calculated from equation (B2): 𝑠 = 𝜃 ∙ 𝑅 (B2) where 𝜃 is the heading change calculated from the last two points in the red avoidance path in figure B30, and s is the arc distance between those two points. In this example case, the left-turning avoidance maneuver shown in figure B30 terminated approximately 32 deg earlier than it should have. Although this extrapolation method worked reasonably well on most runs, it was also susceptible to noise in the source data, causing some uncertainty in the results. An alternative method could use an extrapolated radius based on the average arc over the previous several frames.

Figure B31. Differential time element illustration.

For the extrapolated runs the delta time and delta heading were based on the new incremental elements described above.

Appendix C: Flight Log

This appendix presents a summary of all flights that were related this project. Table C1 presents the first few flights that were accomplished on the DROID aircraft to obtain basic trajectory prediction data along with the SUAV system checkout flights. Table C2 presents all of the SUAV test flights that were accomplished to evaluate the overall system and obtain test data.

Table C1. Preliminary flights.

Flight Flight Duration, Smartphone number date min location Test site Test types* Notes DROID #1 4/14/2011 47 N/A North Base 10 sequential axis PARS runs** Normal inputs to Piccolo II DROID #2 4/14/2011 33 N/A North Base 8 sequential axis PARS runs** Normal inputs to Piccolo II DROID #3 4/14/2011 10 N/A North Base No test runs** Normal inputs to Piccolo II DROID #4 4/14/2011 29 N/A North Base 9 sequential axis PARS runs** Normal inputs to Piccolo II DROID #5 4/14/2011 35 N/A North Base 13 sequential axis PARS runs** Normal inputs to Piccolo II SUAV #1 9/28/2011 44 Van North Base 9 combined axis PARS runs** UI inputs to Piccolo II SUAV #2 9/28/2011 42 Van North Base 7 Auto GCAS functional checks None SUAV #3 9/28/2011 51 Van North Base 17 combined axis PARS runs** UI inputs to Piccolo II SUAV #4 10/7/2011 26 Van Rosamond Lakebed 3 Auto GCAS runs Intermittent RPM sensor SUAV #5 10/7/2011 15 Van Rosamond Lakebed No test runs Failed RPM sensor SUAV #6 10/7/2011 27 Van Rosamond Lakebed 3 Auto GCAS runs Intermittent RPM sensor SUAV #7 10/18/2011 53 Van Rosamond Lakebed 12 Auto GCAS runs TM dropouts near hill 2 Auto GCAS runs SUAV #8 10/18/2011 51 Van Rosamond Lakebed None 11 combined axis PARS runs Notes: * The number of runs listed represents the runs attempted. Some runs were not completely successful for a variety of reasons.

** The six flights identified with double asterisks were accomplished to determine the parameters needed to define the trajectory predictions. Pilot Activated Recovery System (PARS) maneuvers were initiated in each axis to obtain the required data. PARS maneuvers were executed using the same command sequence intended for the corresponding flyup maneuvers.

Table C2. Primary test flights.

Flight Flight Duration, Smartphone number date min location Test site Test types* Notes Top of small hill SUAV #9 10/21/2011 55 Van GCAS valley 13 Auto GCAS runs TM dropouts near hill Top of small hill SUAV #10 10/21/2011 55 Van GCAS valley 11 Auto GCAS runs TM dropouts near hill Base of small hill SUAV #11 11/7/2011 49 Van GCAS valley 7 Auto GCAS runs TM much improved SUAV #12 11/7/2011 35 Van GCAS valley 6 Auto GCAS runs Medium hill 6 Auto GCAS runs (no flyups) No flyups: traced to Piccolo II roll rate SUAV #13 3/5/2012 59 Aircraft Rosamond Lakebed 1 nuisance test rehearsal values SUAV #14 3/15/2012 32 Aircraft GCAS valley 5 Auto GCAS runs Smartphone data did not record SUAV #15 3/15/2012 2 Aircraft GCAS valley No test runs Smartphone dislodged on takeoff SUAV #16 3/15/2012 53 Aircraft GCAS valley 9 Auto GCAS runs FAIL states disrupted testing Near Fremont Peak SUAV #17 3/29/2012 59 Van GCAS valley 13 Auto GCAS runs (Smartphone back in van to allow testing without disruptions due to FAIL states.)

Lakebed and medium hill SUAV #18 3/29/2012 63 Van GCAS valley 11 Auto GCAS runs Guest pilots.

SUAV #19 3/29/2012 54 Van GCAS valley 12 Auto GCAS runs Medium hill Small hill SUAV #20 5/31/2012 40 Aircraft GCAS valley 7 nuisance test ridge crossings (Smartphone back in DROID with improved FAIL states.)

12 Auto GCAS runs SUAV #21 5/31/2012 52 Aircraft GCAS valley Medium hill 3 nuisance test valley patrols Note: * The number of runs listed represents the runs attempted. Some runs were not completely successful for a variety of reasons.

Appendix D: Open Discrepancy Reports

Table D1 provides a list of each NASA Discrepancy Report (DR) that remained open at the end of the SUAV Auto GCAS project.

An additional 29 DRs were written during the project but were closed by the Configuration Control Board, normally because the problem had been confirmed as fixed. The main intent of table D1 is to allow staff of similar future projects to decide whether each item warrants improvement.

A paraphrased and shortened description for each DR is provided in the table.

Table D1. Open Discrepancy Reports.

DR # Title Short description and disposition Auto GCAS component  When a flyup is engaged, transitioning to IDLE keeps the flyup TRANSITION TO IDLE active. User interface 11 - 125 KEEPS FLYUP ACTIVE  Workaround: Click on another mode to transition out of flyup mode  User interface timeout on AGCAS connection occurs at random INTERMITTENT times, causing a FAIL mode to be asserted. Frequency is very User interface / smartphone CONNECTION BETWEEN 11 - 126 low, but random in nature. software PHONE AND USER  Workaround: Click on appropriate mode on user interface to INTERFACE resume normal function.

 Termination of some flyups is delayed longer than necessary.

Data show that flyups to the r ight result in delayed termination; INAPPROPRIATE FLYUP flyups to the left result in early termination.

11 - 130 TERMINATION DUE TO Algorithm on sma rtphone  Recommendation: Future SUAV Auto GCAS projects should P - FACTOR consider modifying the termination logic to use a straight trajectory that is unaffected by P - factor.

 FAIL indications occurred shortly after flyup initiation on at least three test runs. These FAIL indications were somewhat disruptive to the normal test flow and degraded the intended test data. On later missions the FAIL “ timeout” was changed from 0.5 to 5.0 s 12 - 108 FAIL INDICATIONS to ensure no test disruption, but 5.0 s may not be the optimal setting.

 Recommendation: Determine optimal FAIL timeout value for future Auto GCAS projects.

 When the DROID pilot is commanding the from the pilot control STREAMLINE CAPABILITY station at the time a FLYUP is initiated, the pilot does not have a FOR PILOT CONTROL single - action command ability to regain control of the DROID. Ground control van and user 12 - 110 AFTER AGCAS interface  Recommendation: Mechanize system so that a single a ction FLYUP/ABORT command by the pilot will regain control after a FLYUP or ABORT.

 A right FLYUP command was sent but there was no response AUTOPILOT DID NOT from the autopilot. This occurred onl y once on the three flights on Smartphone to Piccolo II 12 - 111 RESPOND TO FLYUP March 29.

interface COMMAND  Recommendation: Additional research as needed to support future Auto GCAS development efforts.

 The left, straight, and right trajectories recorded at FLYUP ILLOGICAL FLYUP initiation were all identical.

12 - 112 TRAJECTORIES IN Algorithm on smartphone  Recommendation: Additional research as needed to support future RECORDED DATA Auto GCAS development efforts.

 The actual flyup trajectory went outside the scan pattern by approximately 50 ft horizontally. The altitude approached 25 ft of the TCB, indicating that going outside the scan may have FLYUP WENT OUTSIDE 12 - 113 contributed to reduced terrain clearance. This run may have been Algorithm on smartphone SCAN PATTERN influence d by wind changes after FLYUP initiation.

 Recommendation: Additional research as needed to support future Auto GCAS development efforts.

 Three additional short flyups occurred after termination of the first flyup (over total duration of 7 s). Cases with one or two additional flyups occurred on other flights. It would be INDECISIVE FLYUP 12 - 114 appropriate for every flyup to terminate without subsequent Algorithm on smartphone TERMINATION flyups for at least several seconds.

 Recommendation: Additional rese arch as needed to support future Auto GCAS development efforts.

 Flyup was as much as 130 ft inside the scan. Flyups with this characteristic pose an increased risk of nuisance potential but are FLYUP WENT INSIDE SCAN 12 - 115 unlikely to increase risk of terrain impact. Algorithm on smartphone PATTERN  Recommendation: Additional research as needed to support future Auto GCAS development efforts.

 The data show that the predicted trajectories were well below the buffered terrain for both left and right options. This result implies 12 - 116 POSSI BLE LATE FLYUPS a large jump in the TPA compared to a single frame earlier. Algorithm on smartphone  Recommendation: Additional research as neede d to support future Auto GCAS development efforts.

 GPS position discontinuities up to 132 ft occurred (in between frames) on several flights. This problem was later traced to a known problem in the Piccolo II software version used during 12 - 117 GPS DISCONTINUITIES Piccolo II sof tware flight - testing.

 Recommendation: Update Piccolo II software before future tests using the DROID.

 The DROID was able to establish a descent rate over 2500 ft/min AUTOPILOT EXCEEDS and a bank angle of almost 50 deg. These values were well in Ground control van to 12 - 118 LIMITS excess of expected Piccolo II limits of @ 1000 ft/min and 40 deg Piccolo II interface bank. This problem was late r traced to large - amplitude rudder inputs applied by the pilot in the ground cockpit (rudder inputs went directly to the control surfaces and were not limited by the Piccolo II).

 Recommendation: Reconfigure rudder pedal commands so Piccolo II limits will n ot be exceeded, or advise pilots on limited use of rudder pedals.

 The ground cockpit pilot could not disengage a flyup and regain control (as the desig n was then implemented). The main impact of not implementing this capability was the inability to properly NO CAPABILITY FOR conduct Auto GACS nuisance testing. Ground control van to user 12 - 120 COCKPIT PILOT TO interface  Recommendation: The ideal mechanization would allow a single DISENGAGE FLYUPS HOTAS action to terminate a flyup in progress and n ot allow that flyup to resume. “New” flyups could occur unless the pilot chooses to continue holding the HOTAS.

 The UI ABORT button sends a signal to terminate the current flyup but another flyup is almost immediately recalculated and ABORT BUTTON performed.

12 - 121 User interface FUNCTIONALITY  Workaround: Instead of clicking on the UI ABORT button, change the AGCAS mode to STANDBY or IDLE until ready to resume avoidance maneuvers.

 For eight flight - test events the Auto GCAS software recorded AGCAS SOFTWARE redundant trajectory prediction latitude and longitude data on the RECORDED REDUNDANT 12 - 122 smartphone SD card. These cases are related to DR 12 - 112. Algorithm on smartphone LATITUDE AND LONGITUDE  Recommendation: Additional research as needed to support future VALUES IN SMARTPHONE Auto GCAS development efforts.

References

1. Skoog, Mark, and Kevin E. Prosser, “Advanced Fighter Technology Integration/ F-16 Automatic Ground Collision Avoidance System Evaluation, ” AFFTC -TR-99-28, December 2000. Available from AFRL/VAAI, Wright-Patterson AFB OH 45433-7542.

2. Sorokowski, Paul, et al., “Automatic Ground Collision Avoidance System Fighter Risk Reduction Project, ” AFFTC -TIM-10-05, December 2010. Available from AFRL/RBCC, Wright-Patterson AFB, OH 45433-7542.

3. Moore, Duane P., and Kyle W. Schlappi, “ USAF F-16 Block 40/50 Test and Evaluation for the Automatic Ground Collision Avoidance System (Auto GCAS) and Pilot Activated Recovery System (PARS), 412TW-TR-13-04, October 2013. Available from AFLCMC/WWM, Wright-Patterson AFB, Ohio 45433-7424.

4. Webb, Charles H., and Ryan K. Owen, “ F-22 Line in the Sky (LIS) Automatic Ground Collision Avoidance System (AGCAS) Test and Evaluation, ” 412TW-TR-13-42, January 2014. Available from Air Force Life Cycle Management Center, F-22 Division, Wright-Patterson AFB OH 45433-7017.

5. Rumsfeld, Donald, “Memorandum for Secretaries of the Military Departments , Subject: Reducing Preventable Accidents, ” U06916-03, May 19, 2003.

6. Gates, Robert M., “Memorandum for Secretaries of the Military Departments , Subject: Zero Preventable Accidents, ” May 30, 2007.

7. Defense Safety Oversight Council Aviation Safety Improvements Task Force Safety Technology Working Group, “ Fighter/Attack Automatic Collision Avoidance Systems Business Case, ” May 2006.

8. Skoog, Mark, Jack Ryan, and Mike Marston, “ACAT/SUAV Objectives and Requirements Doc ument,” ACAT/SUAV -ORD-001, January 2011. Available from AFRC – Code Z.

9. “Automatic Collision Avoidance Technology / Fighter Risk Reduction Project Auto GCAS Requirements Guide, ” draft version 1.0, February 9, 2008. Available from AFRL/RQQ.

10. Gruca, Dena, and Jack Ryan, “ACAT/SUAV System Design D escription (SDD), ” ACAT/SUAV-SDD-001, January 2012. Available from AFRC – Code Z.

11. Marston, Michael L., “Dryden Remotely Operated Integrated Drone (DROID) Project ,” Operational Concept Document, Revision A, December 2009.

12. Preliminary patent information, “Global Elevation Data Adaptive Co mpression System (GEDACS),” NASA ID DRC-012-001.