Appendix A – Controller and Pilot Questionnaires
Appendix A – Controller and Pilot Questionnaires
Five questionnaire forms were formulated to capture user inputs during the FT3 testing. The subjects included ATC controllers, ghost controller, and UAV pilots, as it related to their physical working environment. The HSI team developed their own and more extensive questionnaire to capture pilot feedback on the interaction with the ground control station, and are not included in this report.
Date: / / Time: UTC /Local Controller #: ATC Controller Questi onnaire for Flight Test 3, Configuration 2 (IT&E Form 1 - Training and Exit) Scenario Traffic Density Q1. The traffic density in the test sector was realistic relative to current - day operations: _ Much less busy than normal _ Somewhat less busy than normal _ Typical _ Somewhat busier than normal _ Much busier than normal Q2. The flow of traffic in the test sector was representative of a low altitude en route sector: _ Strongly disagree _ Somewhat disagree _ Neutral _ Slightly Agree _ Strongly Agree Number of encounters Q3. As compared with normal real - world operations, the number of the encounters in this trial was: _ Much less frequent _ Somewhat less frequent _ Typical _ Somewhat more frequent _ Much more frequent Traffic Complexity Q4. As compared with normal real - world operations, the complexity of the encounters in this trial was: _ Much easier to detect and resolve _ Somewhat easier to detect and resolve _ Neither easier nor more difficult to detect and resolve _ Somewhat more difficult to detect and resolve _ Much more difficult to detect and resolve Rev 20150731 1 Date: / / Time: UTC /Local Controller #: Workload Q5. As compared with normal real - world operations, how would you rate your workload during this scenario?
_ Much lower than normal _ Somewhat lower than normal _ Typical _ Somewhat higher than normal _ Much higher than normal Q6. The complexity of the scenario negatively impacted my ability to provide additional services (direct routings, traffic advisories, etc.)?
_ Strongly disagree _ Somewhat disagree _ Neutral _ Sligh tly Agree _ Strongly Agree Rev 20150731 2 Date: / / Time: UTC /Local Controller #: Run # : ATC Controller Questionnaire for Flight Test 3, Configuration 2 (IT&E Form 2 – Post Run) Simulation Environment Q1. Did you notice any act ion or behavior (ghost controller, pilot, software, etc.) in the simulation that did not conform to your expectations or that was unlike behavior in the real world?
_ Uncertain _ No _ Yes (if yes, explain): Rev 20150731 1 Date: / / Time: UTC /Local Controller #: ATC Controller Questionnaire for Flight Test 3, Configuration 2 (IT&E Form 3 – Exit) Q1. As compared to current operational air traffic control displays, the quality of the simulated ATC display (user interface, interaction, etc.) used in the simulation was adequate for controlling air traffic and performing routine tasks: _ Strongly disagree _ Somewhat disagree _ N eutral _ Slightly Agree _ Strongly Agree Q2. The simulated ATC display was a realistic emulation of the controller workstation.
_ Strongly disagree _ Somewhat disagree _ Neutral _ Slightly Agree _ Strongly Agree Comments: Q3. What enhancements would you like to see added to the controller station to allow you to accomplish your tasks, beyond what is already available today?
Comments: Rev 20150731 1 Date: / / Time: UTC /Local Controller #: Q4. What features would you like to see in controller display to provide situation awareness of UAV aircraft?
Comments: Rev 20150731 2 Date: / / Time: UTC /Local Controller #: Ghost Controller Questionnaire for Flight Test 3, Configuration 2 (IT&E Form 4 – Post Run) Q1. As compared with previous data runs, how would you rate your workload during this scenario ?
_ Much lower than normal _ Somewhat lower than normal _ Typical _ Somewhat higher than normal _ Much higher than normal Q2. Did the workload and complexity of this data run cause you to perform any additional coordination or aircraft maneuvering to set up the encounters (not including conflict avoidance maneuvers)?
_ Much less than normal coordination and aircraft maneuvering _ Somewhat less than normal maneuvering _ Typical maneuvering to set up encounters _ Greater than normal maneuvering to set up encounters _ Much greater than normal maneuvering to set up encounters Rev 20150731 1 Date: / / Time: UTC /Local Pilot #: Run # : Pilots Questionnaire for Flight Test 3, Configuration 2 (IT&E Form 5 – Exit) Voice Communication System Q1. The clarity of the voice system is: _ Much clearer than normal _ Somewhat clearer than normal _ Typical _ Somewhat noisier than normal _ Much noisier than normal Q2. Did you observe any delay in response time from the voice sys tem?: _ Very noticeable _ Somewhat noticeable _ Not noticeable Q3. The amount of frequency congestion is: _ Much more congested than normal _ Somewhat more congested than normal _ Typical _ Somewhat less congested than normal _ Much more congested than normal Physical Simulation Environment Q4a. The lighting at the pilot station is: _ Much darker than normal _ Somewhat darker than normal _ Typical level _ Somewhat brighter than normal _ Much brighter than normal Rev 20150731 1 Date: / / Time: UTC /Local Pilot #: Run # : Q4b. Would you like ability to adjust the lighting at the pilot station?
_ Yes _ No Comments: Q5. The ambient noise level at the pilot station is: _ Much quieter than normal _ Somewhat quieter than normal _ Typical noise level _ Somewhat louder than normal _ Much louder than normal What background or ambient noise sources are missing at the pilot station?
What background ambient or noise were a distraction at the pilot station?
Q6. Interac tion levels with other people (i.e. mission coordinator) or systems (besides Vigilant Spirit) is: _ Much less interaction than normal _ Somewhat less interaction than normal _ Typical level of interaction _ Somewhat more interaction than normal _ Much more interaction than normal Rev 20150731 2 Q7a. The temperature at the pilot station is: _ Too hot _ Just right _ Too cold Q7b. Would you like the ability to adjust the temperature at the pilot station?
_ Yes _ No Comments: Rev 20150731 3
Appendix B – Enlarged Full Mission System Design
Appendix B – Enlarged Full Mission System Design
References
Marston, M., Sternberg, D., Valkov, S., “ UAS -‐ NAS ITE Flight Test Series 3, Flight T est Report, Rev B ” , NASA Armstrong Flight Research Center, Oct. 2015 Department of Defense: “Modeling and Simulation Master Plan ,” DoD 5000.59P, Oct 1995 Henninger, Amy E., Cutts, Dannie, Loper, Margaret, et al, “Live Virtual Constructive Architecture Roadmap (LVCAR) Final Report”, Institute for Defense Analysis, Sept, 2008 Bezdek, W. J., Maleport, J., Olshon, R., “Live, Virtual & Constructive Simulation for Real Time Rapid Prototyping, Experimentation and Testing u sing Network Centric Operations, ” AI AA 2008 -‐ 7090, AIAA Modeling and Simulation Technologies Conference and Exhibit, August 2008 Murphy, J. Jovic, S., Otto, N., “Message Latency Characterization of a Distributed Live, Virtual, Cons tructive Simulation Environment, ” AIAA Infotech@Aerospace Conference, January 2015 Suarez, B., Kirk, K., Theunissen, E., “Development, Integration and Testing od a Stand -‐ Alone CDT I with Conflict Probing Support, ” AIAA 2012 -‐ 2487, AIAA Infotech@Aerospace, June 2012 Santiago, C., Mueller, E., “Pilot Evaluation of a UAS Detect -‐ and -‐ Avoid System’s Effect iveness in Remaining Well Clear, ” Eleventh UAS/Europe Air Traffic Management Research and Development Seminar (ATM2015), June 2015 Munoz, C., Narkawicz, A., Hagen, G., Upchurch, J., Dutle, A., Consiglio, M., Chamb erlain, J., “DAIDALUS: th Detect and Avoi d Alerting Logic for Unmanned Sy stems,” 2015 IEEE/AIAA 34 Digital Avionics Systems Conference (DASC), September 2015 NASA Armstrong Flight Research Center, “Integrated Test and Evaluation, Objectives & Requirements Document (ORD), Flight Test 4 (FT -‐ 3), NASA IT&E ORD -‐ 01 Rev A,” March 2015 Software website: http://www.pitch.se, April 201 6 Prevot, T., Mercer, J., "MACS -‐ A Simulation Platform for Today's and Tomorrow's Air Traffic Operations", AIAA MST Conference and Exhibit, August 20 -‐ 23, 2007 Hilton Head, SC, AIAA -‐ 2007 -‐ 6556 Prevot, T., “Exploring the Many Perspectives of Distribu ted Air Traffic Management: The Multi Aircraft Control System (MACS ),” International Conference on Human -‐ Computer Interaction Aeronaut ics, HC I -‐ Aero 202, 23 -‐ 25 October 2002, Massachusetts Institute of Technology, Cambridge, MA Prevot, T. , Smith, N. M. , Palmer, E. , Callantine, T. J. , Lee, P. U. , Mercer, J. , Martin, L. , Brasil, C. , and Cabrall, C . "An Overview of Current Capabilities and Research Activities in the Airspace Operations Laboratory at NASA Ames Research Center", 14th AIAA Aviation Technology, Integration, and Operations Conference, AIAA Aviation, (AIAA 2014 -‐ 2860).
RTCA: “ Minimum Operational Performance Standards (MOPS) for Aircraft Surveillance Applications System (ASAS) ,” RTCA DO -‐ 317, Apr 14 , 20 09 Feitshans, G. L., Rowe, A. J., Davis, J. E., Holland, M., and Berger, L., “Vigilant Spirit Control Station (VSCS) -‐ “The Face of COUNTER”,” Proceedings of the AIAA Guidance, Navi gation and Control Conference and Exhibition, Honolulu, Hawaii, AIAA 2008 -‐ 6309, August 2008.
Soler, G., Jovic, S., Murphy, J.R., "RUMS -‐ Realtime Visualization and Evaluation of Live, Virtual, Constructive Simulation Data", AIAA SciTech, Jan. 5 -‐ 9, 2015, AIAA -‐ 2015 -‐ 1648 Doc. No. FAA -‐ 2007 -‐ 29305, 75 FR 30194, May 28, 2010; Amdt. 91 -‐ 314 -‐ A, 75 FR 37712, June 30, 2010; Amdt. 91 -‐ 316, 75 FR 37712, June 30, 2010