Document
WHAT WOULD YOU HAVE DONE?
LEARNING FROM OTHERS’ AVIATION SAFETY EVENTS Jolene Feldman NASA Ames Research Center Moffett Field, CA Sepehr Bastami & Jon Holbrook NASA Langley Research Center Hampton , VA The capacity to learn from both expected and unexpected events has been identified as a critical component of resilient performance. One’s own experiences, however, are not the only opportunities to learn from these types of events. In the aviation domain, operator - submitted safety event reports can also provide a rich source of opportunity for learning from what others have experienced. In addition to maintaining the world’s largest collection of voluntarily submitted aviation safety incident reports, NASA’s Aviation Safety Reporting System (ASRS) also publishes a monthly online safety newsletter, CALLBACK , which periodically features a segment called “What Would You Have Done?” These segments offer readers a chance to “interact” with information from previously submitted reports. Readers are presented with “the first half of the story”, describing a situation leading up to a critical decision, and then asked to exercise their own judgment and decision - making skills by thinking through “what would you have done?”. Later in the newsletter, “the rest of the story” is presented, so readers can see what actions were taken by the incident reporters. The intent of this feature is to “stimulate thought, training, and discussion” related to the reported incidents. There is a rich research literature on the learning benefits of elaborating and organizing information. The current study seeks to expand on the question “what would you have done?” to explore further ways to consolidate learning from others’ experiences, using reports submitted to ASRS by Un crewed Aircraft Systems (UAS) operators.
“I was flying legally at 120 ft. above the tree tops outside of Class C [airspace] when two alerts popped up on my screen saying that aircraft were approaching at the same altitude. I quickly turned the UAS around and spotted a Blackhawk helicopter (UH - 71) at 500 ft. which went to my east…. The other I spotted at the same altitude approaching fast. I was headed to my landing spot but would have had to climb to around 300 ft. due to terrain even through the drone was in visual line of sight the whole time…” Now reader, the question is, “What would you have done?” This is an excerpt from the narrative section of an incident report submitted to NASA’s Aviation Safety Reporting System (ASRS) database. This database currently houses over 1.7 million safety reports. In addition to reports submitted by commercial and general aviation pilots, maintenance personnel, air traffic controllers , and others, in 2021 the ASRS report submission form was updated, with inputs and in tandem with the FAA and industry, to include reports for Uncrewed Aircraft Systems ( UAS ) /drone operators and about UAS operations . In addition to this growing repository of reports, the ASRS provides newsletters that highlight various emerging safety topics (e.g., lost link, procedural issues, collisions). Like the original ASRS CALLBACK monthly safety newsletter, a newsletter dedicated to UAS/ d rone operations called UAS Safety In Sight is available . Both newsletters include distinctive issues entitled, “What Would You Have Done?” These issues provide readers with the first half of a narrative, up to a decision point, and then pose a question to the reader, “What would you have done?” Readers are encouraged to reflect and exercise their knowledge, expertise, experience, and decision - making skills. Then, they are invited to read the second half of the narrative to learn what happened and what actions were or weren’t taken by the reporter . In the second half of the event describe d above , the reporter , “…made the decision that this was an emergency and it was safer to try and put the drone as close to the treetops without crashing and hope that they didn’t lose altitude… I was prepared to crash my drone into the trees if needed but that could have caused a forest fire which would have put the nearby homes and residents at risk …” ( A SRS Report A cc e s sion No. 1845815 ) Much can be learned from others’ experience. It is possible that even reading a single ASRS narrative can have such an effect. Every submitted report can be viewed as an opportunity to learn from others. This knowledge can be further expanded by posing questions, like “what would you have done ? ” , that encourage readers to reflect upon what they know, learn what worked well or didn’t in that situation , and how they might carry these “lessons learned” with them going forward – ideally, contributing to individual and airspace safety.
The purpose of this work is to describe an approach to utilize ASRS reports and newsletters to develop stimuli to expand on learning from others’ experience . Because these data sets are already available , their use in combination with simple prompts such as “what would you have done ? ” represents a learning opportunity requiring minimal resources . It is possible to identify exemplar narratives, create a series of meaningful prompts to elicit reader engagement with the material, and add more value to the existing material which can be useful for individual operators as well as for more formal training purposes.
While n arrative databases such as ASRS afford opportunities to learn about safety based on analyses of patterns and trends across incidents , i ndividual narratives also represent opportunities for learning from others’ experiences. Speer et al., (2009) recorded brain activity using functional magnetic resonance imaging (fMRI) while participants read short narratives.
The same brain regions activate d when people perform ed , observe d , imagine d , or read about real - world activities. Findings such as this indicate the potential value of using stories as learning opportunities. Maximizing that learning potential can depend on how the learner interacts with the material to be learned. For instance, “cognitively active” learning is generally superior to “cognitively passive” learning in educational settings (Stanger - Hall, 2012), and there is a rich research literature on the learning benefits of factors including motivation (Weiner, 1966); rehearsal and practice (Craik & Lockhart, 1972); elaboration (e.g., Yogo & Fujihara, 2008; McLeod et al., 2010); spacing of practice (e.g., Ebbinghaus, 1885); organizing information (e.g., Bellezza, 1981), etc.
Activities such as the “what would you have done?” feature d in ASRS CALLBACK and UAS Safety I n Sight instantiate some of these principles of elaboration and organizing information to promote cognitively active interactions with event narratives. The reader is invited to not only read the narratives, but to consider what they would have done in that situation and whether they agree with the reporter’s decision. Indeed , several additional elaborative questions could also be asked, such as: • Do you agree with the r eporter’s decision , or would you have taken a different action?
(this question already included in UAS Safety In Sight ) • How is your solution similar and different from what the event crew did?
• What else could the event crew have done?
• Of all the solutions you considered, which do you think would have worked best? Why?
• What do you think were the “learning moments” or “teachable moments”?
• What could you take from this situation to add to your own “strategy toolbox”?
• Does this situation remind you of one that you have personally experienced? What do you think contributed to your own situation working out successfully or unsuccessfully?
Not every event narrative, however, lends itself to this type of elaboration. Some narratives, for instance, may be too sparse in detail, or describe a situation that has only a single obvious “right” course of action. Thus, the initial task of identifying appropriate event narratives could represent a challenge for this learning approach. A primary goal of the current work was to identify properties of event narratives that represent good candidates for a “what would you have done” task and explore approaches to more easily identifying those candidates.
Method Using the ASRS database online search tool, 165 event narratives filed by UAS operators were identified. The date range for the events was April 2021 to September 2024, with the Reporter Organization identified as Commercial Operator (UAS) for 79 reports and Recreational/Hobbyist (UAS) for 86 reports. The narratives were distributed among the authors for coding, with each narrative initially coded individually. Narratives for which there were coding questions or uncertainties were discussed together by all three authors and coded to consensus. Th ey were characterized based on specific report coding criteria , as follows : 1. Length of narrative in words 2. Was there at least one decision point in the narrative for which there was some uncertainty about the outcome?
3. Did the narrative describe a situation that went beyond ignorance of either regulations or standard operating procedures (SOPs) ?
4. Did the narrative provide sufficient information/detail to ask, “what would you have done” (WWYHD) ?
5. If the answers to questions 2, 3, and 4 were all “Yes”, the event was coded as a possible “ WWYHD? ” candidate 6. Did the narrative contain explicit discussion of goal tradeoffs?
7. Did the reporter describe what they were thinking?
Results Using the coding criteria describe above , 27 of the 165 (16%) reports met the criteria for “WWYHD?” candidacy . In comparing report length by coding factor, two - sample t - tests were performed. S ignificant difference s were found between report length and all coding factor s, except for whether the report describe d situation s that went beyond ignorance of regulations or SOPs (i.e., knowledge of airspace requirements , rules, restrictions) . When a significant difference was found, reports that received “Yes” codes for that factor were longer than those that received “No” codes (Table 1). Word count did not differ between recreational (median = 166) and commercial (median = 170) operator reports (t[1,163] = 1.388, p = 0.167).
Table 1 Report word count by coding factors.
"Yes" codes "No" Codes N Mean StDev N Mean StDev t(1,163) P-value Included critical 52 330.08 333.57 113 159.91 124.08 4.766 < .0001 decision point Beyond ignorance 87 227.85 270.41 78 197.58 165.30 0.856 0.394 of regs/SOPs Sufficient 40 395.05 368.79 125 155.46 105.75 6.510 < .0001 detail "WWYHD?"
27 415.74 408.93 138 173.98 142.00 5.501 < .0001 candidate Discussed goal 30 365.60 315.04 135 179.75 187.37 4.269 < .0001 tradeoffs Described their 106 269.01 262.56 59 113.88 69.46 4.447 < .0001 thinking Differences between recreational and commercial operators reports by coding factors were explored. Of the coded factors, only “Beyond ignorance of regulations or SOPs” was significantly associated with operation type, with more reports filed by recreational operators than commercial operators associated with ignorance of regulations and SOPs (Table 2).
Table 2 Operation type by coding factors.
Critical Beyond Discussed Described Sufficient "WWYHD?"
decision ignorance of goal their detail candidate point regs/SOPs tradeoffs thinking Y N Y N Y N Y N Y N Y N Recreational 24 62 36 50 17 69 11 75 15 71 50 36 Commercial 28 51 51 28 23 56 16 63 15 64 56 23 χ2 (1, N = 165) 1.68 2.91 1.08 8.51 1.96 0.07 P-value 0.162 0.196 0.797 0.088 0.298 0.004 Figure 1 shows the distribution of identified “what would you have done?” candidates by word count. A median split of the reports (i.e., word counts above and below the median of 170 words) showed that , for those below the median, 3.6% were identified as possible candidates; while for those above the median, 29.3% were identified as possible candidates.
Figure 1 Distribution of identified “what would you have done?” candidates by report word count.
Discussion Learning opportunities from others’ stories exist through ASRS narratives and the constructive questions posed in ASRS newsletters. Ways to enhance these rich sources were explored. Coding specific properties of narratives and developing an approach to identify candidate reports for “what would you have done” material was described and showed that w ord count could be used as an easily computed initial screening criteria for identifying “what would you have done” candidates. Reports shorter than 170 words may be unlikely to be productive for this task. Focusing on reports over 170 words, however, may significantly increase the “hit” rate for identifying candidate reports. In the current sample, for example, the overall candidacy rate of 16% improved to 29.3% for reports over 170 words. This does not imply, of course, that shorter reports cannot have value for other purposes.
It is possible that v arious data mining techniques to query substantiative databases could also be valuable in expanding these efforts . N atural language processing (NLP) tools to explore aviation report narratives in the ASRS database were developed by NASA over twenty years ago (McGreevy, 2005) . Efforts to develop these i terative text - based search tools and systematic search methods using data mining techniques continue today ( e.g., Paradis, 2025). Application of these and other NLP tools to the rapid identification of “what would you have done?” candidates will be explored.
Additional planned next steps include using the identified set of “what would you have done?” candidates to systematically explore how particular parameters impact learning, ranging from what specific questions are asked, response medium (e.g., internally generated, written, verbal), and learning setting (e.g., individually or in a group). Applying established principles of learning can help to make the most of safety learning opportunities. Individuals can apply these principles on their own to reinforce learning from their own and others’ experiences. Likewise, organizations can apply these principles through policies that help preserve, reinforce, extend, and expand good practices.
Acknowledgments The authors would like to thank the members of NASA’s Human Contributions to Safety team for engaging discussions on this topic . This work was supported by NASA’s System - Wide Safety project, under the Airspace Operations and Safety Program within the Aeronautics Research Mission Directorate.
References Bellezza, F. S. (1981). Mnemonic devices: Classification, characteristics, and criteria.
Review of Educational Research , 51 (2), 247 - 275.
Craik, F. I., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research.
Journal of verbal learning and verbal behavior , 11 (6), 671 - 684.
Ebbinghaus, H. (1885). Forgetting curve. Memory A Contribution to Experimental Psychology , 76157803 - 1.
McGreevy, M. W. (2005). Perilog text mining methods and software. Moffett Field, CA: NASA Ames Research Center.
McLeod, J. M., Kosicki, G. M., & McLeod, D. M. (2010 ). Levels of analysis and communication science . na.
Paradis, C. V., Hong, C., Matthews, B., Davies, M. D., & Hooey, B. (2025). Kaona: Deep Searching and Curating Aviation Safety Reporting Systems. In AIAA SCITECH 2025 Forum (p. 2152).
Speer, N. K., Reynolds, J. R., Swallow, K. M., & Zacks, J. M. (2009). Reading stories activates neural representations of visual and motor experiences . Psychological science , 20 (8), 989 - 999.
Stanger - Hall, K. F. (2012). Multiple - choice exams: an obstacle for higher - level thinking in introductory science classes . CBE — Life Sciences Education , 11 (3), 294 - 306.
Weiner, B. (1966). Motivation and memory. Psychological Monographs: General and Applied , 80 (18), 1.
Yogo, M., & Fujihara, S. (2008). Working memory capacity can be improved by expressive writing: A randomized experiment in a Japanese sample. British Journal of Health Psychology , 13 (1), 77 - 80.