General Aviation Pilots' Perceived Usage and Valuation of Aviation Weather Information Sources

NASA/TM-2002-211443

E I . i

General Aviation Pilots' Perceived Usage and

Valuation of Aviation Weather Information

L

Sources

Kara Latorella and Suzanne Lane Langley Research Center, Hampton, Virginia Daniel Garland Embry-Riddle Aeronautical University, Daytona Beach, Florida

February 2002

The NASA STI Program Office... in Profile CONFERENCE PUBLICATION.

Since its founding, NASA has been dedicated to the advancement of aeronautics and space Collected papers from scientific and science. The NASA Scientific and Technical technical conferences, symposia, Information (STI) Program Office plays a key seminars, or other meetings sponsored or part in helping NASA maintain this co-sponsored by NASA.

important role.

SPECIAL PUBLICATION. Scientific, The NASA STI Program Office is operated by technical, or historical information from Langley Research Center, the lead center for NASA programs, projects, and missions, NASA's scientific and technical information.

often concerned with subjects having The NASA STI Program Office provides substantial public interest.

access to the NASA STI Database, the largest collection of aeronautical and space TECHNICAL TRANSLATION. English- science STI in the world. The Program Office language translations of foreign scientific is also NASA's institutional mechanism for and technical material pertinent to disseminating the results of its research and NASA's mission.

development activities. These results are published by NASA in the NASA STI Report Series, which includes the following report Specialized services that complement the types: STI Program Office's diverse offerings include creating custom thesauri, building customized TECHNICAL PUBLICATION. Reports of databases, organizing and publishing completed research or a major significant research results.., even providing videos.

phase of research that present the results of NASA programs and include extensive For more information about the NASA STI data or theoretical analysis. Includes Program Office, see the following: compilations of significant scientific and technical data and information deemed • Access the NASA STI Program Home to be of continuing reference value. NASA Page at http:llwww.sti.nasa.gov counterpart of peer-reviewed formal professional papers, but having less • "Email your question via the Internet to stringent limitations on manuscript length and extent of graphic help@sti.nasa.gov presentations.

Fax your question to the NASA STI TECHNICAL MEMORANDUM.

Help Desk at (301) 621-0134 Scientific and technical findings that are preliminary or of specialized interest, Telephone the NASA STI Help Desk at e.g., quick release reports, working (301) 621-0390 papers, and bibliographies that contain minimal annotation. Does not contain Write to: extensive analysis.

NASA STI Help Desk CONTRACTOR REPORT. Scientific and NASA Center for AeroSpace Information 7121 Standard Drive technical findings by NASA-sponsored Hanover, MD 21076-1320 contractors and grantees.

m

NASA/TM-2002-211443

General Aviation Pilots' Perceived Usage and

Valuation of Aviation Weather Information

Sources

Kara Latorella and Suzanne Lane Langley Research Center, Hampton, Virginia Daniel Garland Embry-Riddle Aeronautical University, Daytona Beach, Florida National Aeronautics and Space Administration Langley Research Center Hampton, Virginia 23681-2199

February 2002

Acknowledgments This survey was originally developed by Dr. Daniel Garland, then at Embry-Riddle Aeronautical University, under the auspices of the NASA Aviation Weather Information (AWIN) Operator Support team, then led by Dr. Marianne Rudisitl. We are grateful for the assistance of Search Technologies, Inc. for hosting the website and thereby enabling online participation. Dr. Daniel Garland provided NASA AWlN with the data so acquired. Ms. Suzanne Lane, as a Langley Summer Scholar, and Dr. Kara Latorella analyzed and interpreted the results. We greatly appreciate the careful and thoughtful reviews by Mr. Paul Stough, AWlN Lead, and Ms. Monica Hughes of NASA Langley's Crew/Vehicle Integration Branch, and Mr. Jim Chamberlain of NASA Langley's Crew/Systems Operations Branch.

We are also grateful to the many anonymous participants who responded to this survey in the spirit of volunteerism and improving aviation safety.

Available from: NASA Center for AeroSpace Information (CASI) National Technical Information Service (NTIS) 7121 Standard Drive 5285 Port Royal Road Hanover, MD 21076-1320 Sprhlgfield, VA 22161-2171 (301) 621-0390 (703) 605-6000 Table of Contents Introduction .................................................................................................................................................................. i Objective ...................................................................................................................................................................... i iii List of Figures Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure I 1. Unavailability of In-Flight Weather Services and Products ................................................................. !9 Figure Figure Figure Figure Figure Figure Figure List of Tables iv ABSTRACT Aviation suffers man), accidents due to the lack of good weather information in flight. Existing aviation weather information is difficult to obtain when it is most needed and is not well formatted for in-flight use.

Because it is generally presented aurally, aviation weather information is difficult to integrate with spatial .flight information and retain .for reference. Efforts, by NASA s Aviation Weather Information (AWIN) team and others, to improve weather information accessibiliO,, usabilio_' and decision aiding will enhance General Aviation (GA) pilots weather situation awareness and decision-making and therefore should improve the safeO' of GA flight. Consideration of pilots economic concerns will ensure that in-flight weather information systems are ftnancially accessible to GA pilots as well. The purpose of this survey was to describe how aviation operator communities gather and use weather information as well as how weather related decisions are made between flight crews and supporting personnel. Pilots of small GA aircraft experience the most weather-related accidents' as well as the most fatal weather related accident. For this reason, the survey design and advertisement focused on encouraging participation from GA pilots.

Perhaps as a result of this emphasis, most responses, 97 responses or 85%0 of the entire response set, were.from GA pilots. This paper presents only analysis of these GA pilots' responses. The insights provided by this survey regarding GA pilots perceived value and usage of current aviation weather information, services, and products provide a basis.for technological approaches to improve GA safety. Results of this survey are discussed in the context of survey limitations and prior work, and serve as the foundation for a model of weather information value, guidance for the design of in-flight weather information systems, and definition of further research toward their" development.

= Introduction The statistics concerning small general aviation (GA) aircraft and weather related accidents reveal a clear need for improvement. Eighty-five percent of aviation accidents occurring from 1990-1996 involved small GA aircraft, and weather was considered a factor or a cause in nearly a third of these, often fatal, accidents [1]. Unfortunately, many of these accidents may be attributed to a fairly obvious problem, lack of available usable weather information for in-flight use in the GA pilot's cockpit. General aviation is particularly affected by weather because GA pilots today rely primarily on external, or "out-the-window," weather cues for weather information and aural weather information sources. Unlike larger aircraft, most small GA aircraft are not equipped with onboard weather detection equipment. In addition, onboard weather systems that are available for small GA aircraft are typically relatively expensive and limited in performance by size and power constraints. Aural weather information sources can include direct queries to Flight Service Station (FSS), En Route Flight Advisory Service (EFAS, or "Flight Watch"), calls to Air Traffic Control (ATC) personnel, and monitoring frequencies to hear other pilots' comments.

Unfortunately, the frequencies used to obtain this information often become saturated, limiting access to the information at times when it is needed most. Pilots can also tune radios to receive automated weather services such as Hazardous In-flight Weather Advisory Service (HIWAS), Automated Weather Observing System (AWOS) / Automated Surface Observing System (ASOS), and Automatic terminal Information Service (ATIS) to obtain a broadcast over large areas or specific reporting stations. However, the information from these aural sources is limited and may be relevant only for a very localized area. It is clear that more accessible, complete, and usable weather information needs to be made available to GA pilots in order to reduce the number and severity of weather related accidents in this group. Reducing GA weather related accidents is one of the primary goals of the Aviation Weather Information (AWlN) element of NASA's Aviation Safety Program. AWIN will decrease accident rates by improving the availability and usability of airborne weather information. Specifically, AWIN is developing technology and design/use guidelines that provide improved in-flight weather information via graphical displays of data-linked weather information. NASA researchers expect that this will ultimately result in safer flights by improving pilots' in-flight weather situation awareness and decision quality.

Objective Surveys, like the one detailed in this report, investigate how pilots gather and use weather related information during preflight and while in-flight. The purpose of this specific survey is to describe how aviation operator communities gather and use weather information, as well as how weather related decisions are made between flight crews and supporting personnel. This report presents and analyzes data collected by a survey entitled "Aviation Weather Services and Products." The information collected in this survey was used to obtain pilots' opinions of their acquisition and use of aviation weather information, and perspectives on implementation issues. Topics address: the use and importance of specific information during different flight phases, knowledge and use of specific weather products and services currently available, and preferences associated with the cost and methods of payment for a flight deck weather information system. Respondents were asked to characterize themselves according to flight experience, typical flight region, and aircraft types flown and hours in those types. This information will aid in the design of in-flight weather information systems. It is anticipated that use of such systems will reduce the number of weather related accidents involving small GA aircraft, as well as other aircraft.

Methods This survey was developed for the NASA Langley AWIN (Aviation Weather Information) element by Dr. Dan Garland, on leave from Embry-Riddle Aeronautical University, then working for SA Technologies, Inc. The survey was hosted at Search Technologies, Inc. Table 1 lists website locations that linked to the survey website. The survey (Appendix A) consists of 136 questions. Major topics include: Demographic Information (8 questions) Preflight Weather Information (15 questions), Preflight Weather Services and Products (30 questions), Internet Website Usage (12 questions), In-Flight Weather Information (19 questions), In-Flight Weather Services and Products (33 questions), Usefulness of Weather Data During Specific Phases of Flight (10 questions), and Weather Technology Cost (3 questions) and Implementation Issues (6 questions).

Table I. Websites with a Link to the Survey.

Organization Website URL (when survey conducted) AOPA www.aopa.org/members/wx Aviation Digital Data Service www.adds.awc-kc.noaa.gov Aviation Weather Center www.awc-kc.noaa.gov National Weather Service www.nws.noaa._ov DUATS www:sk_ycentral.gte.com Intellicast www.intellicast.com/weather.usa.others Purdue University www.wxp.atms.purdue.edu University of Wyoming www.das.uwyo.edu Colorado State University www.cira.colostate.edu/ramm/adimtFy/toc.htm NCAR www.rap.ucar.edu.weather Acuweather www.accuwx.com Naval Research www.nr/mry.navy.mil/sat-products.htm Nexrad www.cirrus.sprl.umich.edu/wxnet.radsat.html www.erau.edu Embry Riddle Aeronautical University www.searchtech.com Search Technology , _±:: Results One hundred and fourteen individuals provided survey responses. Ninety-seven of these, roughly 85%, identified themselves as GA pilots. The group also consisted of a small number of business jet/corporate, charter, major commercial airline, military, regional commercial airline, and student pilots; and a few dispatchers. Due to the scarcity of data for these other user groups, this paper only analyzes responses from the GA pilots.

Flight Experience and Area of Flight Hours of total flight experience ranged between 20 and 10,000 hours with an average of 1553.87 hours and a standard deviation of 2189.43 hours. The distribution of total flight hours reported by these GA respondents is heavily skewed towards lower flight experience (skewness = 2.136) and is also fairly peaked (kurtosis = 4.224). Approximately 50% of the respondents had fewer than 587 flight hours, and 80% had fewer than 2100 flight hours. When these pilots were asked their typical region of flight, the most common response, 28.9%, was the Northeast region of the United States. Other common responses included the Southeast (22.7%), Southwest (14.4%), Midwest (18.6%), and West (13.4%) (Figure I).

Other West 1% 13% Northeast 29% Southwest 14% \ Midwest 19% Southeast 23% Trans Continental U.S.

1% Figure 1. Flight Regions.

Usefulness of Weather Information During Phases of Flight In order to investigate the changing importance of weather information during flights, respondents rated the usefulness of weather information during these flight phases: Preflight, Taxi-out, Take-off, Climb, Cruise, Descent, Approach, Landing, and Taxi-in. Pilots were given the options of, "essential," "very useful," "useful," "limited," and "no utility" to rank the usefulness of weather information during each individual phase (Figure 2). The greatest number of respondents regarded weather information most essential for the preflight, cruise, approach, and landing phases of flight. Respondents considered weather information to be very useful during the descent period. Information during taxi-out/in and take-off phases was characterized by most respondents as having limited usefulness.

9O

8O 7O

E 5o

• 40 ++ ++, o.,oo oo.G.o.o, ,=, [ @Essential • Very Useful J Figure 2. Usefulness of Weather Information during Flight Phases.

Preflight Weather Information During preflight preparation, a Variety of information is available that describes current and future weather conditions. Respondents were asked to rate the importance of several sources of weather information with respect to usefulness flight planning. ResponSe options included: "information not available, "not at all important," "somewh._t important," very important, and 'extremely important" (Figure 3). Ceilings (IFR), Ceilings (VFR), Ceilings (MVFR), Convective Activity (I_S_WR- Terminal), Convective Activity Nowcast - Enroute, Icing - Airborne (known and forecast), Icing - Ground conditions and Forecast, Lightning, SIGMETS, and Visibility were all frequently rated as extremely important sources of preflight weather information. Turbulence (known and forecast) as well as Surface Wind informationwasmostfrequently considered veryimportant. ;remperature (Density Altitude) was most often considered somewhat important. Certain weather information sources are not considered to be available to the respondents for preflight planning (Figure 4). Approximately 26% of the respondents did not think they had access to convective enroute - nowcast information during preflight planning. All other sources were considered to be available to at least 87% of responses. Table 2 provides descriptive statistics for ratings associated with this question, and one-tailed (>3) t-test statistic and significance level for each information source/service/product.

G) O 0 m O O 0 _ t_ t,,- U i__

-ooo oo v

[] Extremely Important[] Very Importan_ Figure 3. Importance of Preflight Weather Information.

e_ ,i m e- = O'J C u k.

G) fl.

Figure 4. Unavailable Preflight Weather Information.

"Fable 2. Prefli Weather InformationRatin 95 1 4 • 3.38 4 4 4.770 0.000 Ceilings (VFR) Ceilings (MVFR) 90 1 4 3.33 4 4 3.672 0.000 Convective - Terminal 85 2 4 3.46 4 4 6.365 0.000 Convective - Enroutc Nowcast 69 2 4 3.51 4 4 6.216 0.000 94 1 4 3.43 4 4 4.861 0.000 Airborne Icin E 89 1 4 3.16 3 4 1.521 0.066 Ground Icing, 88 1 4 3.15 3 4 1.533 0.065 Lightning SIGMETS 95 2 4 3.38 3 4 5.502 0.000 96 1 4 2.53 2 2 -6.213 NS Temperature Turbulence 94 1 4 2.80 3 3 -2.592 NS 97 2 4 3.27 3 4 3.552 0.00 !

Visibility Winds Aloft 97 2 4 2.71 3 2 -4.024 NS Surface Winds 97 I 4 2.90 3 3 - 1.452 NS * Because these tests are one-tailed, negative t-statistics imticate non-significant results, and therefi_re p-values are not reported.

Respondents were invited to suggest other sources of preflight weather information, and rate these as weii. The most commonly reported omissions were Precipitation Trends (3 respondents) and Pilot Reports (3 respondents), with 2 respondents indicating that Cloud Tops should have been listed and one indicating that NOTAMS should have been listedl Pilot Reports were included in another section of the survey as a Service. Rating responses for Pilot Reports (PiREPS) are covered in the next section.

Preflight Weather Services and Products During preflight preparation, a variety of additional aviation weather services and products are available to inform pilots about weather conditions. Respondents were asked to rate the importance of specific sources listed with respect to the usefulness of these for flight planning preparations. Respondents were asked to use the same five importance ratings as used in the previous question. Figure 5 presents those with rankings of "extremely" or "very" important. The Direct User Access Terminal System (DUATS), Flight Service Station Briefing, METARS, and Terminal Aerodrome Forecast (TAF) sources were most often rated as extremely important. "Very Important" was the most common response for Aviation Area Forecast (FA), Convective Outlook, Internet, Radar Summary Charts, Severe Weather Watch Bulletin (WW), Surface Analysis Charts, Weather Depiction Charts, and Winds and Temperatures Aloft Forecast (FD). Services and products described most frequently as somewhat important include: Automated Surface Observation System (ASOS), Center Weather Advisory (CWA), Composite Moisture Stability Charts, Constant Pressure Analysis Charts, Pilots by Word of Mouth, Television, and Transcribed Weather Broadcast (TWEB). Respondents most commonly considered the Newspaper "not at all important." Pilot's Automatic Telephone Answering System (PATWAS) was most commonly rated as either not available or unimportant. Table 3 provides descriptive statistics for ratings associated with this question, and one-tailed (>3) t-test statistic and significance level for each information source/service/product. Meteorological Impact Statements (MIS) (53 reports) and Radiosonde Additional Data reports (RADAT) (49 reports) were often cited as unavailable (Figure 6).

Figure 5a. Importance of Preflight Weather Services and Products.

Figure 5b. Importance of Preflight Weather Services and Products (continued).

t- O Q.

iv, G) Ill r- U i._ 13.

_ o_ _

17 +i

[] Extremely Important• Very Important Figure 5c. Importance of Preflight Weather Services and Products (continued).

Table 3. Prefli ht Weather Services and Products Ratin ASOS AWOS 84 1 ] 4 2.68 3 2 -3.861 NS FA 94 1 ] 4 2.96 3 3 -.542 NS CWA 75 I 4 ] 2.61 2 2 -3.725 NS Composite Moisture Stability 69 I 3 I 1.78 2 2 -15.291 NS Constant Pressure Analysis 71 I 4 ] 1.80 2 2 -13.473 NS Convective Outlook 87 1 4 2.82 3 3 -I .812 NS DUATS 92 I 4 3.28 4 4 2.916 0.002 96 1 4 3.46 4 4 5.966 0.000 FSS Briefing T_ Intcrnet 91 1 4 2.9 3 3 -1.026 NS METARS 85 I 4 3.18 3 4 1.919 0.029 MIS 34 I 3 1.91 2 2 -8.911 NS 84 1 3 1.32 1 1 -29.666 NS Newspaper PATWAS 62 1 4 1.71 2 1 -13.456 NS PIREPS 95 I 4 3.05 3 4 .600 0.275 Other Pilots 88 t 4 2.25 2 2 -7.603 NS 90 1 4 3,04 3 3 .498 0.310 Radar Summary Chart i 82 1 4 2.65 3 3 -3.379 NS Radar Weather Report RADAT 39 I 4 1.82 2 ! -8.059 NS Satellite Weather Pictures 91 1 4 2.70 3 2 -3.233 NS Severe WW Bulletin 78 I 4 2.81 3 3 -2.105 NS 91 1 4 2.81 33 3 -2.223 NS Surface Analysis =_ TIBS 76 ! 4 2.45 2.5 3 -4.479 NS Television 93 ! 4 2.38 2 2 -7.221 NS TAF 90 I 4 3.38 3.5 4 5.031 0.000 TWEB 79 1 4 2.18 2 2 -8.668 NS 86 1 4 2.71 3 3 -3.329 NS Weather Depiction Chart 94 ! 4 2.84 3 3 -2.019 NS Winds and Temps Aloft * Because iheSe tests are one-tailed, negative t-statistics indicate non-significant results, and therefore p-values are not reported.

IO0 8O e_ > 6O C ::) G) O) C u L_ G) I1.

L& ._ _,_- Figure 6a. Unavailability of Preflight Weather Services and Products.

Figure 6b. Unavailability of Preflight Weather Services and Products (continued).

Use of Internet Web Sites for Preflight Planning A variety of aviation weather resources are available on the Internet. Pilots were asked to indicate whether they use any of I0 specific web sites listed to aid in preflight planning (Figure 7). The two web sites most commonly used are the AOPA web site (http://www.aopa.org) (72%) and the GTE DUATS web site (http:llwww.skycentral.gte.com) (71%). About 25%-30% of the respondents have used the Intellicast, National Weather Service, and NEXRAD web sites. Less than 15% of the respondents used any of the other five other web siteslistedl Many of the GA pilots respond{ng to the survey also listed additional web [ sites that they used to assess weather during preflight planning. More than one pilot mentioned the AOPA site (apparently not recognizing it in the original list) and the WeatherTap site (h_ttp://weathertap.com).

z Other sites that were mentioned include: WeatherConcepts (http://www,weatherconcepts.com, now FlighiBrief'), the Airline Dispatcher Federations' weather briefing web site (http://www.dispatcher.

org/brief/adfbrief.html), and a page containing international weather information (http:// www.

avnet.co.uk/weather, now http://www, avbrief.com#weather).

¢) (/) ¢- O Q.

tw O ¢- tl) I1.

Figure 7. Preflight Use of Web Sites.

In-Flight Weather Information Once the flight is enroute, different weather information sources are available. Respondents were asked to rate the importance of specific types of information for in-flight decision-making (Figure 8).

These items required respondents to rate perceived importance using the same five response categories as used for the preflight items ("information not available," "not at all important," "somewhat important," "very important," and "extremely important"). In-flight weather information considered extremely important by the highest number of pilots includes Airborne Icing (known and forecast), Lightning, SIGMETS, PIREPS overheard from other aircraft on their frequency, Visibility, and Low Level Wind Shear. ATIS, Ceilings, Turbulence (Known and Forecast), and Surface Winds were most frequently rated "very important." ATC discussion of ride reports was most frequently ,rated as a somewhat important enroute weather information source. Table 4 provides descriptive statistics for ratings associated with this question, and one-tailed (>3) t-test statistic and significance level for each information source/service/product. The survey included a free response section that allowed respondents to comment on other sources of weather information they use while in-flight. One respondent mentioned using ASOS/AWOS broadcasts when flying over airports that are so equipped. The survey included ASOS/AWOS as a service/product. Ratings are therefore provided in the following section. Two respondents listed Flight Watch as being an essential tool for their in-flight weather information. Two respondents pointed out that they use Strikefinder to monitor lightning activity while flying. ACARS, Convective Activity Nowcast - Enroute, and Radio-accessible Dispatcher were most often found unavailable to these respondents (Figure 9).

9O 8O i 7O 6O Figure 8. Importance of In-Flight Weather Information.

Table 4. In-Fli_ hl Weather ACARS** I z ATC 60 I 4 2.57 2 -4.375 NS = ATIS 93 2 4 3.17 3 2.268 0.013 88 I 4 3.20 3 2.385 0.010 Ceilings 56 i 4 Convective - Terminal (ITWS/TDWR) 3.25 4 2.236 0.015 42 I 4 3.10 3 .726 0.236 Convective - Enroute (Nowcast) , 23 I 4 2.48 2 -2,517 NS Dispatch 79 1 4 3.29 4 2.943 0.002 Airborne Icing 74 I 4 3.11 3 3 1.033 0.153 Ground Icing: 77 2 4 3.42 4 Lightning 4.755 o.ooo SIGMETS 87 2 4 3.38 3 4 5.157 0.000 Turbulence 85 1 4 3.11 3 3 1.291 0,100 PIREPS 92 1 4 3.20 3 4 2,382 0.010 95 2 4 3.25 3 4 Visibility, 3.033 0.002 Winds Aloft 93 1 4 2.72 3 2 -3.495 NS Surface Winds 92 2 4 3.0 3 3 .000 0,500 Low Level Wind Shear 87 1 4 3.30 4 4 3.388 0,001 * These tests are one-tailed. Negative t-statistics indicale non-significant results, therefore, p-values are not reported.

** Only lwo GA respondents answered this item, therefore l-statistics were not computed, ............... . ......... .: _.. ....

Figure 9. Unavailability of In-Flight Weather Information In-Flight Weather Services and Products Respondents were asked to rate the importance of existing weather services and products with respect to their actual or potential importance for in-flight weather-related decision-making (Figure 10). As with the aforementioned survey items, these questions required a response from the set of five importance levels ("information not available," "not at all important," "somewhat important," "very important," and "extremely important"). The services and products most often considered extremely important were Convective SIGMETS, Enroute Flight Advisory Forecast (EFAS), and Pilot Weather Reports (PIREPS).

AIRMETS (WA), Automated Surface Observation System (ASOS), Automated Weather Observing System (AWOS), and Hazardous In-flight Weather Advisory Service (HIWAS) were most often considered to be very important sources of in-flight weather information. None of the services/products received a significant number of responses indicating that it was not at all important. Table 5 provides descriptive statistics for ratings associated with this question, and one-tailed (>3) t-test statistic and significance level for each information source/service/product. As we expected, these service/products were often rated as unavailable in-flight: Composite Moisture Stability Charts, Constant Pressure Analysis Charts, Convective Outlook Charts, Direct User Access Terminal system (DUATS), Internet, METARS, Meteorological ImpactStatement (MIS), Newspaper, Pilot's AutomaticTelephone AnsweringSystem (PATWAS), Radar Summary Chart,Radar Weather Report(SD),Radiosonde Additional Data(RADAT), SatelliteWeather Pictures, Severe Weather WatchBulletin (WW), Surface AnalysisCharts, Telephone

InformationBriefing Service(TIBS), Television,TerminalAerodromeForecast (TAF), Transcribed

Weather Broadcast (TWEB),Weather Depiction Chart,WindsandTemperatures Aloft Charts, andWinds and Temperatures Aloft Forecast(FD) (Figure l !). Interestingly,respondents also indicatedthat "(listening to)Pilotsacross theparty-line," wasalsounavailable asaninformation source.

A free responsesectionallowed respondents to indicate other in-flight weatherinformation

services/products theyuse. Twocomments-were commonly made in this section.Several pilotsexpanded on theiruseof FlightWatchasa crucialaspect of weather-related decision-making in-flight. Secondly, respondents usedthis freeresponse areato notefrustrationaboutnot beingableto receive mostof these services andproducts whilein-flight. Onerespondent stated, "Wehaveverylittle weather available from thepilot'sseat."Another respondent remarked, "Oncetheflight is enroute, mostof thismaterial is simply notavailable.It's 1999.Whenwill thischange?"Several otherrespondents asked abouthowtheymight acquire thisinformation in-flight.

i i_l "Very" or "Extremely" Important • lnfo Not available Figure i 0a. Importance of In-Flight Weather Services and Products.

c [] "Very" or "Extremely" Importanl Info Not availabl_ Figure lOb. Importance of In-Flight Weather Services and Products (continued).

Table 5. In-Fli :es and Products Ratin AIRMETS -1.919 ASOS 84 ! 4 3.02 3 3 .266 0.396 AWOS 82 I 4 3.05 3 3 .563 0.288 FA 68 I 4 2.74 3 2 -2.496 NS 3 -. 155 NS Center Weather Advisory 60 1 4 2]98 3 Composite Moisture Stability 32 I 3 1.63 2 1 -11.787 NS Constant Pressure Analysis 34 1 4 1.74 2 1 -8.538 NS Convective Outlook Charts 36 1 4 2.17 2 I -4.737 NS SIGMETS ] 75 I 4 3.17 3 4 1.605 0.057 DUATS 20 1 4 2.50 3 I -1.876 NS EFAS 72 1 4 3.14 3 4 1.235 0.111 3 1.070 0.144 In-Flight Weather Advisory 78, I .4 ..... 3.10 3 Internet 14 I 4 2.00 I 1 -2.876 NS METARS 51 I 4 2.96 3 3 -.280 NS MIS 18 I 3 1.61 2 2 -9.697 NS I -17.616 NS Newspapers 20 1 2 1.25 1 PATWAS 12 1 3 t.58 1 I -6.189 NS !

PIREPS 84 I 4 3.36 3 4 4.641 0.000 "party-line" 55 i 4 2.71 3 3 -2.171 NS 2 -2.906 NS Radar Summary Chart 29 I 4 2.41 2 Radar Weather Reports 29 1 4 3.38 2 1 -2.997 NS RADAT 12 I 4 1.75 1.5 I -4.486 NS Satellite Weather Pictures 21 1 4 2.43 2 2 -2.434 NS Severe WW Bulletin 35 1 4 2,66 3 3 -2.163 NS 2 -2.921 NS Surface Analysis Charts 23 1 4 2.35 2 TIBS 12 1 4 1.92 1.5 1 -3,463 NS Television 18 1 4 2.06 2 1 -3.610 NS TAF 49 1 4 3.12 3 4 .948 0.174 TWEB 48 1 4 2.85 3 3 -1.155 NS 3 -3.392 NS Weather Depiction Charts 25 1 4 2.28 3 -2.850 NS Winds and Temps Aloft Charts 27 1 4 2.44 3 2 -2.731 NS Winds Temps Aloft Forecast 40 I 4 2.58_,, 3 * Because these tesis are one-tailed, negative t-statisilcs indicalenon-significantresults, and therefore p-values are not reported.

N oo_t

|

!

'::_:a]_ :_ii.i£; .... _,g_l_:._j L_ _;¢ " _q ._ :K:;:_ ::::_:: .. _Td_'_il._ Figure 11 a. Unavailability of In-Flight Weather Services and Products.

lO0 e_ _m ,m m c =) 4) u) c k,, I1.

Figure I lb. Unavailability of In-Flight Weather Services and Products (continued).

in-Flight Decision Making Strategies GA pilots were asked whether they thought there is a difference between the information they use during a flight to "keep up with the weather situation" and the information they use to "make decisions enroute." A bit less than 58% Of responden-ts answered, "No, there is not a difference," with about 36% responding, "Yes, there is a difference." Roughly 7% of the GA respondents did not answer the question.

Respondents that noted a distinction were asked to further describe this difference in a free response section.

A review of the free response content for this question revealed that five of the thirty-three respondents who provided comment, misinterpreted the question. It seems that they interpreted it as asking whether there was a difference between ground weather information and enroute weather information rather than whether there iS a d;Stinction having to do with how ava_iabie information might be used differently.

Other responses indicated that "keeping up with weather," means comparing indications of current weather with preflight forecasts. Flight Watch and automated systems (i.e., ASOS/AWOS, ATIS, HIWAS) were largely mentioned as the source for this purpose. "Making decisions" about the weather was typically associated with discrepancy notification (realizing that current conditions did not match expectations from forecasts), active information seeking (typically from FSS, Flight Watch, ATC), interpretation of the level of threat presented, and developing a response plan (often in conjunction with ground based operators).

When specifically asked how they personally make in-flight decisions related to weather, respondents most frequently reported that they contact Flight Watch (Figure 12). It should be noted that two subjects, however, specifically noted "not having much luck with" and "difficulty obtaining this service."

Respondents also reported contacting Air Traffic Controllers, including the Air Route Traffic Control Centers (ARTCC), Approach Control, and Flight Following; and Flight Service Stations. Listening on frequencies to what other pilots ahead of them are being told by ATC and Flight Watch or FSS, as well as specifically acquiring pilot reports was also a frequently reported source of information. All the enroute automated services (AWOS, ASOS, HIWAS, TWEB) added together were only reported by approximately 20 percent of the sample. Typically, if a respondent indicated using one of these automated services, s/he reported using the others as well. ATIS and visceral or visual cues were even less frequently cited. One respondent mentioned using Stormscope in addition to the more commonly available ground-based services.

t- O e_ O ..... i e- R a.

Figure 12. In-Flight Weather Decision-Making Resources.

The survey requested that respondents list additional weather information that they would consider useful for in-flight decision-making (Figure 13). The weather information sources indicated by at least 10 respondents were: icing reports, ceilings, current radar, convective activity, winds aloft, precipitation, visibility, turbulence, and thunderstorm activity. Other sources of weather information mentioned (by 9 or fewer respondents) included: Nexrad radar graphics, PIREPS, cloud tops, activity levels, updated forecasts, ASOS, AWOS, ATIS, all information in a full briefing, visual depiction during IMC, Intellicast, surface winds, SIGMETS, frontal activity, satellite pictures, ground condition at destination, windshear, alternate routes, METARs, TAFs, GPS location, DUATS, and urgent alerts.

t- O e_ IX O e,- t_ IX

oo

Figure t 3. Most Desired Weather Information for In-Flight Decision-Making.

When asked about the usefulness of near real-time weather information displayed in the Cockpit, responses were favorable. Provided with a five-point scale ("no utility," "limited," "useful," "very useful," "essential"), 86% of the pilots indicated that this kind of information would be essential or very useful (Figure 14). The respondents were also asked about the potential usefulness of near real-time weather information on a moving map display. Again, the responses were overwhelmingly positive. In fact, 89% of the pilots (86 individuals) opined that moving weather maps with near real'time data would be at the least, very useful (Figui-e 15). Seventy-one percent of the respondents (69 individuals) reported that a handheld weather information system with near real-time data would be either an essential or very useful tool to them while enroute (Figure 16).

& 4o

"5 30 G) U Essential No No Utility Limited Useful Response Figure 14. Usefulness of Near Real-Time Weather.

6O O C

& 4o

=____ ¢Y _ 30 (,1 L_ o. 10 No No Utility Limited Useful Very Useful Essential Response Figure 15. Usefulness of Near Real-Time Weather with Moving Map.

6O 5O ¢) t--

&

4O re _6 3O O'J f_ 2O to lb.

I1.

No No Utility Limited Useful Very Useful Essential Response Figure 16 Usefulness of Handheld Portable Device with Near Real-Time Weather.

- _ .....

The surveyqlso askedwheiher the decision making processc0uld be characterized as collaborative or whether pilots tend to rnake most weather-related decisions alone' Unfortunately, only 11 of the respondents answered this part of the survey. Of the 11 that did answer, 5 pilots expressed that they preferred to make decisions with the help of others w_aen possl_ble, but 6_tated t_aat they viewed the decision as thelr6wn and did not c0ns6it others There rnay have been some confusion in this question as to whether "collaboration" referred to involving others as sources of information v. allowing others to make the ultimate decision! :_ ...... :- =i_ ............ , = Flight Experience & Weather Information Valuation Linear regressions of total flight hours On ratingsof perceived usefulness for preflight weather information, preflight weather services and products, and enroute weather information did not demonsirate any significant relationships (all R _ < 0.08). However, classifying respondents ]mo two categories based on median flight hours for each variable does show significant effects of experience on ratings for some weather information, services and products. The median number of flight hours for all respondents is 586.9. Due to missing data, the categories of "higher hours" and "lower hours" for each weather information source, service or product are defined by the median flight hour value of those respondents who rated the importance of that item (Table 6). GA pilots with the median number of flight hours or more tend to value IFR ceilings (F(1,80)=5.856, p=O.O18), convective activity in the terminal area (ITWS/TDWR) (F(I,83)=5.058, p=0.027), nowcast convective activity for enroute (F(I,67)=4.385, p=O.040), airborne icing (F(I,92)=2.956, p=0.089), and PIREPS (F(1,93)=5.475, p=O.021) more highly for preflight weather awareness and decision making than do GA pilots with fewer flight hours.

Conversely, GA pilots with less than the median number of flight hours tend to value DUATS (F( 1,90)=3.948, p=O.050), the newspaper (F(!,82)=3.124, p=O.081), and TIBS (F(1,74)=3.437, p=0.068).

GA pilots with the median number of flight hours or more tend to value convective enroute nowcast information (F(1,40)=7.481, p=O.O09), and airborne icing information (F(1,77)=3.664, p=0.059) more for enroute use than those with fewer flight hours. Dispatch information over the radio while en route was rated significantly more important for GA respondents with fewer than the median number of hours (F(1,21 )=5.976, p=0.023).

Jrvey Item.

Table 6. Median Hours of Total Flight' Fime per S Med_n Median Enroute WX Info Median Preflight WX lnfo Preflight Service/Product 737.5 ASOS 593.45 ACARS- Dispatch Ceilings (IFR) 500 Area Forecasts 593.45 825.5 Ceilings (VFR) ATC's Ride Reports ATIS 593.45 586.9 586.9 CWA Ceilings (MVFR) Convective - Terminal 593.45 650 500 Moisture Stability Chart Ceilings Convective - Enroute 662.5 Constant Pressure Chart 737.5 Convective - Terminal 586.9 Convective Outlook 625 Convective - Enroute 662.5 Airborne Icing 593.45 DUATS 586.9 Dispatch on Radio 250 Ground Icing 625 625 586.9 Lightning FSS Briefing Airborne Icing SIGMETS 586.9 lnternet 543.45 Ground Icing 543.45 586.9 METARS Temperature (D.A.)

Turbulence 593.45 MIS 6O0 586.9 1050 Visibility Newspaper_ Winds Aloft 586.9 PATWAS 450 Surface Winds 586.9 PIREPS 825 Word of Mouth 568.9 Radar Summary Chart Radar Weather Report RADAT 650 Satellite Pictures 1300 Severe Wx Watch Bulletin 625 Surface Analysis Charts TIBS 625 Television 725 ._= TAF 600 TWEB 625 Weather Depiction Chart Winds & Temp. Aloft * Only two responses were given for GA res 9ondents in this category.

Cost of Aviation Weather Technology The survey questioned participants about the amount they would be willing to pay for aviation weather information technology. No specific definition of an aviation weather information system was provided.

For these questions, participants were required to answer using the categories: "Under $1000," "$1000- $5000," or "Over $5000" (Figure 17). Ninety-one responses were obtained for these questions. The majority of 91 respondents, 49.5% (48 pilots), were willing to pay between $1000 and $5000 for an aviation weather information system. About 39% (38 pilots) were willing to pay under $1000. Next, the respondents were asked how much they would be willing to pay to install the cockpit weather system (Figure 18). A significant majority, 75.3% (73 pilots), answered this question in the under $1000 category.

The third question asked participants how much they would be willing to pay for a weather service (e.g., satellite broadcast direct to the cockpit during flight). Again, the majority, 75.3% (73 pilots), answered under $1000. Finally, respondents' ratings indicated that most would prefer to pay for this service by individual access (41%), than by-flight (27%), and fewest preferred to pay for the service monthly(18%).

I 7O

80!

5O Amt Willing to Pay Amt Willing to Pay Amt Willing to Pay for System to Install System for Weather Service / Im Under $1000111 $1000-$5000 [] Over $500d / Figure 17. Total Acceptable Expenditures for Cockpit Weather Information System.

4O 3O Q.

n,, 2O L_ a.

By Access By Flight By Month No Response Figure 18. Acceptable Methods of Weather Information Service Payment.

Discussion To meaningfully discuss the results of this survey, we must first recognize and discuss the survey's limitations as a measurement instrument. In spite of these limitations, we find interesting results regarding how GA pilots use current weather information and what they desire in future aviation weather information systems. We discuss how user preferences for aviation weather information system features and expenses compare to other studies of these, and how such user opinions may affect an AWIN system's introduction to the market. These results are interpreted for insights valuable to designers of aviation weather information systems, and research to support this design process.

Critique of the Survey & Limitations of Generalizability This survey was designed with a distinction between weather information and weather information products and services. The intent of this distincti0n wa_to encourage respondents-to think about the value of information about certain types of weather apart from ho_v they are currently packaged, as products or services. In retrospect, we realize that this classification was not well defined and items were imperfectly assigned to these categories. Further, to answer these questions, respondents must have imagined what this information would look like, and, absent any further clarification in the survey, we assume they relied on their current experience to provide this reference, in which case weather information was likely conceived of as a product or service. Therefore, the distinction between information and products/services is not a clear one. For the purposes of the following discussion, these categories are collapsed and referred to as weather information sources.

This survey was designed to be relevant to National Airspace System (NAS) users of various categories. The fact that the vast majority of respondents classified themselves as GA pilots was biased, to some degree, by the venues in which the survey was advertised. Most of the web sites selected are principally designed for pilots of small GA aircraft. These sites were selected because we wanted to specifically encourage GA pilots, those most severely affected by weather hazards, to complete the survey.

While responses to other pilot categories were too sparse for analysis, the sample size of GA pilots is adequate. The results presented in this report, and pursuant discussion, really generalize only to GA pilots, and more specifically to those who have Internet access and visit the sites on which this survey was hosted, and who have the willingness and skills to complete an online survey. The majority of respondents were not high-time pilots. Different regions of the continental U.S. were fairly equally represented by the GA respondents. Very few GA respondents indicated they flew internationally. z The survey asked participants to provide an "importance rating" to different weather information sources. The options for this rating were: "information not available," "not at all important," "somewhat important," "very important," and "extremely important." The survey was implemented such that all these options were mutually exclusive, when "information not available" should have been a separate question from the other options that pertain to an "importance rating." This is significant for two reasons. We failed to capture information from each respondent about the potential usefulness of weather information sources that currently are unavailable. In addition, when asking these questions for the in-flight use of weather information, there is some confusion as to whether "unavailable" means that a source is not available as it would be on the ground (e.g., timely, dynamic, display of Internet radar looping), or is available in a different form (e.g., a dated, static print of a frame from the radar looping animation). While surveys allow the opportunity to canvas a wide range of users, the number of responses is, unfortunately, directly related to the ease of the survey, and therefore the lack of depth in responses. Surveys such as this one provide useful initial data that then guide more targeted exploration in, for example, structured interviews and other forms of cognitive task analyses.

The results of this type of survey represent users' perceptions of information utility, usability and reports of usage. While such valuations are important, objective evaluations of information requirements and usage are also required for system design. In addition, results of user surveys are usually anchored in the present. Users tend to rely on their current and recent experience to answer questions of utility and usage. Therefore, a rating of a weather information element's utility is a rating of the information provided, in the format that it is currently provided and assuming the reliability of the information as it exists today. Usage ratings are further derived, because they rely on not only the perceived utility of the information, but assume the current aviation operational context. Finally, the ratings provided do not necessarily reflect an appreciation of the costs of using this information in the actual operational context.

That is, "importance" doesn't convey relative importance, or the opportunity cost, of directing attention to that information at the expense of other in-flight demands. The results found in this survey should be compared to those derived from simulation and flight experiments to ascertain their validity in the context of realistic scenarios or actual operations.

When Pilots Say Weather Information is Most Valuable Weather information was reported as most essential to these GA pilots in preflight briefings. This is appropriate. Determining not to conduct a flight in unsafe conditions is the most effective way to avoid having a weather-related accident. Once in flight, only the cruise phase received ratings over "very good" that were commensurate with those received for the preflight phase. These results indicate that pilots value weather information most during periods of relatively low workload, and when their focus is on planning. Supporting this strategic use of weather information during these phases of flight is important.

The first item on the NTSB's top ten list of causes for GA accidents is "inadequate preflight preparation and/or planning"[2], and has been directly associated with accidents during take-off [3]. Similarly, AOPA Air Safety Foundation [4] analyses have demonstrated that while the seed of most GA accidents occurs in cruise, the accident event occurs most often during descent. The fact that these strategic phases of a mission are cited as the times when weather information is most valuable may be an artifact of the weather information sources that are currently available to GA pilots. That is, because current weather information sources require focused attention and some concerted effort to interpret and to, further, determine the significance of this information for flight decisions, these sources may only be usable and therefore perceived as useful during lower workload periods. We cannot therefore eliminate the notion that weather information sources are equally or more important during other phases of flight. In fact, the need for tactically relevant weather information is revealed in fewer, but still numerous responses indicating its importance for approach and landing phases, higher workload phases with higher tempo and consequences. We must surmise, therefore, that if weather information sources are to be available during higher workload/tempo/consequence phases, the information must be designed to be more easily accessible, interpretable, relevant to current mission goals, and action-oriented.

Preflight Weather Information: Valuation & Availability Many of the weather information sources included in the survey were identified as at least "very important" for supporting preflight briefings. The phenomenological information that is most important to preflight includes terminal and enroute convective activity information, airborne icing, ceilings (VFR, MVFR, 1FR), and visibility. This result reflects GA pilots' appropriate concern for three of the most fatal weather situations: VFR into IMC, convective weather, and in-flight icing. Lightning and ground icing were also generally rated more than "very important," but this difference did not reach statistical significance. Preflight weather information sources that were also considered significantly more than "very important" included summarizations of weather phenomenological information for the route or a point along the route. These included, FSS briefings, SIGMETs, TAFs, DUATS briefings, and METARs.

The significance of FSS and DUATS briefings is confounded by the fact that pilots must receive one of these FAA approved weather briefings before flying. When assessing weather information, several subjects also volunteered that pilot reports and precipitation trends are also very useful types of preflight information. This survey did not probe, in detail, how respondents would use these weather sources during preflight. Further investigation might find a distincti0fi between the :_valuation of weather information sources for determining whether the flight is go/no-go; for determining whether to file IFR or VFR; and, if a "go," for planning the route, destination and alternates. This Survey did not distiflguish between IFR- rated and VFR-rated pilots. To the degree that the type of flight taken (e.g., cross-country v. short trips; etc.) affects valuation of weather sources, this bias cannot be definitively estimated in these data. One can, however, see a differentiation between the sources of preflight weather information that GA pilots with more flight hours value highly (IFR Ceilings, Convective terminal (ITWS/TDWR), Convective Enroute (Nowcast), Airborne Icing, and PIREPS), and those that respondents With fewer hours value highly (DUATS, Newspaper, TIBS). This distinction is consistent with the expectation that GA respondents with higher flight time are more likely to take IFR flights.

It is:important to emphasize that several of the consistently highly valued types of preflight weather information were also noted as unavailable by a fair number of respondents. In particular, about 26% of the respondents stated that they do not have access to nowcast information, yet this weather information source was rated as at least very important by 63% of these G,_pHots_ Convective activity information in general was highly valued by 80% of the respondents, and 12% of these stated that they do not have access to it. Assuming that those who responded to this online survey have personal Internet access also indicates that a significant percentage of these respondents may noi be aware of the resources available to them on the Internet. Convective activity information, including Nowcast convective activity, is available on several Internet sites.

Roughly 60% of the GA respondents indicated that the Internet is a very or extremely important preflight weather information source in general, and DUATS is one of the most frequently cited sources among preflight services/products that are highly valued. The perceived utility of the AOPA and DUATS sites is reflected by the high percentage (72%, 71% respectively) of respondents who indicated that they use these sites. All other sites were used by less than 35% of the survey respondents. Recall that the individuals who participated in this sfirvey have Internet access that would allow them to complete this survey online. It is unlikely that pilots using the Internet simply for weather briefing purposes at a FBO would use it to complete such a survey. It is reasonable to assume then that the respondents are those who have personal Internet access. Therefore, the absolute percentage results are likely to be inflated when compared to the larger population of GA pilots. In asking whether participants use a site, we cannot distinguish between the usability of the weather information on the site from the functional utility that other features of the site might provide. In particular, one reason for this preference might be that the

DUATSsite(andtheAOPAsitethrougha link to theDUATSsite)qualifiesasaFAA-approved official

weather briefing. Anotherreason might be thatthesesitesalsohavetoolsto assist pilots with flight planning andfiling. Whiletheamalgamation of suchsupport functions limitsone'sabilityto ascertain the valueof the sitefor weather informationpurposes, thefact thatrespondents find thesesitesmostworth theirtime is instructive.Designers of airborne weather information systems mustensure thattheuseof weather information andassociated tasks aresupported andintegrated.

In-Flight Weather Information: Valuation, Availability & Decision Making The most highly rated weather information sources for in-flight use are those associated with hazards, specifically Lightning, SIGMETs, Low Level Windshear, and Airborne Icing. Many respondents rated PIREPs as a very important source of weather information in-flight. While PIREPs are most often used to communicate the observation/experience of a hazard they may also provide valuable action-oriented information, i.e., for successful hazard avoidance. Phenomenological information perceived as significantly more than very important by the respondent pool included Ceilings, Visibility, and Convective Activity. While not hazards per se, ceiling and visibility are likely important because they indicate the potential for the most hazardous general situation a GA pilot can encounter; that is, continuing under VFR into IMC enroute, or to an alternate or destination that is below minimums. Convective activity can be considered a general hazard because many specific hazards emerge in the presence of convective activity including thunderstorm activity, heavy precipitation and icing, and severe winds and turbulence.

Because accidents in these circumstances are fatal to so many GA pilots, we suspect that pilots are particularly concerned with acquiring a big picture for these conditions. The only summarized weather information source that reached this level of importance for in-flight use was ATIS. In addition to weather information for the terminal environment, ATIS also contains runways-in-use and other pertinent information. Pilots are required to have the most current ATIS information before landing. As such, it is not surprising that this was a highly rated weather information source. Because this survey was originally designed to be relevant for a broader class of pilots, it included items more typically used by commercial or business aviation operations such as ACARS and Dispatch. Not surprisingly, these two weather information sources were noted as "unavailable" in the GA responses that are analyzed here. GA respondents with less flight time rated "(listening to) Dispatcher on Radio" higher than those with more experience. This is an odd result, as GA pilots would not be expected to be listening to a Dispatcher.

More experienced respondents appropriately appreciated the value of convective - enroute and icing information; and more so than their less experienced counterparts.

This survey did not ask respondents to evaluate the usefulness of individual weather information sources by flight phase. We can see in these results that, in the higher tempo, higher consequences environment of flight in general, pilots value higher order information that has already been deemed worthy of instigating action. In fact, those weather information sources that were rated, on average, less than "somewhat important" included predominantly those sources that provide static, large scale depictions of a weather phenomenon (Convective Outlook Charts, Weather Depiction Charts, Surface Analysis Charts, Radar Summary Charts, Satellite Weather Pictures, Winds and Temperatures Aloft Charts). In general, respondents considered these not-very-useful sources to be available in-flight. While they are not currently available on an airborne weather information system, perhaps respondents considered them available because hard copies can be taken to the cockpit and used in flight. It is important to realize that, in the only implementation in which these sources currently could be used in flight, they would contain stale information, and therefore not be particularly useful. This supposition is supported by responses to free-form questions about what additional in-flight information would be useful. While respondents found Radar Summary Charts, and Winds & Temperature Aloft Charts to be less than very useful, free-form responses indicated that the most desired in-flight weather information is exactly that which is contained in these charts. More than 15% of the GA respondents specifically requested Current Radar (25%) and Winds Aloft (23%) information. This result is instructive for airborne weather information system designers on two points. Weather depictions formatted for ground use are not likely to be appropriate for airborne use where they must be interpreted at a glance. Further, the temporal relevance and spatial relevance of weather information is of utmost import for airborne use. Temporally-irrelevant, static images are clearly not perceived as useful, appropriate update rates must be carefully designed and indicated on displays for proper use of the weather information. Scaling and image size must also be sensitively designed for effective in-flight use.

It is somewhat surprising that the respondent pool's average ratings of EFAS (Flight Watch) and "party- line" information, was not significantly more than an average rating of "very important." We see that several other radio-acquired weather information sources are similarly scored. Considering that the specific intent of Flight Watch is to provide pilots with weather information, the fact that only about 50% of the respondents indicated using this resource may also indicate problems with either the usability of the information provided and/or the ability to access this information. This result may also indicate a limitation of current operations rather than the information content of these sources. When weather information is most useful and required, many pilots want it and EFAS/Flight Watch personnel have more information to convey. Responses indicating that party-line information is unavailable may be due to this frequency congestion problem. It is an unfortunate irony that in current operations, when weather is of most concern, radio frequency congestion makes this information most difficult to obtain. Airborne weather information systems have been shown to reduce the communications between pilots and ground support personnel in simulation investigations [5] and GA pilots have expressed less need to use these ground services when using an airborne weather infoi'mafion system during a flight experiment [I].

Today's operations, however, do rely heavily on this radio-delivered information provided by ground- based operators and automated services to provide weather information to p_lots.

Only three GA respondents made mention of onboard weather equipment used to augment visual and aural weather information sources (2 pilots mentioned Strikefinder, and 1 pilot mentioned Stormscope).

The most commonly reported methods for in-flight weather decision-making, EFAS/Fiight Watch and ATC Services, reflect this reliance on ground operators. Despite this reliance in current operations, the indication that pilots perceive weather information available in the cockpit as more valuable than ground- based services that provide weather information is supported by the fact that 86% of the responses agreed that having near real-time weather in the cockpit would be "essential" or "very useful." Prior research suggests that pilots trust weather information more when it is directly perceived/controllable, even if computer-mediated, than when it i_ perceived and communicated by a distal person who is unknown to them [6]. Proper design and use of an airborne weather information system will reduce the dependency of GA pilots on ground support services, but should also indicate when to consult these professionals and facilitate discussions about weather decision-making, in doing so making these communications more effective and efficient. One requirement for improving such communications, as well as improving in- flight decision-making, is to provide a common interpretation of geo-reference and aircraft position and common formatting conventions. Research has indicated that establishing this "common ground" [7] for communication is critical for ensuring that communication is meaningful and efficient for those involved.

Respondents recognized the advantages of providing a moving map display with weather information by rating this integrated airborne system more useful than one with only weather information. General aviation pilots using prototype GWISs [1] have had improved confidence in their situation awareness, and have enthusiastically received these systems when integrated with traffic information [8].

Implementation: The Value of Weather Information in the Cockpit While the majority of respondents liked the idea of a portable handheld device with weather information, more respondents rated the generic concept of weather in the cockpit more favorably. This suggests that alternative implementations may be preferred. Over 88% of the GA respondents were willing to pay under $5000 for an in-flight weather information system, and more than half of these people were willing to pay between $1000 and $5000. Kauffmann and Pothanun [9] found that the average acceptable cost to GA pilots for an in-flight weather information system on a moving map was $5892.

The response categories used in this survey were less sensitive for characterizing the amount GA pilots were comfortable spending to install this system and acquire the weather information service than the continuous variable estimate acquired in the Kauffmann and Pothanun survey. Over 75% of the respondents were unwilling to pay more than $1000 for an aviation weather information subscription service. Kauffmann & Pothanun's findings support this finding. The average acceptable recurring cost for an in-flight weather information system to the GA pilots in their survey was $433. Kauffmann and Pothanun distributed their survey instrument to key decision-makers in organizations that they determined would be familiar with different piloting categories. That survey, then, included the perspectives of avionics companies, airframe manufacturers and trade groups, rather than individual end users. In a subsequent survey, Kauffmann and his colleagues [10] asked participants to describe the features they would expect to find in an aviation weather system and, for all the features they rated a four or five, on a five-point scale, how much non-recurring and recurring costs they would expect to incur. Roughly 70% of both private and instrument-rated pilots expected non-recurring costs to be less than $2000. Only 50% of the recreational pilots expected this cost to be below $2000. Eighty-one percent of the respondents expected to pay annual recurring costs of less than $500.

As a population, wide variance exists in the number of flights general aviation pilots conduct per year.

For each pilot, circumstances can dictate that flying is curtailed during some times and frequently enjoyed during others. Not surprisingly, then, GA pilots who expressed an interest in an in-flight weather information system would prefer to pay for it by access. While this is an economically rational choice from the pilot perspective, subscription-based fees might be required to make this service feasible from the provider's perspective. An additional consideration argues for a subscription or by-flight service: it removes economics from consideration once in flight in determining whether it is "worth" getting weather information. Not only is this an additional factor likely to be considered, at least implicitly, by pilots while flying, and therefore adding to workload; but it would likely result in more conservative use of the weather information service when there is some indication it should be used. In addition, if the system is only accessed when weather is difficult, pilots will have less familiarity with the system interface. Lack of familiarity, particularly when weather is significant, could increase system fixation, and decrease efficient and appropriate use of the information. If per-access service is allowed, the cost must be carefully designed to ensure that this factor does not significantly inhibit use. Additionally, evaluations should be conducted to ensure that pilots can effectively use the system and appropriately assess the quality of data provided by the system when their interaction with it is infrequent and workload conditions are high.

Conclusions This preliminary survey provided many useful insights into the perceived value and usage of weather information sources by pilots of small GA aircraft. Given the current state of the information available and aviation operations, pilots seem to use weather information the most when they have time and attention to acquire and interpret it. Generally, information most valued in preflight can be distinguished from that most valued in-flight by the degree to which it is comprehensive v. indicative of specific hazards. This observation implies that respondents implicitly considered the opportunity cost of accessing and interpreting weather information in flight, and recognized the cognitive economy of having interpreted weather hazard information over that of generally descriptive information. More experienced GA pilots tended to be more interested in lower level phenomena weather information sources, perhaps indicating different flight missions (IFR) and/or more experience in interpreting these more detailed weather information sources. The GA respondents to this survey seemed to be unaware of resources available to them on the Internet. Observations from the results of this survey form the basis of a model for establishing the value of weather information sources to GA pilots, guidance for the design of aviation weather information systems, and direction for future research towards the improved development of such systems.

A Model for Establishing the Value of Weather Information Sources Based on inspection of the results, we hypothesize five factors that affect the valuation of in-flight weather information for the purposes of weather hazard avoidance enroute: 1) spatial/temporal relevance, 2) degree of consequence, 3) reliability of the information (3D spatial /temporal, and intensity level accuracy), 4) usability of information presentation, and 5) availability/accessibility of information.

Spatial temporal relevance refers to the degree that the actual weather phenomenon or predicted weather phenomenon is proximal or predicted to be proximal to the a_rcraft_s position or planned route. This indicates the probability that the aircraft will encounter it. This degree of proximity can be expressed in terms of 3-dimensional distance- or time-to-encounter, and as such, also depends on the aircraft's speed.

Degree of consequence is determined by phenomenological intensity as well as the aircraft's ability to withstand this intensity. Phenomenological intensity refers to the characteristics of the weather that are used to determine the degree to which it is hazardous, should one encounter it. That is, for winds, it is the speed, direction, and gust factor; for precipitation, the size, solidity, and intensity; etc. Different aircraft will have different thresholds for defining a hazard based on these phenomenological intensity levels.

Information sources that are more reliable (i.e., provide accurate and specific current and/or forecasted weather locations, intensities, and dynamics of movement and intensity) are more valuable than those that are less reliable. Usability of the information presented is inversely related to the degree to which a pilot must apply focused attention to interpret weather information for defining actions; that is, the amount of effort and attention required to ascertain whether a hazard exists (assess the relevance, intensity, and reliability of the information) and to determine required responses. Accessible and available information sources, those that provide immediate information on demand, are more valued than those that are susceptible to outages or interference, or are cumbersome to acquire. The most valuable information, therefore, is reliable, usable, available/accessible weather information that is spatially/temporally relevant and has a significant level of consequence. These parameters are expected to vary for different types of weather phenomena, and over different flight phases.

Design Guidance for Aviation Weather Information Systems Results from this survey suggest guidance for improving aviation decision-making with weather information. While this guidance is based on responses from GA pilots, it is useful for the design of aviation weather information systems for other pilots as well.

Preferences for alert and mission-relevant weather information indicates that aviation weather information sources must be easily accessible, interpretable, relevant, and action-oriented. Accessible information is designed by attending to interface control structures (menus, direct access keys), window layering, and system modes. Interpretable information is that which is not cluttered, provides appropriate contextual referents (aircraft position, track, geo-references) and relevant weather information (location, direction of movement, and intensity). The relevance of information can be established by attending to false alarm and miss rates for alerts and presenting information that is oriented to the pilots' task at hand.

Action-oriented information indicates the specific problems that weather poses to a flight, and assists the pilot to determine the appropriate response (e.g., offer suggestions for alternate flight paths, indicate when a pilot should seek additional sources of weather information, etc.).

Aviation weather information should be infused into support systems developed for aiding pilot tasks, not simply developed to provide aviation weather information. This guidance derives from the observation that web sites which support the entire task of preflight planning are preferred to only weather information web sites. However, this guidance is particularly important for single pilot operations to minimize pilots' responsibility for integrating information across cockpit / flight deck systems.

Weather depictions as formatted for ground use were judged to be not very useful for in flight use, but the information contained therein was that most requested for in-flight use. This observation emphasizes the need to design weather information presentation for in-flight use where information must be acquired at-a-glance, in challenging ambient conditions, on smaller displays, and where current information and spatially relevant information provides the most value.

The GA respondents in this study as well as others emphasize their desire for "weather information in the cockpit" and don't universally regard EFAS/FW as the best or most available source of weather information. Onboard weather information systems provide benefits over FW services by obviating the frequency congestion problem, affording a graphic depiction of spatial information, and providing the benefits of redundancy and in situ sensing. However, FW professionals still have the big picture view of weather systems, the experiences of other pilots with it, and the expertise to interpret these data. As such, an aviation weather information system should indicate when to contact these professionals to supplement its information, and should be designed (by providing contextual features and common formatting conventions) to facilitate establishing common ground in communications about weather and developing strategies for flying around it.

Highly valued in-flight weather information is generally more integrated than that which is highly valued for preflight use. The level of information integration initially available to pilots, particularly in single pilot operations, should be commensurate with the costs of attending to weather information and the urgency of that information. That is, during higher workload flight phases or when encountering a hazardous weather event is imminent, integrated hazard areas and alerts are more relevant than status information about individual weather phenomena. During the highest levels of activity and urgency, action-oriented or even directive information is arguably most appropriate. Users should be allowed to alter the level at which information is presented. The ecological interface approach espoused by

Rasmussen and Vicente[11] mayprovidea usefulframework for presenting weather informationat

various levels of abstraction to support flexibleusein thedynamic decision-making context of flight.

Future Research Directions Future research efforts generally include defining information requirements for weather displays appropriate to flight mission and pilot goals; developing specific decision aids that consider the role of weather in these decisions, and improving NAS users' communication about weather information. Two of these efforts are briefly described below.

While this survey, and other surveys, provide important information about user preferences and impressions of use, more objective and task-specific methods are also necessary to identify information requirements for system design. Other AWIN studies will address the weather information requirements, apart from the sources that provide them, for supporting specific aviation decisions in preflight and in flight. Such preflight decisions include: go/no-go determination, filing VFR/IFR, planning a route that avoids hazardous weather, selecting usable destination and alternates, contingency planning, and planning when to obtain weather information in-flight. In-flight decisions include: determining if conditions exist to enact a contingency plan or need to replan (considering divert, return to origin, replan route), reconsidering I when and where to obtain weather information, determining whet_ it is possible to complete the'plan-tied approach, determining whether the selected runway is appropriate, deciding when and how to use any onboard weather sensing or mitigating systems, etc. The model of aviation weather information value may prove useful for structuring cognitive task analyses aimed at extracting pilots' weather information requirements for these specific aviation decisions.

z The GA pilots who responded to this survey most frequently expressed interest in obtaining information about convective activity and related weather phenomena in flight. Next Generation Radar (NEXRAD), composite radar imagery, provides information about convective activity. These NEXRAD images will be the first graphical weather information products available, for a fee, as augmentations to the free FAA-sponsored FISDL (Flight Information Services Data Link) weather information service. Early studies with candidate prototype interfaces for enabling this service demonstrate the importance of appropriately updated information, geo- and aircraft reference features, and salient indication of weather information age [6,12]. Additional concerns include harmonizing colors used in displaying radar information across NAS users, including not only the variety of displays available to pilots for aviation weather information systems, but also those for displaying onboard radar information, and those available to ATC, Flight Services and Dispatch professionals to ensure common ground for communications about this weather information. AWIN researchers helped define appropriate indications for weather product age in the Minimum Aviation System Performance Standards (MASPS) for FISDL [13], and participate in an industry working group (RTCA SC-195) to develop common guidelines for color-coding radar return intensity levels across aviation weather information systems for different NAS users.

The results of this survey provide insight on how GA pilots value and use current aviation weather information, services, and products. These results were extrapolated to develop a model of how these users value weather information for flight decisions, to provide design guidance for aviation weather information systems, and to frame future research for their development. With a better understanding of what makes weather information valuable to a pilot, we can improve aviation safety by helping to make sure GA pilots have access to the right information in a readily usable format and ensure they are provided with appropriate and meaningful indications of reliability, phenomenological intensity, and spatial/temporal relevance. NASA AWIN, working with academia, other government agencies, and industry, conducts research in these areas to improve GA pilots' weather situation awareness and the use of weather factors in aviation decision-making, and thereby improve safety in general aviation operations.

References

[I] Latorella, K.A. & Chamberlain, J.P. (2001). Decision-making in flight with different convective weather sources: Preliminary Results. In Proceedings of the Eleventh International Symposium on Aviation Psychology. Columbus, OH.

[2] Imeson, S. (2001) Scud Running (http:// www. moutainflying.com! scudrun.htm#PageTop).

Mountain Flying, LLC (accessed on September 25, 2001). Harbor City, CA.

[3] U.S. Federal Aviation Administration. (1985). Advisory Circular No: 61-84B, Role of Preflight Preparation. U.S. Federal Aviation Administration. Washington, DC.

[4] Aircraft Owners and Pilots Association (AOPA) (1996). Safety Review General Aviation Weather Accidents: An Analysis and Prevention Strategies. AOPA Air Safety Foundation. Frederick, MD.

[5] Yucknovicz, D., Burgess, M., Heck, M., & Novacek, P. (2000). Assessment of the effects of delayed weather information datalinked to the cockpit on pilot navigation decision making. Presented at IEEE AIAA Digital Avionics Systems Conference. Philadelphia, PA.

[6] Latorella, K.A., Pliske, R., Hutton, R., & Chrenka, J. (2001). Cognitive task analysis of business jet pilots' weather flying behaviors: Preliminary results. NASA-TM-2001-211034. NASA Langley Research Center. Hampton, VA.

[7] Clark, H. H. & Brennan, S. E. (1993). Grounding in communication. In Baecker, R. M (Ed.)

Readings in Groupware and Computer-Supported Cooperative Work, Assisting Human-Human Collaboration, pp. 222-233. Morgan Kaufmann Publishers, Inc. San Francisco, CA.

[8] Chandra, D.C. (1995). Pilot evaluation of datalink services for general aviation. The Eighth International Symposium on Aviation Psychology, 679-685. Columbus, OH.

[9] Kauffmann, P. & Pothanun, K. (2000). Estimating the rate of technology adoption for cockpit weather information systems. In Proceedings of the Society of Automotive Engineers General Aviation Technology Conference. Huntsville, AL.

[10] Sireli, Y., Ozan, E. & Kauffmann, P. (2001). A market research study for future weather information systems in general aviation. In Proceedings of the American Society of Engineering Management Conference. Witchita, KS.

[11] Vicente, K,, & Rasmussen, J. (1992). Ecological interface design: theoretical foundations. IEEE Transactions on System, Man, and Cybernetics, 22(4): 589-606.

[12] Chamberlain, J.P. & Latorella, K.A. (2001). Convective weather detection by general aviation pilots with conventional and data-linked graphical weather information sources. In Proceedings of the 20th Digital Avionics Systems Conference. Daytona Beach, FL.

[13] Radio Technical Commission for Aeronautics, Special Committee-195. (2000). Minimum Aviation System Performance Standards (MASPS) for Flight Information Services-Broadcast (FIS-B) Data Link. Radio Technical Commission for Aeronautics (RTCA) DO-267. Washington, DC.

Acronyms & Abbreviations ! 22.0 Aviation Communication Frequency used for EFAS AC Advisory Circular ACARS Aircraft Communications Addressing and Reporting System AIAA American Institute of Aeronautics and Astronautics AOPA Aircraft Owners and Pilots Association ARTCC Air Route Traffic Control Centers ASOS Automated Surface Observing System Air Traffic Control ATC Automatic Terminal Information Service ATIS AWlN Aviation Weather Information (a NASA Aviation Safety Program Element) AWOS Automated Weather Observing System CWA (ATC) Center Weather Advisory DA Density Altitude DUATS Direct User Access Terminal System EFAS En Route Flight Advisory Service - also known as Flight Watch FA Area Forecast Federal Aviation Administration FAA FBO Fixed Base Operator FD Winds and Temperatures Aloft Forecast FISDL Flight Information Services Data Link FMS Flight Management System FSS Flight Service Station FW Flight Watch GA General Aviation

GAMA

General AviationManufacturers Association

GPS

Global Positioning System

HIWAS

Hazardous In-flightWeather Advisory Service

1FR

Instrument FlightRules

IMC

Instrument Meteorological Conditions

ITWS

Integrated Terminal Weather System

MASPS

Minimum AviationSystem Performance Standards

METAR

Aviation Routine Weather Report

MIS

Meteorological Impact Statement

MVFR

Marginal Visual Flight Rules

NAS

National Airspace System . : .

NASA

National Aeronautics and Space Administration

NAVAID

Navigational Aid

NOTAM

Notices to Airmen

NTSB

National Transportation Safety Board

PATWAS

Pilot Automated Telephone Weather Answering Service

PIREP

Pilot Weather Report

RADAT

Radiosonde Additional Data

RTCA-SC 195

Radio Technical Commission for Aeronautics, Special Committee-195 (Flight Information Services Communications)

SA Situation Awareness

SD

Radar Weather Report

SIGMET

Significant Meteorological Information

TAF

Terminal Aerodrome Forecast

TDWR

Terminal Doppler Weather Radar

TIBS

Telephone Information Briefing Service

TWEB Transcribed Weather Broadcast

VFR

Visual Flight Rules

VMC

Visual Meteorological Conditions

WA

In-Flight Aviation Weather Advisories Severe Weather Watch Bulletin WW Appendix: Survey Instructions and Instrument.

5A technologies

Resear_ . Z_es/_ • tnfc_rmatiot_ , Systems Survey on Aviation Weather Services and Products With the support of NASA, Embry-Riddte Aeronautical University, SA Technologies, Inc., and Search Technology, Inc. are obtaining information to develop a model of how aviation operator communities gather and use weather information and how weather-related decisions are made between flight crews and supporting personnel. General Aviation, Business Jet/Corporate, Commercial Transport Pilot, and Dispatcher/ATC/FBO input is critical to this design process. Your participation is deeply appreciated.

Participation in this survey is completely voluntary. It is not necessary to give your name at any point. You may decline to answer any of the questions in this survey. All surveys will be de-identified and all information obtained from any individual survey will be kept confidential by Embry-Riddle Aeronautical University, SA Technologies, Inc., Search Technology, Inc. and NASA.

Please do not forget to SUBMIT the survey once completed. If you have any questions or comments regarding the survey, please feel free to contact us.

Thank you for your time and cooperation.

Daniel J. Garland, Ph.D.

SA Technologies, Inc.

4035 Watkins Glen Drive Woodstock, GA 30189 USA Tel: (+ 1)770-592-8124 Fax: (+ 1)770-592-9434 E-Mail:garland@satechnologies.com 1. Please provide the following information so we may contact you directly (all information provided will be kept confidential).

First Name Last Name

I I I I

Street Address City State Zip

I I I I

Home Phone Work Phone

I I I I

Email Address

I I

2. Would you be interested in participating in future NASA surveys or experiments? O Yes O No 3. Please select the most appropriate operation for which you fly and/or work: Please select one (pull down selection list): I Major commercial airline Regional commercial airline Major commercial airline Regional commercial airline Foreign commercial airline Freight carder Business Jet/Corporate Charter Military General Aviation Helicopter Student Pilot Dispatcher FBO Air Traffic Control j_.N :_ j i [¢1:i IITA_| __t i |i:li|j l i =[e]i| ?_ F.! i [,], ----- 4. During preflight preparation, a variety of information is available about current and future weather conditions. Please rate the importance of these types of information with regards to how they affect your flight planning decisions. If this information is not available to you, click on the "Information Not Available" choice.

Preflight Weather Information Info Not NotAt All Somewhat Very Extremely Importance Rating I Available Important Important Important . Important Ceilings (IFR) o o o o o Ceilings _MVFR) o o o o o _onvectlve _ct_vity (ITW_DW_ - Termmal)'_--_d ....................... o_ ................... _0 o o ._=_,:_:;.

Convective Activity NowCast-Enroute o o o o o Icin_ - Ground Conditions and Forecast o o o o o _i_i_ _ .................. :"_: .................... .................................................... ;........... _ .... _ o o '_ __----- SIGMETS o o o o o _°_'-(a ) .... 5 .............................. b"....... o o o Turbulence.(Known and Forecast), o o o o o :'_ - - ....... . "L_=L_=C.L.L_ JL. L ....... -.=- - ........

Winds Aloft o o o o o Other: Please List: _ __= == = 5.During preflight preparation, a variety ofadditional aviation weather services andproducts

areavailable about weather conditions. Please ratetheimportance ofthesesources with

regards to howtheyaffectyourflightplanning preparations. If thisinformation is not

available toyou,clickonthe"Information NotAvailable" choice.

Preflight Weather Services and Products lnfo Not NotAt All Somewhat Very Extremely

I Im portance Rating I

Available |mportant Important lmporta, nt Important Automated Surface Observation System (ASOS) o o o o o Aviation Area Forecast {FA) ..-_ ................... o o o o o Composite Moisture Stabili_ Charts _ o o o o o • . o .....

Convective Outlook O O O o o _rect User Access Terminal Sy@tgm (_Y_ : --0 ....... _3 e : b_ -Fiiai_t Service Station Briefing., o o o o o METARS o o o o o Newspaper o o o o o An swerin_ System (PATWAS) o o o Pilots by word of mouth o o o o o S0m_ .. ................................ .......................................... ................................... b- ............................

Radar Weather O O O o O Satellite Weather Pictures O O O O o Surface Analysis Charts o o o o o Televisi__.:.&_Weathe r Channel) o o o o o Transcribed Weather Broadca_ .................... _ o o o o o _==_= _..-.. ............................................ __ ,-, ._ ................................. _-. .............................. .,;_ ....... _ ........................................ _..-.

i Weather Depiction Chart ...... ...........

Winds and Temperatures Aloft Forecast (FD) o o o o o Other: Please List:

6. Forpreflight preparation, a variety of aviation weather resources are available on the

Internet. Please indicate any Internet weather resource(s) you use for preflight planning and decision making. In addition, please provide the information you obtain from these lnternet sites and how you use the information for preflight preparations. Check all that apply.

Use? Internet Web Site [] AOPA (aopa.org) [] Aviation Weather Center (awc-kc.noaa.gov) [] DUATS (sk_central.comJ [] NCAR (rap.ucar.edu/weather) [] Naval Research _.miIlsat_ Comments: 7. How do you make preflight decisions related to weather (e.g., Who do you talk to?, Is it a collaborative decision makin_ process?). Please explain.

8 Once the flight is enroute additional information is available about weather conditions Please rate the importance of this information for in flight decision making If this information is not available to you click on the Information Not Available' choice tnfo Not Not At All Somewhat Very Extremely in-Flight Weather Information Importance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Available..... impoffant Important Important Important AIRMETS (WA) ........ o o o o o A T C d i s t : u s s i o n o f r i d e relpo_s _ O O O O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . , O 1 . . . . . . . . . . . . . . . . . . . .

Ceilings _ o o o o o Convective Activity NowOast Enroute o o o o o Icing - Airborne (Known and Forecast) o o o o o Lightning o o o O o Turbulence .(Known and Forecast) o o o o o from other aircraft on your frequency o o o o O i]_.ViS_ibi!ity : v ..... _,' .... _.

Winds Aloft o o o o o 0 0 0 0 0 Wind Shear Low level Other: Please List 9.Oncetheflightisenroute, additional weather services andproducts areavailable about weather conditions• Please ratetheimportance ofthisinformation forin-flight decision making• If this information is notavailable to you,clickonthe"Information NotAvailable" choice.

Other: Please List

10.Istherea difference between the information used during the flight to "keep up with the weather situation" and information used to "make decisions enroute"?

[] Yes [] No If yes, please explain: 11. If near real-time weather information could be displayed in the cockpit, how useful would it be?

Please select one (pull down selection list): I Limited Useful No Utility J Very Useful Essential 12. If near real-time weather information could be displayed onto a moving map display, how useful would it be?

Please select one (pull down selection list): I Limited Useful No Utility t Very Useful Essential 13. If near real-time information could be displayed via a handheld portable device, how useful would it be?

Please select one (pull down selection list): I Limited Useful No Utility J Very Useful Essential 14. To be useful during in-flight decision making, what weather information needs to be displayed in the cockpit? Please explain.

L

15. How do you make in-flight decisions related to weather (e.g., Who do you talk to?, Is it a collaborative decision making process?). Please explain

I

16. We would like all weather information to be timely so that pilots can know far in advance what lies ahead in order to avoid making spur of the moment decisions based on last minute weather information. Please rate the usefulness of weather information for each phase of flight.

I ........... Usefulness of Weather Data Phase of Flight No Utility Limited Useful Very Useful Essential

I

Prefl_ht o o o o o Takeoff o o o o o Cruise o o o o o _eroach o o o o o ........... ...................................... ::: ............... _0 ......... ........................ .........................

Taxi-in o o ......... 5 .......................................... o o 17. Flight experience: Total hours in flight operation category chosen in section General Experience Information above I I hours Total hours of flight experience ] hours 18. Where do you typically fly?

[] Southeast (FL, GA, SC, NC, TN, AL, MS, KY) [] Northeast (VA, WV, OH, PA, NY, DE, MD, MA, N J, CT, RI, NH, VT, ME) [] Southwest (TX, LA, AR, NM, AZ, OK) [] Midwest (KS, MO, IO, NE, SD, ND, MN, Wl, WY, CO, IL, MI, IN) [] West (CA, UT, NV, ID, MT, WA, OR, HI, AK) [] Trans-Continental (U.S.)

[] International (Trans-Atlantic) [] International (Trans-Pacific) [] Other 19. Please list the three most recent types of aircraft on which you have experience, beginning with the most recently flown and check the column to indicate your approximate number of hours flying experience.

Aircraft Type Under 300 300-1000 Over 1000 © O O O O O I ........................ I....... Hours in Type O © O 5O

20.Please check theappropriate "Cost(inthousands)" column foreachofthe

questions below.

I

............................................................................. Cost (in thbusands) ........................

Weather Technology Cost Information Under $1,000 $1,000-$5000 Over $5000 1 How much would you be willing to pay to 0 0 0 buy a cockpit weather system?

How much would you be willing to pay to 0 0 0 install the cockpit weather system?

How much would you be willing to pay for 0 0 0 a weather service (e.g., satellite broadcast direct to the cockpit during flight)?

21. How would you prefer to pay for the weather service?

Please select one (pull down selection list): By Month I ' 1999 Copyright SA Technologies, Inc. All Rights Reserved.

Form Approved REPORT DOCUMENTATION PAGE OMBNO. 0704-0188 Public reporting burden for this collection of informationis estimated to average1 hour per response, including the time for re'.,dewing insfruc'ffions, Searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite t 204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-01138), Washington, DC 20503.

ii 1. AGENCY USE ONLY (Leave blank) 12. REPORT DATE 3. REPORTTYPE AND DATES COVERED February 2002 Technical Memorandum

I

4. TITLE AND SUBTITLE 5. FUNDING NUMBERS General Aviation Pilots' Perceived Usage and Valuation of Aviation Weather Information Sources WU 728-40-10-03 6. AUTHOR(S) Kara Latorella, Suzanne Lane, and Daniel Garland 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZA'I'ION REPORT NUMBER NASA Langley Research Center Hampton, VA 23681-2199 L- 18159 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITORING AGENCY REPORT NUMBER National Aeronautics and Space Administration NASA/TM-2002-211443 Washington, DC 20546-0001 11. SUPPLEMENTARY NOTES Latorella and Lane: Langley Research Center, Hampton, VA; Garland: Embry-Riddle Aeronautical University, Daytona Beach, FL.

12a. DISTRIBUTION/AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE Unclassified-Unlim ited Subject Category 03 Distribution: Standard Availability: NASA CASI (301) 621-0390 13. ABSTRACT (Maximum 200 words) Aviation suffers many accidents due to the lack of good weather information in flight. Existing aviation weather informa- tion is difficult to obtain when it is most needed and is not well formatted for in-flight use. Because it is generally pre- sented aurally, aviation weather information is difficult to integrate with spatial flight information and retain for reference.

Efforts, by NASA's Aviation Weather Information (AWlN) team and others, to improve weather information accessibility, usability and decision aiding will enhance General Aviation (GA) pilots' weather situation awareness and decision-mak- ing and therefore should improve the safety of GA flight. Consideration of pilots' economic concerns will ensure that in- flight weather information systems are financially accessible to GA pilots as well. The purpose of this survey was to describe how aviation operator communities gather and use weather information as well as how weather related decisions are made between flight crews and supporting personnel. Pilots of small GA aircraft experience the most weather-related accidents as well as the most fatal weather related accident. For this reason, the survey design and advertisement focused on encouraging participation from GA pilots. Perhaps as a result of this emphasis, most responses, 97 responses or 85% of the entire response set, were from GA pilots. This paper presents only analysis of these GA pilots" responses. The insig.hts provided by this survey regarding GA pilots' perceived value and usage of current aviation weather information, services, and products provide a basis for technological approaches to improve GA safety. Results of this survey are dis- cussed in the context of survey limitations and prior work, and serve as the foundation for a model of weather information value, guidance for the design of in-flight weather information systems, and definition of further research toward their development.

14. SUBJECT TERMS 15. NUMBER OF PAGES General aviation, pilots, usability, weather information systems, graphical weather, AWIN.

avionics, design 16. PRICE CODE A04 17. SECURITY CLASSIFICATION 18. SECURITY CLASSIFICATION 19. SECURITY CLASSIFICATION 20. LIMITATION OF REPORT OFTHIS PAGE OF ABSTRACT OF ABSTRACT Unclassified Unclassified Unclassified UL NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) Prescribed by ANSi Std Z39-18 298-102