Document
Paper number 10.2
NASA/TM—2011-217141
Comparison of In-Situ, Model and
Ground Based In-Flight Icing Severity
Christopher J. Johnston, David J. Serke, and Daniel R. Adriaansen National Center for Atmospheric Research, Boulder, Colorado Andrew L. Reehorst Glenn Research Center, Cleveland, Ohio Marcia K. Politovich, Cory A. Wolff, and Frank McDonough National Center for Atmospheric Research, Boulder, Colorado
December 2011
NASA STI Program . . . in Profile
Since its founding, NASA has been dedicated to the • CONFERENCE PUBLICATION. Collected advancement of aeronautics and space science. The papers from scientific and technical NASA Scientific and Technical Information (STI) conferences, symposia, seminars, or other program plays a key part in helping NASA maintain meetings sponsored or cosponsored by NASA.
this important role.
• SPECIAL PUBLICATION. Scientific, The NASA STI Program operates under the auspices technical, or historical information from of the Agency Chief Information Officer. It collects, NASA programs, projects, and missions, often organizes, provides for archiving, and disseminates concerned with subjects having substantial NASA’s STI. The NASA STI program provides access public interest.
to the NASA Aeronautics and Space Database and its public interface, the NASA Technical Reports • TECHNICAL TRANSLATION. English- Server, thus providing one of the largest collections language translations of foreign scientific and of aeronautical and space science STI in the world. technical material pertinent to NASA’s mission.
Results are published in both non-NASA channels and by NASA in the NASA STI Report Series, which Specialized services also include creating custom includes the following report types: thesauri, building customized databases, organizing and publishing research results.
• TECHNICAL PUBLICATION. Reports of completed research or a major significant phase For more information about the NASA STI of research that present the results of NASA program, see the following: programs and include extensive data or theoretical analysis. Includes compilations of significant • Access the NASA STI program home page at scientific and technical data and information http://www.sti.nasa.gov deemed to be of continuing reference value.
NASA counterpart of peer-reviewed formal • E-mail your question via the Internet to help@ professional papers but has less stringent sti.nasa.gov limitations on manuscript length and extent of graphic presentations. • Fax your question to the NASA STI Help Desk at 443–757–5803 • TECHNICAL MEMORANDUM. Scientific and technical findings that are preliminary or • Telephone the NASA STI Help Desk at of specialized interest, e.g., quick release 443–757–5802 reports, working papers, and bibliographies that contain minimal annotation. Does not contain • Write to: extensive analysis. NASA Center for AeroSpace Information (CASI) 7115 Standard Drive • CONTRACTOR REPORT. Scientific and Hanover, MD 21076–1320 technical findings by NASA-sponsored contractors and grantees.
Paper number 10.2
NASA/TM—2011-217141
Comparison of In-Situ, Model and
Ground Based In-Flight Icing Severity
Christopher J. Johnston, David J. Serke, and Daniel R. Adriaansen National Center for Atmospheric Research, Boulder, Colorado Andrew L. Reehorst Glenn Research Center, Cleveland, Ohio Marcia K. Politovich, Cory A. Wolff, and Frank McDonough National Center for Atmospheric Research, Boulder, Colorado Prepared for the 15th Symposium on Integrated Observing and Assimilation Systems for the Atmosphere, Oceans, and Land Surface (IOAS-AOLS) sponsored by the American Meteorological Society Seattle, Washington, January 27, 2011 National Aeronautics and Space Administration Glenn Research Center Cleveland, Ohio 44135
December 2011
Acknowledgments This research is supported by the NASA Aviation Safety Program under the Atmospheric Environment Safety Technologies (AEST) project.
Trade names and trademarks are used in this report for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.
Level of Review : This material has been technically reviewed by technical management.
Available from NASA Center for Aerospace Information National Technical Information Service 7115 Standard Drive 5301 Shawnee Road Hanover, MD 21076–1320 Alexandria, VA 22312 Available electronically at http://www.sti.nasa.gov
Comparison of In-Situ, Model and Ground Based In-Flight Icing Severity
Christopher J. Johnston, David J. Serke, and Daniel R. Adriaansen National Center for Atmospheric Research Boulder, Colorado 80307 Andrew L. Reehorst National Aeronautics and Space Administration Glenn Research Center Cleveland, Ohio 44135 Marcia K. Politovich, Cory A. Wolff, and Frank McDonough National Center for Atmospheric Research Boulder, Colorado 80307 and other weather-related conditions. Both a subjective icing
1.0 Introduction
severity and icing type (rime, clear or mixed) are included.
PIREP reports of no icing are useful as well, since the absence In-flight icing is a significant hazard for the aviation indus- of icing is important information. The shortcomings of PIREPs try. It occurs when supercooled liquid water (SLW) comes in are well documented and include non-uniformity in time or contact with, and freezes to, the leading surfaces of an aircraft.
space and contamination by errors in location, altitude and time This can significantly alter the aircraft’s aerodynamic (Brown et al., 1997; Kelsch and Wharton, 1996). PIREPs can properties by increasing the amount of drag on the aircraft, sometimes be inaccurate due to time lags before the pilot reports and reducing the lift. Since practical airborne remote detection the observed icing condition, and whether he or she reports the hardware has not yet been developed, a ground-based correct altitude and location. The reported severity is also detection system that can provide information to all aircraft somewhat subjective as it can vary based on aircraft type, phase entering and departing a terminal area (Fig. 1) is a key element of flight, and pilot experience. Nevertheless, PIREPs are our in facilitating icing avoidance (Serke et al., 2010).
only means of in-situ diagnoses of actual atmospheric condi- Currently there are two systems that are being developed for tions encountered by pilots and their aircraft in the absence of the detection of in-flight icing. The first detection system is the expensive icing research flights or specially instrumented fleet NASA Icing Remote Sensing System (NIRSS) a testbed that aircraft. The objective of this study is to examine how the integrates three vertically pointing sensors; a Vaisala Laser testbed NIRSS icing severity product and the operational CIP Ceilometer, a Metek K -band radar, and a Radiometrics Corpora- a severity product compare to PIREPs of icing severity, and how tion 23-channel radiometer (Fig. 2), (Reehorst et al., 2006).
The multichannel microwave radiometer has the ability to the NIRSS and CIP compare to each other.
derive integrated liquid water (ILW), atmospheric water vapor and temperature profiles (Solheim et al., 1998). A Vaisala laser
2.0 Methodology
ceilometer is used to define cloud base heights, and a Metek K - a band radar is used to delineate cloud top and base heights.
A 3-year database of CIP, NIRSS and PIREP data was NIRSS combines the ILW, radar reflectivity, temperature compiled focusing on winter periods from early November profile, and cloud top and base heights to determine the 2008 to late March 2010. During these three winter seasons, presence of in-flight icing conditions in the atmosphere.
917 icing PIREPS were collected within 40 km of the NIRSS The second in-flight icing detection system is the Current system located in Cleveland, Ohio. CIP icing severity output Icing Product (CIP) which was developed at the National from the nearest RUC gridpoint to the NIRSS location was Center for Atmospheric Research. This system combines archived and icing severity values were extracted at the time visible and infrared satellite imagery, radar reflectivity, and height of each icing PIREP. A similar process was lightning observations, Pilot Reports (PIREPs) and standard conducted for NIRSS severity output. If there was no icing ground-based weather observations with numerical model data at the exact PIREP altitude, a vertical search was output to produce a gridded, hourly, three dimensional performed for the nearest altitude with icing severity data.
representation of icing probability and severity (Bernstein et Once this temporal and spatial matching was completed for all al., 2005). Each horizontal grid point of CIP is based on a PIREPs, a statistical comparison was begun. Analysis 20 km by 20 km Rapid Update Cycle (RUC) model grid point.
occurred from the ground level to ~ 30,000 ft (or 9,144 m).
First developed during the winter of 1997/98, CIP became an For this study, a PIREP reported over a range of heights is operational National Weather Service product in 2002.
treated as multiple PIREPs spread over 1000 ft increments Icing-related PIREPs are voluntary reports made by pilots to (Wolff et al., 2010).
report on the presence or absence of in-flight icing conditions NASA/TM—2011-217141 1
3.0 Analysis and Discussion
3.1 Case Study Comparison—December 15, An example icing case study is presented here to illustrate how the comparison of NIRSS, CIP and PIREP severity looked for a single event. In the next section, statistics for 3 years of such cases are discussed.
On December 15, 2009, at 0000 UTC, a surface low was dominant over the central Great Lakes region (Fig. 3). The warm front extended from the southeast portion of Lake Huron eastward into southeastern New York. The cold front No icing was oriented from north central Ohio through southwest Ohio.
The cloud top temperatures over the Cleveland area were Figure 1.–NIRSS in-flight icing detection concept.
between –5 and –15 °C (color scale). This temperature range has been shown in previous research to be conducive to supercooled liquid water (Rogers and Floyd, 1989).
At approximately 0300 UTC, the cold front passed through Cleveland, Ohio. In the hours following the cold frontal passage, drizzle and rain fell over the metropolitan area, which changed to snow by 1500 UTC.
Figure 4 shows time versus height of icing severity from NIRSS (top), PIREPs (top numerals) and CIP (bottom) for December 15, 2009. A zero to eight scale for icing severity was used for these plots, and the comparisons throughout the rest of this study where zero is no icing, one is trace amount of icing, two to three is light, four to five is moderate and six to eight is heavy.
Twenty-eight positive icing PIREPs were recorded between 1100 and 2200 UTC around Cleveland. In addition, three negative PIREPS were recorded during this time period.
NIRSS diagnosed significant icing between 0900 and Figure 2.—Image of the NIRSS hardware located at the NASA 2400 UTC from 1 to 6 kft AGL. CIP diagnosed icing from Glenn Research Center in Cleveland, Ohio.
0900 to 2400 UTC as well, from roughly 1 to 9 kft AGL. For this case, the temporal variability and the magnitude of the severities generally match between the two products despite the fact that CIP has a 1-hr time resolution and NIRSS has one-minute resolution. CIP has a conservative cloud top and base estimate scheme (done purposely to insure thorough warnings).
3.2 3-Year Archive Comparison In the previous section we explored the comparison of icing products for a day-long icing case. This section will be a statistical intercomparison of all three icing detection methods for the full 3-year study period. Similar to the case study presented above, the closest NIRSS and CIP icing severity Figure 3.—0000 UTC surface pressure (yellow lines, [mb]), measurements were found to each of the 917 PIREPs within cloud top temperature (color bar, [°C]) and frontal analy- 40 km of the NIRSS location. The matched severity categories sis for December 15, 2009.
were plotted in Figures 5 to 7.
NASA/TM—2011-217141 2
NIRSS
[kft] [hh]
CIP
[kft] [hh] Figure 4.—Time [hh] versus height [kft] plots of NIRSS (top, color scale), PIREP (top, red numerals) and CIP (bottom, color scale) icing severity from December 15, 2009 from 0000 UTC to 2400 UTC.
NIRSS versus PIREP severity is shown in Figure 5, with NIRSS is still a testbed. A linear best-fit line is again shown in one-to-one severity correlation bins highlighted in orange. The blue, with NIRSS having a severity category correlation of numbers in each bin represent the total number of icing 0.18 to the CIP severity category. There seems to be a severity matchups recorded for the 3-year period. A linear significant spread in the collocated severity values between best-fit line is overlain in blue. Taking the square root of the the two products during the 3-year study period. This spread is resulting R-squared value gives NIRSS a severity category likely due to the difference in temporal resolution of the correlation coefficient of 0.35 to the PIREP severity category. products, and the fact that the two products arrive at hazard NIRSS appears to do well locating negative PIREPs, as well estimates based on different input datasets.
as finding moderate icing PIREPs. Very few severe or heavy PIREPs were reported during this time period. There are a significant number of positive PIREPs that NIRSS identifies 8 0 0 0 0 0 0 0 0 0 as negative severity, possibly due to the high time resolution N 7 0 0 0 13 0 0 0 0 0 of NIRSS’s ILW algorithm when viewing localized SLW I 6 1 1 0 18 4 29 0 0 0 cases.
R 5 1 7 0 28 8 23 0 0 0 CIP versus PIREP severity is shown in Figure 6. A linear S 4 1 5 0 46 7 15 0 0 0 best-fit line is again overlain in blue, with CIP having a severity category correlation of 0.21 to the PIREP severity S 3 6 18 0 95 18 22 0 0 0 category. CIP and NIRSS both do well at finding PIREPs from 2 9 20 0 71 3 28 0 0 1 light to moderate values. CIP seems to correctly identify a 1 28 10 1 71 12 14 0 0 2 much smaller fraction of negative PIREPs than NIRSS. This 0 115 19 0 93 16 34 4 0 0 could be a result of the conservative cloud base and top 0 1 2 3 4 5 6 7 8 diagnosis in CIP.
PIREP NIRSS versus CIP severity is shown in Figure 7. The opera- tional CIP product is treated as ‘truth’ in this comparison, as Figure 5.—Overall PIREP severity versus NIRSS severity.
NASA/TM—2011-217141 3 TABLE 1.—OVERALL POD AND POD STATISTICS n y 8 0 0 0 0 0 0 0 0 0 [N versus P (NIRSS versus PIREPS), C versus P 7 1 1 0 5 0 4 0 0 0 (CIP versus PIREPS), N versus C (NIRSS versus CIP)] 6 4 0 0 1 1 1 0 0 0 N versus P C versus P N versus C C 5 5 10 0 21 7 13 0 0 3 POD 0.78 0.90 0.79 y POD 0.71 0.29 0.59 12 n I 4 13 0 78 20 43 0 0 0 P 3 22 14 0 94 20 35 0 0 0 2 32 12 1 97 13 40 0 0 0 NIRSS detected greater than 70 percent of both positive and 1 38 23 0 92 6 7 0 0 0 negative PIREPs. CIP detected 90 percent of positive PIREPs 0 47 4 0 47 0 22 4 0 0 but only 29 percent negative PIREPs. The percentage of the 0 1 2 3 4 5 6 7 8 time averaged vertical profile that a product has identified a PIREP positive severity value is termed the warning volume. The warning volume for this study is calculated from the surface to Figure 6.—Overall PIREP severity versus CIP severity.
the average height of the tropopause. A successful product must find an optimal balance between POD yes and no and warning volume because it would not be very useful for a 8 0 0 0 0 0 0 0 0 0 product to have a POD of 1.0 (perfect icing detection) if the y N 7 2 6 5 0 0 0 0 0 0 entire column is warned on at all times. Ideally, a product I 6 5 4 8 11 15 7 1 2 0 would have a maximized POD and POD with a minimized y n R 5 12 7 7 15 14 8 1 3 0 warning volume. For the 3-year study period, NIRSS had a S 4 7 9 13 17 21 6 1 0 0 mean warning volume of 13 percent, and CIP had a mean warning volume of 34 percent. CIP detected 10 percent more S 3 9 32 36 37 35 9 0 1 0 positive PIREPs in over twice the warning volume. Further- 2 14 18 41 30 23 6 0 0 0 more, CIP’s high warning volume causes it to classify regions 1 3 29 42 27 20 10 3 4 0 as positive icing where they should be devoid of icing, based 0 76 61 43 0 38 13 1 1 0 on negative PIREPs.
0 1 2 3 4 5 6 7 8 CIP
Summary
Figure 7.—Overall CIP versus NIRSS severity.
In-flight icing detection is crucial to achieving a high safety standard for the national fleet of commercial and general Another useful statistic is the probability that each product aviation aircraft. In this study a comparison was done showing will detect negative and positive icing PIREPs, or the the quantitative severity categories of negative and positive Probability of Detection (POD). These statistics are termed icing PIREPs to the quantitative icing severity derived from POD and POD To get POD n y, respectively. n, the fraction of the prototype NIRSS icing detection algorithm and operational negative icing PIREPs that the respective product identifies as CIP icing algorithm. An icing case study from December 15, negative icing is determined and then divided by the total 2009, over the NIRSS location in Cleveland, Ohio, was cases where PIREP severity is equal to zero (Eq. (1)).
discussed to illustrate how the PIREP and icing product severities were compared. A statistical analysis over the full POD = (Total Cases product icing severity = 0 when PIREP severity = 0) (1) n 3-year study period found that NIRSS detected in-flight icing (Total cases PIREP icing severity = 0) and negative icing at least as good as CIP when compared to all PIREPs within a 40 km radius. NIRSS detected almost Similarly, POD is the fraction of positive icing PIREPs of any y 80 percent of positive PIREPs and over 70 percent of negative category (one through eight) that the respective product PIREPs in a relatively smaller warning volume. CIP detected identifies as positive icing divided by the total cases where slightly more positive PIREPs than NIRSS but did fairly poor PIREP severity is greater than zero (Eq. (2)).
in detecting negative PIREPs. This occurred in a warning volume over twice the percent of NIRSS’s warning volume.
POD = (Total cases product icing severity > 0 when PIREP severity >0) (2) y (Total cases PIREP icing severity >0) CIP did very well at detecting positive PIREPs. NIRSS displayed respectable probabilities of icing detection with For the 3-year study period, POD and POD were calcu- y n lower warning volumes than CIP. This is due to NIRSS having lated for the product comparisons shown in Figures 5 to 7. The a higher time resolution and utilizing physically based vertical results are shown in Table 1.
profiles of ILW, temperature and radar reflectivity. Therefore, NASA/TM—2011-217141 4 the NIRSS testbed in-flight icing severity product seems to be Reehorst, A., Politovich, M.K., Zednik, S., Isaac, G.A., and at least as good as CIP. A shortfall of NIRSS is that it Cober, S., 2006: Progress in the Development of Practical currently lacks volumetric scanning capability. This is being Remote Detection of Icing Conditions (NASA/TM—2006- 214242).
addressed by the addition of a 1 ° beamwidth multichannel Rogers, R.R., M.K. Floyd: A short course in cloud physics.
scanning radiometer (Serke et al., 2010). Future work with NIRSS will include exploring Doppler fall velocities to detect Pergamon Press, Oxford, England.
Serke, D.J., Beaty, P., Reehorst, A.L., Kennedy, P., Solheim, possible freezing drizzle and freezing rain, and comparing NIRSS hazard detection to the polarimetric data from future F., Ware, R., Politovich, M.K., Brunkow, D. and Bowie, R.
“A New Narrowbeam Multi-Frequency Scanning Radiome- upgraded NEXRAD.
ter and its Application to In-Flight Icing Detection,” AMS 14th Conference on Aviation, Range, and Aerospace Mete-
References
orology (ARAM), 2010.
Solheim, F., Godwin, J., Westwater, E., Han, Y., Keihm, S., Bernstein, B., McDonough, F., Politovich, M., Brown, B., Marsh, K., and Ware, R., “Radiometric profiling of tem- Ratvasky, T., Miller, D., Wolff, C., and Cunning, G., “Cur- perature, water vapor and cloud liquid water using various rent Icing Potential: Algorithm description and comparison inversion methods,” Radio Sci., 33, pp. 393-404, 1998.
to aircraft observations,” Journal of Applied Meteorology, Wolff, C. A., McDonough, F. “A Comparison of WRF-RR 44, pp. 969-986, 2005.
th and RUC Forecasts of Aircraft Icing Conditions,” AMS 14 Brown, B.G., G. Thompson, R.T. Bruintjes, R. Bullock, and T.
Conference on Aviation, Range, and Aerospace Meteoro- Kane, 1997: Intercomparison of in-flight icing algorithms.
logy (ARAM), 2010.
Part II: Statistical verification results. Wea. Forecasting , 12, 890-914.
Kelsch, M., and L. Wharton, 1996: Comparing PIREPs with NAWAU turbulence and icing forecasts: Issues and results.
Wea. Forecasting , 11, 385-390.
NASA/TM—2011-217141 5 Form Approved REPORT DOCUMENTATION PAGE OMB No. 0704-0188 The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302.
Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.
PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.
1. REPORT DATE (DD-MM-YYYY) 2. REPORT TYPE 3. DATES COVERED (From - To) 01-12-2011 Technical Memorandum 5a. CONTRACT NUMBER 4. TITLE AND SUBTITLE Comparison of In-Situ, Model and Ground Based In-Flight Icing Severity 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER Johnston, Christopher, J.; Serke, David, J.; Adriaansen, Daniel, R.; Reehorst, Andrew, L.; Politovich, Marcia, K.; Wolff, Cory, A.; McDonough, Frank 5e. TASK NUMBER 5f. WORK UNIT NUMBER WBS 648987 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT NUMBER National Aeronautics and Space Administration E-17878 John H. Glenn Research Center at Lewis Field Cleveland, Ohio 44135-3191 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITOR'S ACRONYM(S) National Aeronautics and Space Administration NASA Washington, DC 20546-0001 11. SPONSORING/MONITORING REPORT NUMBER NASA/TM-2011-217141 12. DISTRIBUTION/AVAILABILITY STATEMENT Unclassified-Unlimited Subject Category: 03 Available electronically at http://www.sti.nasa.gov This publication is available from the NASA Center for AeroSpace Information, 443-757-5802 13. SUPPLEMENTARY NOTES 14. ABSTRACT As an aircraft flies through supercooled liquid water, the liquid freezes instantaneously to the airframe thus altering its lift, drag, and weight characteristics. In-flight icing is a contributing factor to many aviation accidents, and the reliable detection of this hazard is a fundamental concern to aviation safety. The scientific community has recently developed products to provide in-flight icing warnings. NASA's Icing Remote Sensing System (NIRSS) deploys a vertically--pointing Ka--band radar, a laser ceilometer, and a profiling multi-channel microwave radiometer for the diagnosis of terminal area in-flight icing hazards with high spatial and temporal resolution. NCAR’s Current Icing Product (CIP) combines several meteorological inputs to produce a gridded, three-dimensional depiction of icing severity on an hourly basis. Pilot reports are the best and only source of information on in-situ icing conditions encountered by an aircraft. The goal of this analysis was to ascertain how the testbed NIRSS icing severity product and the operational CIP severity product compare to pilot reports of icing severity, and how NIRSS and CIP compare to each other. This study revealed that the icing severity product from the ground-based NASA testbed system compared very favorably with the operational model-based product and pilot reported in-situ icing.
15. SUBJECT TERMS Flight safety; Aircraft icing; Remote sensing; Numerical weather forecasting 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF RESPONSIBLE PERSON ABSTRACT OF STI Help Desk (email:help@sti.nasa.gov) PAGES a. REPORT b. ABSTRACT c. THIS 19b. TELEPHONE NUMBER (include area code) 12 PAGE UU U U 443-757-5802 U Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std. Z39-18