Uncertainty Quantification of a Rotorcraft Conceptual Sizing Toolsuite

Uncertainty Quantification of a Rotorcraft Conceptual

Sizing Toolsuite

1 2 3 Manas Khurana , Carl Russell , and Robert Scott Science & Technology Corporation, NASA Ames Research Center, Moffett Field, CA NASA Ames Research Center, Moffett Field, CA US Army Aviation Development Directorate, NASA Ames Research Center, Moffett Field, CA A computational framework to support the quantification of system uncertainties and sensitivities for rotorcraft applications is presented using the NASA Design and Analysis of Rotorcraft (NDARC) conceptual sizing tool. A 90 passenger conceptual tiltrotor con- figuration was used for case demonstration in the modeling of uncertainties in NDARCs emission module. A non-intrusive forward propagation uncertainty quantification approach was applied to ensemble simulations using a Monte Carlo methodology with stratified Latin hypercube sampling. An off-the-shelf software, DAKOTA, which supports trade studies and design space exploration, including optimization, surrogate modeling and uncertainty analysis, was used to address the research goals. Further, a toolsuite was developed incor- porating DAKOTA with automated design processes and methods using function wrappers to execute program routines including support for data post-processing. Uncertainties in rotorcraft emissions modeling using the Average Temperature Response metric for a set mission profile were studied. NDARC under-estimates the effects of emissions when us- ing the baseline modeling parameters for the Average Temperature Response compared with mean results from Monte Carlo simulations. A global sensitivity analysis was further undertaken to quantify the contribution of the various emission species on output sensitiv- ity. The work demonstrates that the developed toolsuite is robust and could support the quantification of system uncertainties and sensitivities in future rotorcraft design efforts.

I. Introduction Engineering design has uncertainties that manifest from data processing and interpretation to system modeling and manufacturing and measuring of component characteristics. Uncertainties, whether random or systematic, originate because of the underlying randomness in physical systems, modeling idealizations, experimental variability, and measurement inaccuracy. The use of computer models to represent the physics of a complex system is an applied process in engineering design. By definition this process does not capture the finer details of the studied phenomena due to the underlying assumptions that are implied to simplify the computing process. Turbulence modeling in computational fluid dynamics (CFD) is an example of this process; Schaefer et al. quantified the uncertainty and sensitivity of commonly used turbulence models using Reynolds-Averaged Navier-Stokes (RANS) codes on airfoil drag coefficient and relevant flow physics at transonic Mach numbers. In computing simulations, uncertainty is introduced even for deterministic systems and in stochastic phenomena where model simplification assumptions are removed. The underlying uncertainties that exist in the system still cannot be neglected.

Eliminating uncertainty in a prediction is often impractical and the goal is to control and/or manage the uncertainty to an acceptable level so that critical design decisions can be made from the simulated data.

If the uncertainty is too high, sensitivity analysis can be undertaken to identify the largest contributors to uncertainty so research can be guided to reduce these parameter uncertainties. In CFD, solver uncertainties can be addressed by reducing discretization error, improving physics modeling, or by setting appropriate boundary conditions. Once the source and magnitude of the uncertainties in a complex system have been identified, then future solver runs can be setup efficiently. The designer is not required to perform multiple runs with different solver settings to establish solution trends. An example is the uncertainty in the setting of 1 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

freestream turbulence intensity where uncertainty bands can be attached to the integrated forces to quantify the expected data variances.

In aircraft conceptual design, simulation models are used to establish system physics to predict flight per- formance for concept development. The accuracy of the computer models is enhanced using experimental data for solver calibration and validation. Often the experimental data is limited to specific test conditions and/or the testing equipment has uncertain properties that compromise the accuracy of the recorded data.

Semmelmayer confirmed the uncertainties in surface pressure measurements obtained with static pressure taps used in wind tunnel experiments. Accordingly, caution must be exercised when calibrating high-fidelity solvers with experimental data that has underlying uncertainties. As a result, the data generated by deter- ministic models will not correlate with measurements from experimental test due to the randomness in the respective data sources. To address this challenge, the use of probability and statistical analysis methods is warranted to guide engineering design.

Historically assessment of system uncertainties was based on subjective experience-based evaluations of the simulation results. Although the approach has merits, a rigorous mathematical framework using a struc- tured uncertainty quantification (UQ) method is needed to establish the estimates of model uncertainties.

Hence, UQ is defined as the process that identifies, quantifies, and assesses the impact of uncertainty sources that are involved in the development and application of simulation models. It is a technique that is applied to quantitatively map the randomness of the input parameters on model outputs where the input uncertain- ties are expressed using interval bounds or a probability density function (PDF). Accordingly the variability, distributions, and statistics of code outputs, given the uncertainty in input parameters, are established.

Post-processing of UQ results permit the assessment of event occurrence by studying the mean or median of the system performances, including the range of possible outcomes, by the evaluation of the probability distributions that represent the likelihood of reaching a critical event. The identification and management of system uncertainties is critical in aerospace design efforts to justify early model-based design decisions and to further ensure the concepts developed will safely and efficiently operate to the set performance thresholds.

In this work, uncertainty analysis is undertaken for rotorcraft at conceptual design. The study is critical at early design phase to ensure the final design will safely meet user requirements. At conceptual design, there are system uncertainties which are typically neglected or are estimated based on the judgement of the engineer running the simulation. The primary challenge in performing UQ at early design is the inadequate representation of the underlying uncertainty. Use of low fidelity solvers integrated in a multidisciplinary design analysis environment, where each discipline has uncertainties collectively impact the sizing effort.

NDARC is affected by these attributes and results with performance uncertainties need to be quantified.

There has been much work in UQ across multiple disciplines including flight operations, computational 8–10 11 12 aerodynamics and aeroacoustics, structural analysis due to uncertainties in blade geometry and ther- 13 14 15–17 mal loads, aeroelasticity and in experimental procedures. Probabilistic analysis in UQ has also been 18 7 applied to aircraft life-cycle costs. Sankararaman and Daigle applied a UQ approach for flight trajectory prediction using aircraft operation and atmospheric information. A Monte Carlo (MC) approach coupled with global sensitivity analysis was used to transform input PDF to output trajectory distributions to model system uncertainties. In computational sciences, Salahudeen and Baeder propagated input uncertainties to study CFD freestream turbulence intensity on airfoil force coefficients using a MC approach. Peˇ cnik, Witteveen and Iaccarino studied laminar-to-turbulent transition uncertainties for transonic gas turbine compressors using a stochastic method and quantified the uncertainties in the applied transition model on boundary-layer development. Ronch et al. modeled the uncertainties in RANS solvers in the design of an unmanned combat aerial vehicle using a MC approach coupled with a surrogate model. West et al. further conducted an uncertainty analysis using a surrogate based non-intrusive CFD approach to predict low-boom characteristics of a supersonic aircraft with different wing configurations for flight certification prediction.

In rotorcraft aeromechanics, Siva, Murugan and Ganguli simulated helicopter structural and aerody- namic performances at hover, axial climb, and forward flight to then model the uncertainty effects of the aeroelastic input parameters on rotor thrust and power coefficients. A MC approach was applied for uncer- tainty propagation. D´ ıaz and Hern´ andez quantified the uncertainties in the thermal design of a skin panel 2 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

and wing box section due to the randomness of the input variables using a MC approach. In aeroelastic analysis, Murugan et al. investigated the spatially uncertain material properties on the aeroelastic response estimations of a composite helicopter rotor. A MC approach was used to confirm that spatial uncertainty has significant influence on vibratory hub load prediction.

The simplicity of the non-intrusive approach makes it a widely applied method for uncertainty propa- gation using MC simulations. The primary objective of this work is to apply this technique with a robust computational framework to support the quantification of system uncertainties and sensitivities. Capabil- ity demonstration is presented using a 90 passenger tiltrotor that was conceptually sized by NASA for reduced environmental impact in multiple iterations of varying technology and requirements. A tiltrotor configuration is proposed as a future replacement for regional jets to alleviate airspace system delays, and since aviation accounts for 3-5% of man-made radiative forcing, environmental performance metrics were used to drive the design effort with the goal of following up with advanced concepts through detailed studies.

20, 21 Uncertainties in aircraft emissions are considered, based on the work explored in the the literature.

The environmental impact of aviation is important to the industry given that the demand in aircraft op- erations is forecast to grow significantly. The result of aircraft operations on climate change has been 22–25 documented as future aircraft configurations are developed, their impact on the environment is of inter- est. In 2005 alone, aircraft operations generated 4.9% of the worldwide anthropogenic forcing that resulted in climate change, and this trend is expected to increase. Computational simulations are critical for the modeling and coupling of atmospheric processes with aircraft gas emissions as a direct result of fuel combus- tion. This sequence is inherently uncertain due to the variability in the processes that generate emissions, and include variations in environmental conditions related to temperature and sunlight intensity changes.

Modeling assumptions based on limited supporting data and knowledge also give rise to uncertainties which then produce a low-fidelity estimation of the emission metric for a highly complex process. The instruments, tools and methods applied to measure emissions in experimental settings further contribute to system un- certainties.

There is a need to integrate validated computational tools and methods early in aircraft conceptual design for emission metric estimation. Allaire et al. suggests this would then allow for a meaningful approach to compare the climate impacts of disparate platform configurations and technologies in future aircraft design efforts under different operational, policy and market scenarios. In their work, the uncertainties in the U.S.

Federal Aviation Administration supported Aviation Environmental Design Tool (AEDT) were mapped by simulating aircraft operations originating from three major hub airports as a driver to then support intel- ligent decision and policy-making for aviation environmental impact. Lim et al. also used the AEDT to perform parametric uncertainty and sensitivity analysis to identify the main contributors to system output uncertainties for fuel burn, emissions, and noise for fixed-wing, single-aisle Boeing and Airbus category air- craft. In the report by Radian Corporation, it is recommended that the initial step towards enhancing emission estimates is to first understand and quantify the various sources of variability and inaccuracies in the data.

In this work, a parametric uncertainty and sensitivity study will be undertaken that quantifies the im- pact of the flown mission for the conceived platform on Average Temperature Response (ATR), measured in units of temperature using NDARC. The output response quantifies the lifetime climate temperature change due to flight operation by the considered design and mission. The climate model used was developed by Dallara, Kroo and Waitz which computes the emission species including carbon dioxide ( CO ), nitrogen oxide ( N O ), water vapor ( H O ), sulfate ( SO ), and soot from fuel combustion. The species affect both x 2 4 atmospheric composition and cloud formation properties, hence the radiative balance of the Earth system.

System uncertainties will be first quantified by probabilistic means to form the robustness (mean and vari- ance) of the output metric. The methodology will propagate the randomness of the input coefficients that are used to establish the emission species through the simulation driver to then compute the climate response metric, ATR with uncertainty. A global sensitivity analysis (GSA) will then be undertaken to assess the influence of parameter inputs on output variability using large-scale MC simulations. The results to the parametric analysis will guide designers to the expected uncertainties in ATR due to the randomness in climate modeling input parameters.

3 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

The paper is organized as follows: Section II outlines the emission modeling framework; Section III details the computational tools that are used and developed to address the research goals; Section IV describes the applied UQ and sensitivity analysis methodologies; Section V presents the numerical results with discussion; and Section VI summarizes work findings with outline of ongoing research efforts.

II. Aviation Emissions The rotorcraft emissions modeling methodology for this work follows that of Russell and Johnson, which is a slightly modified version of the method developed by Dallara, Kroo, and Waitz. A summary of the emissions modeling is given here, while a more comprehensive description of the methods is contained in the above two references. The metric that was used to quantify the environmental impact of a given rotorcraft design is ATR.

ATR is a measure of the atmospheric temperature change caused by operation of a fleet of aircraft (fixed- or rotary-wing) for a given amount of time, H , typically 30 years. The most basic form is given in Eq.

1: ∫ ∞ AT R = ∆ T ( t ) w ( t ) dt (1) sust,H H ∆ T , is the time-dependent global mean temperature change resulting from the operation of a given sust,H aircraft design, where emissions are assumed to be constant for the first H years and zero thereafter. This temperature change is a function of the solar energy trapped in the atmosphere by a number of different pollutant species. The weighting function w ( t ) allows discounting of temperature change effects in the years following H . This discounting is included so that long-term effects such as CO warming do not necessarily dominate ATR. When considering the impact of aviation on climate response, it may be desirable to focus more on short-term effects (decades), rather than long-term effects (centuries).

In general, the most important pollutant species when considering climatic temperature change due to hydrocarbon (e.g., jet fuel) combustion are CO , oxides of nitrogen, including N O and N O , collectively 2 2 called N O , and aviation-induced cloudiness, or AIC. Other combustion emissions such as soot and sulfates x also contribute to ATR, but their impacts are about an order of magnitude less than those of CO , N O , 2 x and AIC. The first step in calculating ATR is determining the quantity of emissions. For all emission species except N O , this calculation is trivial as it can be well approximated as a constant multiple of the quantity x of fuel burned.

In reference to N O , the emissions quantification is much more involved and uncertain as it depends x significantly on internal engine properties including atmospheric conditions. In fact, many modern engines derived from older models generate significantly more N O emissions per unit of fuel burned than their x older counterparts. This issue is caused by the pursuit of higher engine fuel efficiency, which leads to higher 29 28 engine pressure ratios and combustion temperatures. For this study, the method of Russell and Johnson was used to estimate N O emissions, whereby in-flight turboshaft engine N O emissions are approximated x x 30, 31 using published turbofan emissions data along with the DLR fuel flow method.

There are multiple climate models that can be used to calculate the different components of ATR, but simple linear climate models are appropriate for the conceptual design of rotorcraft. The models used for this work are the same as those used by Dallara, Kroo, and Waitz. These models first calculate the radia- tive forcing (RF), measured in watts per meters squared per kilogram, that represents the amount of heat trapped by each particular emissions species. Radiative forcing is a function of time that decays exponen- tially depending on how long that particular emission species persists in the atmosphere. CO emissions can persist in the atmosphere for centuries, while many of the others may persist no more than a year.

Climate effects due to CO emissions are generally well understood by the scientific community; in short, increased atmospheric CO concentration causes positive radiative forcing, leading to higher global tempera- tures. N O emissions are less well understood and cause changes in RF indirectly through multiple chemical x processes in the atmosphere. Increases in N O lead to both increases in atmospheric ozone (a warming x 4 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

effect), and reductions in methane (a cooling effect). The methane reduction has a secondary effect of reducing ozone, so there are actually three components of RF due to N O emissions. All three are modeled x in the current study.

In the analysis of Aviation Induced Cloudiness, the RF contributions have a much higher level of un- certainty than do the other emission species. At low altitudes or areas of the atmosphere with minimal cirrus cloud potential, the RF due to AIC is negligible. AIC effects peak at altitudes near 35,000 ft, but are still dependent on local atmospheric temperature and moisture content. Because of the high level of uncertainty in the impacts of AIC, and the fact that future aircraft routing may seek to actively avoid contrail generation, it is sometimes desirable to ignore AIC in the ATR calculations.

For each emission species, there are several parameters that are used to calculate the time-varying ra- diative forcing caused by their deposition in the atmosphere. For the long-lived species ( CO , plus methane and ozone changes caused by N O ), exponential functions give the time-dependent decay of RF, with the x RF extending well beyond the operation time of the aircraft. For the short-lived emission species and AIC, RF is assumed to be a constant function of fuel burn during aircraft operations and zero thereafter. For the products of atmospheric N O deposition and AIC, an additional forcing factor is applied that is dependent x on altitude. This altitude-dependent forcing factor is assumed to be constant below 16,000 ft. Parameters are then applied to normalize the effects of RF for the different emission species based on current global distribution of aircraft traffic.

Finally, once the radiative forcing for each emission species is calculated, the total RF is determined and the temperature change due to sustained aircraft operations, ∆ T is computed. ATR is calculated by sust,H integrating ∆ T over time and normalizing by the time period of interest, H . Because ATR has values sust,H on the order of nano degrees, when comparing aircraft designs it is useful to normalize the ATR values by that of a baseline. The baseline aircraft then has a relative ATR value of 1, and the off-baseline designs have relative ATR values that are easy to interpret.

The complete equations for the linear climate models are given by Dallara, Kroo, and Waitz, as are the probability distribution functions for the modeling parameters and coefficients to the equations. The parameters which are the inputs to the uncertainty analysis are briefly summarized in Table 1 for the respective species: 5 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

Table 1. Emission species considered in the uncertainty and sensitivity analysis Emitted Species RF Parameters Description A CO Parameters and time constants used by the exponential decay CO α 2 c 1 , 2 , 3 model for RF caused by CO effects τ c 1 , 2 , 3 A CH 4 Parameters and time constant used by the exponential decay CH model, including ratio f , that governs the effectiveness at τ 4 n changing global temperatures by RF for C effects H 4 f CH A 0 Parameters and time constant used by the exponential decay 3 L O model, including ratio f , that governs the effectiveness at τ 3 L n changing global temperatures by RF for long-lived N O effects x f O ( RF /E ) ratios that govern the effectiveness at changing global temperatures ref ref O 3 s O 3 S by radiative forcing caused by short-lived N O effects f x O ( RF /E ) ref ref H O ratios that govern the effectiveness at changing global temperatures H O by radiative forcing caused by short-lived water vapor effects f H O ( RF /E ) ref ref soot ratios that govern the effectiveness at changing global soot temperatures by radiative forcing caused by short-lived soot effects f soot ( RF /E ) ratios that govern the effectiveness at changing global ref ref SO SO temperatures by radiative forcing caused by short-lived SO effects f SO RF 2 xCO S Parameters and time constants used by the exponential decay ∆ T model for temperature change caused by radiative forcing α t τ t 1 , 2 A stochastic model is setup that involves the random sampling of the above inputs from the respective PDF defined by Dallara, Kroo and Waitz. The data points are propagated to the climate model to establish output statistics. The distributions of potential outcomes due to random input variation is derived using a large simulation sample size (stochastic projections) to ensure output statistical convergence is established, hence leading to the quantification of ATR uncertainty for the flown mission.

III. Computational Toolsuite The UQ processes are adaptable to automation and facilitate the assessment of uncertainties in engineer- ing design. The sub-systems of the proposed toolsuite are described in this section.

A. Overview of Computational Toolsuite In this paper, MC simulations with parameter distribution by the Latin hypercube sampling (LHS) approach is performed using the Design Analysis Kit for Optimization and Terascale Applications (DAKOTA) software package. The DAKOTA executable is available pre-compiled via the Sandia National Laboratory website and provides an interface to black-box programs, in this case coupling with NDARC. In the UQ realm, the software supports the generation of probability distributions based on input parameters, followed by the forward propagation of the select distributions through a mathematical model. Outputs from DAKOTA are then processed to generate variable input and response output probability distributions leading to the quantification of parameter uncertainties and sensitivities. The computational framework uses file-based in- puts and outputs, and function wrappers are developed to automate simulation pre and data post-processing routines. Figure 1 illustrates this computational process.

6 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

>> Define Study Parameters DAKOTA Input File ● Environment ● Method (UQ / SA) ● Variables ● Interface ● Responses Output: Results Processing DAKOTA Execution ● Raw data (x- and f-values) ➢ Regression analysis DAKOTA Variables File ➢ Statistics on f-values (UQ) {x1 = 12.5} ● DAKOTA results files (.dat, .out) {x2 = 135.5}, etc. ➢ Sensitivity Analysis Data Data Pre-processing Post-processing NDARC: Mission Simulation Emission Driver ● Segment Altitudes ● Radiative Forcing ● Fuel Flow ● Temperature Response (thrust & power settings) Input: to Climate Model ● DAKOTA variables ● NDARC outputs Figure 1. EmissionWrapper : A computational framework used to model system uncertainties and sensitivities in rotor- craft emissions modeling with NDARC As first step, the user enters study mode in the form of uncertainty or sensitivity analysis. Based on this input, a template of the DAKOTA input file is formed that further requires the definition of the following categories: • environment: is an optional requirement that is used to define the precision of the results, including the names of the output and error files.

• method: specifies the study algorithm that is to be initiated on the model for the chosen analysis. Some examples include multidimensional parameter analysis; design space sampling; sensitivity analysis using local or global methods; or optimization using gradient-based methods. The input files requires the definition of at least one method block.

• variables: to characterize the number, descriptor (label), type, and ranges of parameters that will be varied by DAKOTA. Variable types include state, continuous or discrete, real, integer, or string.

For uncertainty analysis, parameters of the probability density function need to be defined.

• interface: defines the workflow for the mapping of the input variables to a solver job file to then obtain and store responses from the solver at those values to a results file. The solver path must be specified and a script is further used to orchestrate the routines in this sub-task.

• responses: the quantities of interest (QoI) from the model are stored in an output file and shared with DAKOTA to complete the computational cycle.

A Matlab-based function denoted as the EmissionWrapper is used to execute the computational routines in Figure 1. The wrapper generates, executes and manages the DAKOTA and solver climate modeling job 7 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

input files to completion. It further activates results post-processing scripts to support statistical analysis and variable sensitivity studies. At function execution (Fig. 1), the following events follow in sequence and are repeated for each cruise altitude run: (1) User to enter study type including sensitivity or uncertainty analysis, variable descriptors, probability distribution types with supporting parameters, sample size, and size for incremental LHS runs; (2) A DAKOTA input file is generated using the entered input data; (3) DAKOTA is executed in pre-processing mode to generate and store study input parameters, (emission species coefficients) in text file format; (4) The rotorcraft mission profile (shown in Figure 6) is simulated using NDARC. Performance metrics that are needed to compute ATR are then extracted from the NDARC output file. A parametric cruise altitude sweep is performed and at each run, mission data from NDARC is extracted to then facilitate the quantification of system uncertainties and sensitivities for cruise altitudes in the range of 16,000 to 33,000 feet.

(5) The distribution of the input parameters from (3) and mission-system output metrics from NDARC (4) are saved and parsed to; (6) An emission driver which then computes the radiative forcing, RF, and decay in RF for the respective emission species, hence leading to the measure of ATR; and (7) Data post-processing routines follow to perform statistical (UQ) and sensitivity analysis.

The simulations and supporting data are processed in serial on a local machine configured with an Intel Xeon CPU at 2.60GHz. The workflow guides the generation of individual case folders where the respective DAKOTA input and climate model functions reside and are used to execute the runs. DAKOTA outputs are directed to a results directory that resides one level up to the code execution level. Each simulation requires approximately three seconds of computing time, and at each run end the response metric is appended to a text file that is also transferred to the results directory. At simulation end, the workflow execution case folder is removed for efficient data management and the simulation then cycles to the next design point. The routine is adaptable for analysis with parallel computing and this capability will be introduced in follow-up work.

B. NDARC and Climate Modeling Framework NDARC is a conceptual/preliminary design and analysis code for rapidly sizing and conducting performance 37–39 analysis of new rotorcraft concepts. NDARC has a modular code base, facilitating its extension to new concepts and the implementation of new computational procedures. By using simplified models to represent major rotorcraft subsystems, such as engines and rotors, NDARC can produce multiple rotorcraft designs quickly without requiring repeated runs of time-intensive engine cycle or rotor performance analysis.

A typical NDARC run consists of a sizing task, which can be followed by off-design performance analysis.

During the sizing process, mission performance is calculated and the aircraft is resized both geometrically and mechanically until convergence criteria are met. The software uses reduced-order performance models for various rotorcraft subsystems, such as rotors and engines, in order to facilitate short runtimes. These models require curve-fits to higher-fidelity models or experimental data in order to capture rotorcraft per- formance.

The outputs of the NDARC engine model are used extensively for this study. NDARC implements the Referred Parameter Turboshaft Engine Model (RPTEM), described in the work by Chen et al., to calculate parameters such as fuel flow and power variation with altitude. This model is based on curve fits of engine performance data for both existing and projected engines. These curve fits can be scaled to the required engine size to represent a notional engine. NDARC further generates hundreds of output parameters for a given sizing study, and several of these are required from NDARC output files to compute ATR and include: • Mission segment length; 8 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

• Total fuel burned per mission segment; • Mission segment altitude; • Mach number; • φ , temperature correction factor – total temperature divided by sea-level-static temperature; • δ , pressure correction factor – total pressure divided by sea-level-static pressure; • Fuel flow per engine; and • Engine maximum rated power (sea-level-static) The work by Russell and Johnson involved the calculation of ATR using Matlab scripts to post-process data from NDARC output files. Subsequent to that work, ATR calculations were included into the func- tionality of NDARC and have been available as an output since version 1.10 of the software. The sizing and design tasks for the current study were carried out using version 1.12 of NDARC. To facilitate the computation of ATR, NDARC is pre-compiled with “best estimate” values of the RF coefficients needed to establish the respective emission species from Table 1. Matlab functions were further developed that use NDARC outputs to then facilitate the manipulation of the RF input coefficients for uncertainty propagation and quantification of ATR.

The computational process outlined is further supported with design contingencies to handle failed and/or non-converged solver runs in both NDARC and in the Matlab climate modeling framework. As first step, NDARC runs for the varying cruise altitudes are executed and required outputs are stored. Input argu- ments have been defined in EmissionWrapper that iteratively modify airframe sizing tolerances in NDARC job file if solver non-convergence is detected. In the analysis to follow, all NDARC altitude sweep runs achieved convergence. For ATR calculations, measures have also been imposed to ensure the combination of climate modeling input parameters and data from processed NDARC runs achieve convergence. Routines are integrated that scan and remove failed data points so that the integrity of the uncertainty calculations is preserved. The simulations that follow represent a 100% convergence rate so that the MC population size is maintained.

IV. Uncertainty Quantification for Rotorcraft Emissions Modeling The forward propagation UQ approach is applied where the probabilistic information on parametric inputs are mapped through the computational model to then assess output statistics. The methodology consists of the following tasks: A. Types and Sources of Uncertainties Aleatory uncertainties are considered in this work due to the inherent random variations that can take place in the emission input parameters in Table 1. Accordingly the stochastic uncertainties are characterized by a PDF that quantifies the probability density at any value over the range of the random variable. With a large sampling size of the design space in a MC setting, the topology of the PDF is established and this approach is applied in the analysis to follow.

The climate model used provides an approximate representation of the numerical solution and the char- acterization of the numerical approximation errors are not factored. The model form uncertainties that Roy and Oberkampf define as the statistical quantification of the disagreement between simulation results and experimental measurements were also not considered. These involve the assessment of model validation and accuracy by the comparison of simulation results with experimental data, including the extrapolation of er- rors from the modeled test domain to conditions where experimental data is not available. As experimental readings of ATR for the considered rotorcraft (Fig. 5) and mission envelope (Fig. 6) are not available, a direct comparison of computational ATR result with experimental readings is not possible, hence the relating uncertainties cannot not be factored.

9 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

In this work, the uncertainties considered are variations in model inputs and relate to the design coef- ficients that are used to establish ATR due to the RF from the respective emission species (Tab. 1). The initial model inputs or “best estimate” values with respective PDFs are based on data collected by environ- mental and aviation agencies through a combination of computational and experimental simulations with measurements, theory and expert opinions.

B. Uncertainty propagation using Monte Carlo Simulations Uncertainty propagation is a process that is used to define the influence of uncertainties in model inputs on response output. The methodology is used to calculate the uncertainty statistics of the output quantity of interest using a numerical method. In a generalized form, an input parameter, X , is passed through a function evaluator, f , which then establishes a measured output, Y , such that: Y = f ( X ) (2) Where f represents the system model (for example a sizing code in this case); X is a vector of uncertain input variables; and Y represents a vector of estimated outputs.

The uncertainty propagation process is described in Figure 2. Here the uncertainties in the input variables are represented probabilistically where each component of the input parameter X is a random variable.

Accordingly the system output will also be a random variable, and this randomness is described using PDFs.

A MC approach is used to propagate the input uncertainties with a stratified sampling approach that selects data points from the input distributions for analysis by the model. Specifically the randomness of the input data points is propagated to the solver using a PDF; then the numerical solver is executed at these points; and the resulting PDF of the outputs subject to the defined input distributions is established. As post-processing, the output distributions are compared with performance criteria to determine, for example the likelihood that the system will operate within the specified thresholds. The output statistics including expected value and variance are also established including the quantification of system sensitivities on response outputs.

INPUT PARAMETER UNCERTAINTY ANALYSIS OUTPUT RESPONSE X LATIN HYPERCUBE 1 Y SAMPLING Y output X input n n INPUT PARAMETER results parameters 2 Function X Evaluation 1 X X X SENSITIVITY ANALYSIS INPUT PARAMETER X X X X Figure 2. Uncertainty propagation approach using Monte Carlo simulations The following subsections will further outline these steps.

10 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

C. Analysis setup The MC approach applied is in the following form: ( 1) The inputs are identified by vector X with n parameters. Each variable in the parameter space X requires the definition of a probability distribution and the respective properties of that distribution (means and standard deviation for a Gaussian distribution). When the data is limited, the distributions are often estimated based on engineering judgement. If empirical statistical data is available, then parametric fitting of known distributions are taken, or non-parameter density estimation techniques are used.

( 2) A sample population from the input distribution for the n variables is generated in the form: X = [ x , x , ..., x ] (3) j j 1 j 2 jn To ensure the sample points provide an acceptable coverage of the parameter space, the LHS approach is used. The methodology provides a stratified sampling technique where the range of each uncertain variable is divided into N s segments of equal probability, where N s is the number of user-requested samples. The relative lengths of the segments is determined by the specified probability distribution.

A uniform distribution will have segments of equal width while samples from a normal distribution will have small segments near the mean and larger segments at the tails. Given the probability distribution of each variable is independently decomposed into N s equally divided sub-domains, a sample value is then selected from each sub-domain. Consider a sample of size N s of x , x , ..., x , where the N s j 1 j 2 jn values for x are paired at random without replacement with the N s values for x . These are then j 1 j 2 paired with N s values for x until all variables are paired. Accordingly the interval of each variable j 3 is used only once and each one-dimensional projection of the j -dimensional sample is a representation of the distribution of Xi . The LHS approach further allows for the definition of a arbitrary number of samples where N s will then match the partition size that will be used for each uncertain variable.

Figure 3 outlines this method where N s = 5. Figure 3(a) shows the sampling intervals that have large segments at the tails when the parameter is defined using a normal distribution; Figure 3(b) shows the equal sample segments when drawn from a uniform distribution; and Figure 3(c) shows the combined formation of the samples. It is noted that every row and column in the hypercube of partitions has exactly one sample. In this process, LHS improves the statistical convergence properties of a MC simulation and reduces the required number of forward calculations that are needed due to the extended parameter space coverage.

11 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

(a) Normal distribution (b) Uniform distribution (c) Two-dimensional LHS for parameters with normal and uniform distributions Figure 3. Representation of LHS probable intervals for parameter input space sampling ( 3) The output ATR response is established by the climate model using the values distributed from the input parameter set X .

j ( 4) Steps (2) and (3) are repeated so that a distribution of the output metric is formed. The sample size, N s , needs to be adequate to ensure convergence of the output PDF and related statistics that is to include expectation, μ , and variance, σ as: N ∑ μ = g ( X ) (4) j N j =1 N ∑ 2 2 σ = ( g ( X ) − μ ) (5) j N j =1 The dispersion of the output values are used to evaluate both the estimate (Eq. 4) and the uncertainty (Eq. 5) of the output quantity. The latter is taken as being equal to the standard deviation of the computed data set.

( 5) Results post-processing follows to formalize data sensitivity and uncertainty analysis.

In the analysis to follow, the emissions modeling coefficients are designated as uncertain input parameters. As the magnitude of the respective parameters is not known exactly, engineering estimates of their distributions are used from the literature. A probabilistic approach to represent model parameters that impact the computation of the average temperature change due to the operation of a particular aircraft is a valid approach. This is due to the variation in N O and AIC that can generally be bounded with reasonable x accuracy as a function of cruise altitude. This pattern is further valid for other attributing climate coefficients where linear climate models exist that can be used to coarsely classify the change in endpoint temperatures as a function of altitude.

12 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

D. Uncertainty Management with Sensitivity Analysis Management of uncertainties is critical on two fronts. First the process establishes whether the uncertainties in system output can be reduced, and secondly it allows for the setting of rules and regulations that define how the quantified uncertainties will aid decision-making. Accordingly the following questions evolve: “What if the uncertainty in ATR is too high?” “Is it possible to minimize the uncertainty estimates?”. In this work, the first phase in uncertainty management is addressed by a sensitivity analysis (SA) approach.

D.1. Global Sensitivity Analysis SA allocates the variation in model output, qualitatively and/or quantitatively due to the changes in the input parameter. Alternately UQ is applied to establish data on model output using statistical properties that include confidence intervals (CI) to represent bounds on responses. Apportion of total uncertainty to individual inputs is established with SA by ranking the parameters by their respective level-of-importance on responses. Hence, SA is the next logical study following the quantification of output uncertainties.

Further in SA, if it is established that the change in response output is minimal then it can be concluded that the output is robust to changes in parameter values in the model. It is then reasonable to direct that the uncertainty about the value is relatively small. If the response output alters significantly due to variable changes, this indicates that there is large uncertainty about the parameters value. Uusitalo et al. explain that it would be unrealistic to assume that the values used in the model would match those that take place in nature, and if small differences in these values result in large differences in the outcome, the outcome is expected to be uncertain. A parameter can independently impact response output or through interactions with other variables. Hence, best SA practices govern that the parameters must be varied not just indepen- dently but also in combination with other variables so that non-linear interaction effects can be factored.

This then leads to the grouping of SA into local and global methods.

In local SA, the partial derivatives of the output are established with respect to parameters at a nominal value in the factor space in Figure 4. Typically numerical finite-differences or retrieval of analytic gradients are established and the local derivatives are then used to rank parameter influences. The main limitation of this approach is that it evaluates factors one-at-a-time, and does not allow for the evaluation of simultaneous changes in all model parameters. In the quantification of ATR with respect to RF caused by the considered emission species, the interactions between the studied parameters will be important and needs to be factored.

Further the local SA approach is limited to the immediate region around the nominal parameter value unless a prohibitively large number of different starting points are selected for perturbation in the hyperspace of the input variables. In the case of emission modeling, input values are uncertain and encompass large ranges which can not be efficiently investigated with local techniques.

13 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

∣ ∣

∣ ∣

∂Y

∣ ∣

∂X i x Local derivative of output Y with respect to input X at point x i Figure 4. Graphical representation of local sensitivity analysis To address these limitations, global sensitivity analysis (GSA) by the variance-based decomposition (VBD) Sobol method is used. This approach quantifies the variance contribution of the input parameters to the unconditional variance of model output. This is important in determining input parameters with the greatest influence for parameter prioritization, and input parameters with the least influence for parameter fixing. The Sobol decomposition further models the interactions of input variables through the second and higher order terms, while local sensitivity methods provide only the derivatives with respect to single variables. Consider a model, f , with output, ( Y ), that is a function of input parameters ( X , X , X , ..., X ) 1 2 3 q as: Y = f ( X , X , X , ..., X ) (6) 1 2 3 q The inputs ( X , X , ..., X ) in model (6) are ranked as a function of their respective influence on f based on 1 2 q a set criteria. Here the VBD approach is applied that is based on the framework that the variance, V , of model output, Y , is decomposed for any input, X , from the law of total variance as: V ( Y ) = E [ V ( Y | X )] + V ( E [ Y | X ]) (7) i i Where E [ . ] and V [ . ] represent the expected value and variance, respectively; E [ Y | X ] and V [ Y | X ] denote i i the conditional expected value and conditional variance, respectively of output Y for inputs X . The partial i variances represent the uncertainty of each independent input or through interactions with other parameters.

Further this decomposition states that the total variance of Y , denoted as V [ Y ], is the sum of two terms: one that measures the variance “between” the conditional means, and the other that assesses the mean of the conditional (“within”) variances.

As a measure of sensitivity, two indices are considered: (a) the main effect sensitivity index, S ; and i (b) the total sensitivity index, T SI . The main effect sensitivity index relates to the fraction of the total i uncertainty in the output, Y , that is attributed to input X alone. The sensitivity index compares the i variance of the conditional expectation V [ E ( Y | X )] against the total variance V ( Y ). It further represents X i i the expected reduction in output variance that would result if a given input parameter was to be known precisely. The total effect sensitivity index ( T SI ) corresponds to the fraction of the total uncertainty in the output, Y , that is attributed to input X and the relating interactions with other variables (denoted i by X ). This models the expected amount of output variance that is attached to a given input factor and − i all interactions in which the factor is directly involved.

Mathematically S and T SI sensitivity indices are presented in the following equations where the partial i i 14 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

variance is divided by the model total variance, V ar ( Y ), to establish a normalized index of importance.

V ar [ E ( Y | X )] X i i S = (8) i V ar ( Y ) and E [ V ar ( Y | X )] V ar ( Y ) − V ar [ E ( Y | X )] X − i X − i − i − i T SI = = (9) i V ar ( Y ) V ar ( Y ) The calculation of S and T SI from the approximation of model behavior requires the evaluation of q - i i dimensional integrals using MC sampling. In this work, DAKOTA is used for this task and a detail overview of this process is outlined in the work by Bielen. Post-processing of GSA results will facilitate a targeted approach for the refinement of low-fidelity emission modeling methods to reduce output uncertainties in ATR estimation due to input randomness. The variance-based method has been listed in the U.S. Environmental Protection Agencys (EPA 2009) list of attributes as the preferred sensitivity analysis method, and it is claimed to be robust and independent, irrespective of model assumptions.

V. Numerical Results The propagation of uncertainties and respective uncertainties of the emission modeling parameters from Table 1 on climate model output, ATR are presented. The helicopter in consideration is a 90 passenger configuration in Figure 5 that is conceptualized for a mission profile in Figure 6.

Figure 5. Conceptual design of a tiltrotor platform configured for 90 passengers payload The segments of the mission are explained below and is typical of the mission that has been used for multiple previous NASA design studies on tiltrotor aircraft.

Altitude (ft) Cruise at V Cruise at V (reserve) br br Cruise Altitude Climb at minimum power margin 5,000 30 min loiter at V (reserve) br (ISA +20°C) 3 min taxi, 2 min HOGE 1 min HOGE Range 0 1000 1100 (nm) Figure 6. Rotorcraft mission profile used for the quantification of emission uncertainties and parameter sensitivities 15 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

◦ • Segment 1: 3-minute taxi at 5,000 ft, standard atmosphere +20 C ◦ • Segment 2: 2-minute hover out of ground effect (HOGE) at 5,000 ft, standard atmosphere +20 C • Segment 3: Climb to cruise altitude at maximum continuous power, standard atmosphere at segment midpoint • Segment 4: Cruise at constant altitude at speed for best range ( V ), standard atmosphere; distance br of Segments 3 and 4 add to 1,000 nm • Segment 5: Reserve segment - 100 nm cruise at V and cruise altitude, standard atmosphere br ◦ • Segment 6: Reserve segment - 30 minute loiter at V and 5,000 ft, standard atmosphere +20 C br ◦ • Segment 7: 1-minute hover out of ground effect at 5,000 ft, standard atmosphere +20 C In the ATR calculations, a sweep in cruise altitude for segments 4 and 5 is undertaken for low altitude flight at 16,000 feet to a ceiling of 33,000 feet. It is also assumed that both of the reserve segments are flown in their entirety. This is not realistic but with a goal of computing the ATR, the assumption is not problematic as long as it is applied consistently across all of the designs in the trade study.

A. Uncertainty Propagation A.1. Assessment of Statistical Convergence Monte Carlo simulations from the framework in Figure 2 are executed for varying cruise altitudes. The input parameters in Table 1 are sampled from the probability distributions defined by Dallara et al. As a set or ensemble of calculations are performed to map the uncertain input space to the uncertain output space, large variations in system outputs are expected due to the nonlinear interactions between the input parameters.

Accordingly the assessment of statistical convergence of output ATR is to be established from the mean and standard deviation of the observations as a function of MC simulation sample size, n . A converged state is required to ensure a valid interpretation of system uncertainties can be established from output statistics.

Incremental LHS is activated for sample space adaptation that involves the introduction of new points at set intervals. This ensures that the stratification and correlation structure of the original LHS sample is maintained and the computational load is limited to function evaluations at the newly formed samples only. The convergence of ATR observation mean and standard deviation as a function of cruise altitude from 16,000 to 22,000 feet is presented in Figures 7(a) – 7(b) respectively.

16,000 ft 16,000 ft Alt.

22,000 ft 22,000 ft (a) Mean (b) Standard Deviation Figure 7. ATR statistical convergence from the Monte Carlo runs 16 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

In the work by Porter, Avramova and Ivanov, statistical convergence is assumed when the difference between the respective parameters at successive iterations is less than 1%. Figure 7 confirms the statistical parameters, as a function of MC simulation sample size, n , are in steady-state for the presented cruise al- titudes. Changes in mean ATR (Fig. 7(a)) are minimal with an increase in n and convergence at the 1% threshold for successive iterations is rapidly achieved. Relatively there are active variances in ATR observa- tion standard deviation (Fig. 7(b)) and 1% threshold convergence is achieved closer to the maximum tested sample size of 102,400.

The impact of n on statistical convergence is further represented by the output PDF of ATR at 16,000 ft in Figure 8.

− 07 − 08 n = 400; μ = 1 . 064 × 10 ; σ = 7 . 442 × 10 − 07 − 08 n = 6400; μ = 1 . 076 × 10 ; σ = 7 . 756 × 10 − 07 − 08 n = 102 , 400; μ = 1 . 075 × 10 ; σ = 7 . 682 × 10 Average Temperature Response Average Temperature Response Figure 8. Representation of ATR output distribution for Monte Carlo simulations with sample n at 16,000 ft A Chi-square goodness-of-fit test rejected the null hypothesis at the 5% significance level that the output ATR is from a normal distribution. Hence, a kernel density estimator was used to establish ATR output probability distribution in Figure 8. Here an increase in sample size causes the mode of the data (most frequent value in the sample set) to increase, yet changes to observation mean and standard deviation are minimal between 6400 and 102,400 sample size. An increase in sample size further causes the tails of the distributions, in figure inset, to become smooth. This is expected given that the Kernel density estimator uses a data smoothing algorithm where inferences about the population are made based on a finite data sample. As n rises, additional data points are introduced at the tails which then produce a smoother density estimator. This is validated in output histograms in Figure 9 for cruise altitude at 16,000 feet and this pattern is consistent for all cruise altitudes modeled in this analysis.

Kernel Density Estimation Kernel Density Estimation Frequency Frequency Average Temperature Response Average Temperature Response Estimation Estimation (a) n = 6400 (b) n = 102 , 400 Figure 9. Tail distributions of the Kernel density estimator of output ATR for Monte Carlo simulations at 16,000 ft 17 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

A valid representation of output PDF leads to the formation of standard deviation, hence uncertainty in the observed data. Further when “best estimate” input parameters as defined by Dallara, Kroo and Waitz − 07 are used, ATR =1 . 070 × 10 which closely matches the mean measurement from the MC runs. Yet, 16 , 000 ft − 08 a sample observed uncertainty of σ = 7 . 682 × 10 is attached to this reading. In the UQ analysis that follows, statistical data will be representative of sample size n = 102 , 400.

A.2. Quantifying Uncertainties In the analysis of statistical convergence from the previous section, the data points collected were from one sample study which were then used to measure system uncertainty using the standard deviation. The statisti- cal analysis was based on a point estimate of the population mean which is limited because it only quantifies sample uncertainty and not that of the population. Accordingly quantification of model uncertainty will benefit from the technique of statistical inference where the statistical population will factor all possible ob- servations. As it is impractical to study the patterns from the entire population, a smaller subsample is used to then make inferences about the entire population using CI which can take any number of probabilities.

CI provide more information than point estimates where the interval represents a range of values that with specified certainty contains the true mean of the population. In this case, if q random samples with size n are taken from the population, the mean should fall between the calculated upper and lower ranges 95% or 99% of the times. Hence, CI provide a valid measure of uncertainty where a higher probability associated with the CI means that there is a greater degree of certainty that the parameter is within the bounds of the interval.

To establish the CI, a measure of the standard error of the subsample mean, σ , is first calculated.

SE M This is an estimate of the standard deviation of a large number of sample means drawn from the same population pool as: σ σ = √ (10) SE M n Where σ is the sample standard deviation and is a descriptive statistic that represents the amount of variation in the observations, while σ is an inferential statistic that provides an estimate of how far the population SE M mean is likely to be from the subsample mean. Using σ , the CI is established for any desired degree of SE M confidence, in this case at the 95% level by using the sample size, n , and variability, σ , of the sample. Hence, CI of the true population mean, μ , is given by: 95% CI = μ ± Z . σ SE M σ = μ ± Z . √ (11) n ︸ ︷︷ ︸ margin of error Where Z is the standardized score which represents the number of standard deviations an element is from the mean for a specific level-of-confidence. To establish Z , the Central Limit Theorem applies which states that for any distribution, if the sample size used to generate the data points is sufficiently large such that n > 30, then this will produce a normal distribution of the sample means. Larger sample sizes are also characterized with less spread of the distribution means, hence properties of a normal distribution curve are used to establish the Z – score. At 95% CI, Z in Equation 11 is 1.96 which implies that the population mean will be within 1.96 standard errors of the mean.

Further the second term in Equation 11 represents the margin of error (MoE) which quantifies the un- certainty associated with point estimate. As a representative sample from the population is used to estimate system performance, the sample mean is inferred to establish real statistic of the population. In this process, the estimated mean will be about the real mean with a “plus-or-minus” MoE. The quantification of the error magnitude facilitates confidence in the analysis a certain percentage of the time that the true population mean will be within the range above or below the sample estimate. From this point onwards, all reported uncertainties following the ± notation refer to the MoE or standard error.

In the MC simulations, statistical properties at n = 102 , 400 are recorded based on convergence analysis from Figure 7, hence as per the Central Limit Theorem, the output ATR will be normally distributed.

As confirmation, the Chi-square goodness-of-fit test was undertaken which presented ATR to be normally 18 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

distributed for the tested altitude ranges. Figure 10 shows the CI about the mean ATR performance as a function of data points, n , for a sample study at 16,000 feet cruise altitude.

Figure 10. Mean of ATR (solid line) with 95% confidence intervals (shaded) at altitude = 16,000 ft Equation 10 presented the standard error as inversely proportional to sample size. The larger the sample size, the smaller the standard error where the statistic converges to the true mean value. Figure 10 reinforces this pattern where the CI bands about the sample mean are large for small sizes of n . As n increases, the mean converges to true system performance and degree-of-uncertainty marked by the CI bands reduces. As the 95% CI defines a range of values that with 95% certainty contains the true population mean, an increase in sample size provides an enhanced estimate of performance mean with greater precision than with small samples, hence the CI bands become narrow. In this case, wide CI indicate reduce precision while narrow bands represent greater precision.

Altitude sweep ATR uncertainties are quantified by inferential statistics in Figure 11 by transposing sample statistics on population. The MoE from Equation 11 is used to quantify uncertainty at a 95% confidence level around the mean estimate at n = 102 , 400. The percent uncertainty is also established by: margin of error % uncertainty = × 100 (12) μ Margin of error (Eq. 11) % Uncertainty (Eq. 12) × 1000 Figure 11. ATR uncertainty quantification 19 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

The MoE in Figure 11 confirms that the metric is consistent between 16,000 and 29,000 feet cruise al- titude range and this pattern is primarily related to the evolution of ATR at these flight conditions. At altitudes beyond this point, the increase in MoE is due to the increase in ATR. This pattern does not reflect an increase in uncertainty as the percent uncertainty, quantified as the ratio of the MoE to mean performance (Eq. 12), is relatively flat at the matching region ( > 29,000 feet). The analysis does confirm that there is a significant uncertainty gradient between the 17,000 and 22,000 feet cruise altitude range.

The MoE is further applied to establish ATR uncertainties using the 95% CI range (Eq. 11) as a function of cruise altitude in Figure 12. Coupled into the analysis is the mean ATR from the MC runs at n = 102 , 400 and the baseline ATR reading using the best initial estimate of the emission species coefficients (Tab. 1) from Dallara, Kroo and Waitz.

Figure 12. ATR 95% confidence intervals (shaded) as a function of cruise altitude with predicted sample mean (solid line) and initial best estimate (symbol ◦ ) performances As the 95% CI range in Figure 12 is established using the ATR mean with MoE (Eq. 11), the pattern that evolves as a function of cruise altitude matches the MoE trend in Figure 11. The shaded band between the upper and lower CI limits signifies the range of values that with 95% certainty will contain the true ATR mean of the population. The ATR performance calculated using Dallara’s “best estimate” values to the emission parameters falls below the mean ATR performance from the MC runs and this pattern is consistent for the modeled altitude range. Further the estimated ATR performance is also at the lower boundary of the 95% CI which indicates that NDARC is underpredicting temperature change for this case at the default emission parameter setting.

The chart in Figure 12 is critical as it presents not only the ATR performances but the data is also accompanied with performance uncertainties. Statistical inference through MC simulations allows for an improved estimation of ATR as the properties of the output probability distribution can be estimated from a reduced sample set that is representative of a larger population. Hence, estimated ATR readings are expected to hold true within the established intervals in 95% of the cases. The analysis does have limitations as it does not quantify the factors that directly or indirectly through interactions with other parameters 20 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

impact uncertainty. To address this requirement, parameter sensitivity analysis is undertaken to estimate the sensitivity of ATR to the input sources identified in Table 1.

A.3. Sensitivity Analysis The results to the global sensitivity analysis using the Sobol indices is presented in Figure 13.

1.2 CO CH O 1 3L O 3S H O soot 0.8 SO T 0.6 0.4 Total Sensitivity Index (TSI) 0.2 15000 20000 25000 30000 Altitude (ft) Figure 13. Total Sensitivity Index (TSI) for different modeling parameters as a function of design cruise altitude The analysis shows that there is a large amount of variation between the sensitivities of ATR to the different modeling parameters. The sensitivity of ATR to the coefficients of the temperature change coef- ficients (∆ T in Fig. 13) is significantly higher than the individual emission species modeling parameters.

The temperature change portion of the ATR modeling process takes into account the RF caused by all of the different emission species and attempts to predict the total global temperature change that occurs as a result. This temperature change component of the ATR calculations gets multiplied by the total RF (see Eqns. 14 and 15 in Ref. ). This portion of the ATR calculation is therefore strongly affected by all of the RF components, hence it has higher total sensitivity index (TSI) than that of any one RF component.

There are two factors that affect the sensitivity index of the different parameters as a function of altitude.

The first of these is the total fuel burn. As the fuel burn increases, the ATR calculations become more sensitive to the modeling parameters for RF of the individual emission species, particularly the N O products x ( O , O , CH ). There is a large jump in fuel burn of the design aircraft for cruise altitudes above 31,000 3 S 3 L 4 feet in Figure 14. This large jump is the reason for the sharp change in TSI between 31,000 and 33,000 feet.

21 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

Mission Fuel Burn (lb) 15000 20000 25000 30000 Altitude (ft) Figure 14. Mission profile fuel burn (lb) The second source of variation of TSI with the altitude is the “forcing factor” that is applied to the different emission species as a function of altitude (see Fig. 3 of Ref. ). The forcing factor is a parameter that captures the varying effect of N O emissions and aviation induced cloudiness as a function of altitude.

x In particular, the forcing factor for short lived ozone ( O ) increases as a function of altitude, meaning that 3 S the RF of O will have a larger impact on ATR as the cruise altitude increases. Because the RF of O 3 S 3 S increases with altitude, its TSI increases at lower altitudes, peaking around 22,000 ft. The effect of fuel burn is apparently stronger than that of the forcing factor, and the O TSI begins to decrease for higher 3 S altitudes until the break at 31,000 ft.

The results in Figure 13 are for a single value, r = 0 . 03, of the temporal weighting of climate impacts.

It is possible within the calculation of ATR to place a higher emphasis on shorter-or longer-term effects.

In that case, the relative values of the sensitivity index would be different than the above results, but the trends with altitude would likely be similar.

VI. Conclusion The development of computational processes and methods to quantify system uncertainties and sensitivi- ties in rotorcraft design was presented. Function wrappers were developed to facilitate large-scale simulations using DAKOTA which was then coupled to a solver interface to address uncertainty and sensitivity studies.

The proposed framework requires inputs to define variable types, labels and probability distributions, in- cluding response outputs-of-interest and the MC sampling size. At task completion, results post-processing scripts are activated that use DAKOTA and function solver output files to generate plots-of-interest. This includes the assessment of statistical convergence and the generation of confidence intervals to evaluate mean system performance with expected uncertainties. Post-processing scripts further support the quantification of parameter sensitivities using Sobol indices. The capability permits the ranking of variables as a function of relative influence on response output.

The NDARC emissions module was used for case demonstration. The uncertainties and sensitivities of the emission species on Average Temperature Response were quantified for a 90 passenger tiltrotor config- uration that was conceptually sized for a range of 1,000 nm. A stochastic approach using MC simulations was applied to quantify uncertainties in ATR at a 95% confidence level. The results showed that, in general, uncertainties in ATR increase as cruise altitude rises. A global sensitivity analysis then followed to relate 22 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

parameter influences on system uncertainty. The analysis verified that there are large variances in parameter sensitivities on ATR. The highest contributors to uncertainty were the coefficients of the temperature change model (represented by ∆ T ), which collectively factors the RF caused by all emission species into predicting the total global temperature change. The analysis further showed that mission fuel burn has a significant influence on parameter sensitivity where N O products are most sensitive to increases in fuel consumption x at higher altitudes.

The toolsuite is part of an ongoing design development effort that is to support multidisciplinary design optimization and uncertainty quantification studies. Next steps in the development cycle will focus on the integration of surrogate modeling techniques to support high-fidelity rotorcraft structural and aerodynamic simulations for complex configurations. Parallel processing capabilities will also be integrated into the framework.

References Schaefer, J., West, T., Hosder, S., Rumsey, C., Carlson, J-R., and Kleb, W., “Uncertainty Quantification of Turbulence Model Closure Coefficients for Transonic Wall-Bounded Flows,” AIAA 22nd Computational Fluid Dynamics Conference, Dallas, TX, 2015 Khurana, M., Silva, C., “Optimization of a High Efficiency Civil Tiltrotor Platform for Low Emission Flight,” American Helicopter Society Technical Meeting on Aeromechanics Design for Transformative Vertical Flight, San Francisco, CA, 2018 Semmelmayer, F., Reeder, M. F., and Seymour, R., “Statistical Characteristics of the Influence of an Electronically Scanned Pressure Transducer On Measurement Error and Uncertainty,” AIAA Aviation Forum, Atlanta, Georgia, June 25-29, 4 ¨ Eek, M., Karl´ en, J., Olvander, J., “A Framework for Early and Approximate Uncertainty Quantification of Large System Simulation Models,” Proceedings of the 56th Conference on Simulation and Modelling (SIMS 56), October, 7-9, 2015, Link¨ oping University, Sweden Sandia National Laboratories, “Dakota Sensitivity Analysis and Uncertainty Quantification, with Examples,” Sandia Technical Report (training slides), SAND2014-3134P, 2017 Kenny, S.P., and Crespo, L.G., “The Role of Uncertainty in Aerospace Vehicle Analysis and Design,” NIA Workshop on Uncertainty Quantification and Management in Aircraft Design, Arlington, VA; United States, 17-18 March, 2011 Sankararaman, S., Daigle, M., “Uncertainty Quantification in Trajectory Prediction for Aircraft Operations,” AIAA SciTech Forum, Grapevine, TX, 2017 Salahudeen, A., and Baeder, J.D., “Uncertainty Quantification for Free Stream Turbulence Intensity Effects on Airfoil Characteristics,” AIAA Aerospace Sciences Meeting, Kissimmee, FL, 2018 Peˇ cnik, R., Witteveen, J.A.S., and Iaccarino, G., “Uncertainty quantification for laminar-turbulent transition prediction in RANS turbomachinery applications,” 49th AIAA Aerospace Sciences Meeting, Orlando, FL, 2011 Ronch, A.D., Drofelnik, J., Rooij, M.P.C. van., Kok, J.C., Panzeri, M., and d’Ippolito, R., “Uncertainty Quantification of the Flow Predictions around the NATO STO AVT-251 Unmanned Combat Aerial Vehicle,” AIAA Applied Aerodynamics Conference, June 25-29, 2018, Atlanta, Georgia, 2018 West, T.K., Reuter, B.W., Walker, E.L., Kleb, B., and Park, M.A., “Uncertainty Quantification and Certification Prediction of Low-Boom Supersonic Aircraft Configurations (Invited),” 32nd AIAA Applied Aerodynamics Conference, Atlanta, GA, June 2014 Siva, C., Murugan, M.S., and Ganguli, R., “Uncertainty Quantification in Helicopter Performance Using Monte Carlo Simulations,” Journal of Aircraft, Vol. 48, No. 5, September-October 2011 D´ ıaz. J., and Hern´ andez, S., “Uncertainty quantification and robust design of aircraft components under thermal loads,” Aerospace Science and Technology, Vol. 14, No. 8, Pages 527-534, 2010 Murugan, S., Chowdhury, R., Adhikari, S., and Friswell, M.I., “Helicopter aeroelastic analysis with spatially uncertain rotor blade properties,” Volume 16, Pages 29-39, 2012 Ant´ onio, C. C., and Hoffbauer, L. N., “From local to global importance measures of uncertainty propagation in composite structures,” Composite Structures, Volume 85, Pages 213-225, 2008 Oh, D.H., and Librescu, L., “Free vibration and reliability of composite cantilevers featuring uncertain properties, Reli- ability Engineering & System Safety,” Volume 56, Issue 3, Pages 265-272, 1997 Adhikari, S., Friswell, M.I., Lonkar, K., Sarkar, A., “Experimental case studies for uncertainty quantification in structural dynamics,” Probabilistic Engineering Mechanics, Volume 24, Issue 4, Pages 473-492, 2009 Scott, R., Schrage, D., “Quantifying Cost Uncertainty Effects of Non-Conventional Systems on Rotorcraft Life-Cycle Cost,” American Helicopter Society Specialists Meeting on Systems Engineering, Ithaca, NY, 2011 Russell, C., and Basset, P-M., “Conceptual Design of Environmentally Friendly Rotorcraft – A Comparison of NASA and ONERA Approaches,” AHS 71st Annual Forum, Virginia Beach, Virginia, May 5–7, 2015 Allaire, D., Noel, G., Willcox. K., Cointin, R., “Uncertainty Quantification of an Aviation Environmental Toolsuite”.

Reliability Engineering & System Safety, Vol. 126, pp. 14–24, 2014 Lim, D., Li, Y., Levine, M., Kirby, M., and Mavris, D., “Parametric Uncertainty Quantification of Aviation Environmental Design Tool,” AIAA Multidisciplinary Analysis and Optimization Conference, June 25-29, 2018, Atlanta, Georgia, 2018 Schumann, U., Graf, K., and Mannstein, H., “Potential to reduce the climate impact of aviation by flight level changes,” AIAA 3rd AIAA Atmospheric Space Environments Conference Honolulu, Hawaii, 27-30 June, 2011 23 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.

Ng, H.K., Sridhar, B., Chen, N.Y., and Li, J., “Three-Dimensional Trajectory Design for Reducing Climate Impact of Trans-Atlantic Flights,” 14th AIAA Aviation Technology, Integration, and Operations Conference, Atlanta, GA, 16-20 June, Federal Aviation Administration, “Aviation Emissions, Impacts & Mitigation: A Primer,” FAA Office of Environment and Energy, Technical Report, January 2015 Jardine, C.N., “Part 1: Calculating the Environmental Impact of Aviation Emissions,” Oxford University Centre for the Environment, Technical Report, June 2005 Dallara, E.S., Kroo, I.M., and Waitz, I.A., “Metric for Comparing Lifetime Average Climate Impact of Aircraft,” AIAA Journal, Vol. 49, No. 8, August, 2011 Air Emissions Inventory Improvement Program, “Evaluating the Uncertainty of Emission Estimates,” Radian Corporation Technical Report, July 1996 Russell, C., and Johnson, W., “Application of Climate Impact Metrics to Civil Tiltrotor Design,” AIAA Aerospace Sciences Meeting, Grapevine, TX, 7-10 January, 2013 “Intergovernmental Panel on Climate Change, Aviation and the Global Atmosphere,” Cambridge Univ. Press, Cambridge, UK, 1999 “ICAO Aircraft Engine Emissions Databank,” available at https://www.easa.europa.eu/easa-and-you/environment/icao- aircraft-engine-emissions-databank, accessed October 2018 Deidewig, F., D¨ opelheuer, A., and Lecht, M., “Methods to Assess Aircraft Engine Emissions in Flight,” 20th International Council on Aeronautical Sciences Congress, Sorrento, Italy, 1996 K¨ ohler, M. O., R¨ adel, G., Dessens, O., Shine, K. P., Rogers, H. L., Wild, O., and Pyle, J. A., “Impact of Perturbations to Nitrogen Oxide Emissions from Global Aviation,” Journal of Geophysical Research, Vol. 113, Paper D11305, 2008 Sausen, R., Isaksen, I., Grewe, V., Hauglustaine, D., Lee, D., Myhre, G., K¨ ohler, M., Pitari, G., Schumann, U., Stordal, F., and Zerefos, C., “Aviation Radiative Forcing in 2000: An Update on IPCC (1999),” Meteorologische Zeitschrift, Vol. 14, No. 4, August 2005 R¨ adel, G. and Shine, K. P., “Radiative Forcing by Persistent Contrails and its Dependence on Cruise Altitudes,” Journal of Geophysical Research, Vol. 113, D07105, 2008 Chen, N., Sridhar, B., and Ng, H., “Tradeoff Between Contrail Reduction and Emissions in United States National Airspace,” Journal of Aircraft, Vol. 49, No. 5, Sept.-Oct. 2012 Adams, B.M., Bauman, L.E., Bohnhoff, W.J., Dalbey, K.R., Ebeida, M.S., Eddy, J.P., Eldred, M.S., Hough, P.D., Hu, K.T., Jakeman, J.D., Stephens, J.A., Swiler, L.P., Vigil, D.M., and Wildey, T.M., “Dakota, A Multilevel Parallel Object- Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 6.0 Users Manual,” Sandia Technical Report SAND2014-4633, July 2014. Updated November 2015 (Version 6.3) Johnson. W., “NDARC - NASA design and analysis of rotorcraft,” NASA/TP 2015-218751, NASA Ames Research Center, Moffett Field, CA, United States, 2015 Johnson, W. “NDARC NASA Design and Analysis of Rotorcraft. Theoretical Basis and Architecture,” American Heli- copter Society Specialists’ Conference on Aeromechanics, San Francisco, CA, January 2010 Johnson, W. “NDARC NASA Design and Analysis of Rotorcraft. Validation and Demonstration,” American Helicopter Society Specialists’ Conference on Aeromechanics, San Francisco, CA, January 2010 Chen, N., Sridhar, B., and Ng, H., “Tradeoff Between Contrail Reduction and Emissions in United States National Airspace,” Journal of Aircraft, Vol. 49, No. 5, Sept.-Oct. 2012.

Roy, C. J., and Oberkampf, W. L., “A Complete Framework for Verification, Validation, and Uncertainty Quantification in Scientific Computing (Invited),” 48th AIAA Aerospace Sciences Meeting, Orlando, FL, 4-7 January, 2010 Uusitalo, L., Lehikoinen, A., Helle, I., and Myrberg, K., “An overview of methods to evaluate uncertainty of deterministic models in decision support,” Environmental Modelling & Software, Vol. 63, January, 2015 Sobol, I.M., “On sensitivity estimates for nonlinear mathematical models,” Matematicheskoe Modelirovanie, Vol. 2, No.1, Pages 112–118, 1990 Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., and Tarantola, S., “Variance-based methods,” In: Global Sensitivity Analysis, Wiley & Sons Ltd, Pages 155–182, West Sussex, 2008 Williams, B. J., Adams, B. M., Coleman, K., Gilkey, Lindsay N., Gordon, N., Hooper, R., Khuwaileh, B. A., Lewis, A., Maupin, K., Smith, R. C., Swiler, L. P., and Turinsky, P. J., “User Guidelines and Best Practices for CASL VUQ Analysis Using Dakota,” Los Alamos National Laboratory, Technical Report, LA-UR-17-29083, October 2017 Bielen, A.S., “Sensitivity and Uncertainty Analysis of Multiphysics Nuclear Reactor Core Depletion,” Ph.D. Dissertation, Nuclear Engineering and Radiological Sciences, University of Michigan, MI, 2015 Saltelli, A., Chan, K., and Scott, E.M., “Sensitivity analysis,” Series in Probability and Statistics, Wiley, West Sussex, Porter, N., Avramova, M., and Ivanov, K., “Uncertainty and Sensitivity Analysis of Cobra-TF for the OECD LWR UAM Benchmark Using Dakota,” Nureth-16, Chicago, IL, August 30 – September 4, 2015 24 of 24 American Institute of Aeronautics and Astronautics Distribution Statement A. Approved for public release; distribution is unlimited.