Document
Background Oriented Schlieren (BOS) of a Supersonic
Aircraft in Flight
1 2 3 4 5 James T. Heineck , Daniel W. Banks , Edward T. Schairer , Edward A. Haering, Jr , Paul S. Bean This article describes the development and use of Background Oriented Schlieren on a full-scale supersonic jet in flight. A series of flight tests was performed in October, 2014 and February 2015 using the flora of the desert floor in the Supersonic Flight Corridor on the Edwards Air Force Base as a background. Flight planning was designed based on the camera resolution, the mean size and color of the predominant plants, and the navigation and coordination of two aircraft. Software used to process the image data was improved with additional utilities. The planning proved to be effective and the vast majority of the passes of the target aircraft were successfully recorded. Results were obtained that are the most detailed schlieren imagery of an aircraft in flight to date.
I. Introduction Historically, obtaining schlieren and shadowgraph images required a specialized point light source, mirrors or retro- reflective screens, and a photographic recording method. Mirror optics limited the size of aerodynamic investigations. Retroreflective screen shadowgraphy is not limited by mirror optics but has limited sensitivity. With the invention of synthetic schlieren in 1998 and subsequent improvements and renaming the technique to 2,3 Background Oriented Schlieren in 1999 to 2000 at DLR Goettingen , those field-of-view limitations were removed. All that is required is a digital camera, either still or video, and a suitably textured background that lends itself to image cross correlation.
A systematic approach to obtaining schlieren imagery of an aircraft in flight began with Weinstein , whose telescopic imaging of the s un limb provided a “knife edge” to produce the schlieren effect. This technique required the aircraft to be flown precisely between the observing telescope system on the ground and the sun, which subtends an angle of 1/2˚ . This film-based streak camera method observed and recorded the distortion of the outer edge of the sun limb. The cumulated image sequence of the distorted sun edge was processed to provide a schlieren image. His results are widely recognized as a milestone in flight testing (Figure 1).
NASA Aeronuatics ’ Commercial Supersonics Technology Program has been developing new aircraft designs that reduce the downward propagating shockwaves, or sonic boom. The culmination of this program is the Quiet Supersonic Technology Flight Demonstrator, which is due to fly in 2020. The need for visualizing the shockwaves generated by this aircraft has prompted the development of schlieren imaging methods for the flight testing. Three methods are being developed in parallel, each complimentary to the other. Two methods, Ground-to-Air Schlieren Photography System (GASPS) and Background Oriented Schlieren with Celestial Objects (BOSCO) use ground based telescopes that image the sun as an aircraft passes between the sun and telescope, and are direct outgrowths of Weinstein’s pioneering method. A third method, Air-to-Air Background Oriented Schlieren (AirBOS), images the target aircraft from an observation aircraft flying above it and uses the natural flora on the ground to derive the schlieren imagery. The sparse Mojave Desert flora of the Supersonic Corridor near Edwards Air Force Base provides this nearly ideal background for this technique. This method was demonstrated in 2011 , though the publication has restricted release.
This Air-to-Air method is an extension of the BOS method described by Hughes Richard and Markus Raffel. In 2000, Raffel et. al demonstrated the use of Background Oriented Schlieren as a method to show wingtip and rotor vortices for full-scale aircraft. The technique proved to be effective and they subsequently performed flight tests in Germany using leaves of the forest in the Harz Mountains as their background. In 2012 the DLR Goettingen group demonstrated the AirBOS technique on full-scale helicopters in flight, rendering the blade-tip vortices and engine 6,7 exhaust. Additionally, Raffel, et. al. summarized a number of large scale BOS applications, which include both flight and wind tunnel tests.
Pho tographic Technologist, Experimental Aero - physics Branch, NASA Ames Research Center, Moffett Field, CA , AIAA Member Aerospace Engineer , Aerodynamics and Propulsion Branch, NASA Armstrong Flight Research Center, Edwards , CA , Associate Fellow Aerospace E ngineer, Experimental Aero - physics Branch, NASA Ames Research Center, Moffett Field, CA Aerospace Engineer , Aerodynamics and Propulsion Branch, NASA Armstrong Flight Research Center, Edwards , CA , Aerospace Engineer, Flight Systems Branch, NASA Dryden Flight Research Center, Edwards, CA Figure 1. Weinstein’s in - flight schlieren image of a supersonic aircraft from 1994.
II. The BOS technique The principle, as shown in Figure 2, works by detecting minute shifts of features in a background caused by the presence of a density gradient, which creates a refractive index change in the air. A properly designed BOS system will have the speckle size in the 2-5 pixel range. A quiescent image of the background serves as a reference to the image of the background distorted by the density gradient.
These distortions are often smaller than a pixel, so they are hard to detect with a naked eye. However, image processing techniques such as cross-correlation and optical flow reveal these minute displacements of the speckle pattern. Figure 2 also shows the work flow and principle of BOS. As with all measurement techniques that use image cross correlation, e.g. Particle Image Velocimetry and Surface Deformation Measurements (sometimes known as digital image correlation) the higher the resolution of the camera, the more detail can be gleaned. Also, a sensor with small pixels (4-10 microns) will be more sensitive to detecting weaker gradients that produce smaller background displacements than sensors with 20-25 micron pixels, because cross-correlation detection algorithms comfortably detect displacement of less than a tenth of a pixel.
The data that results from the cross correlation is an array of points with displacements in the horizontal x (D ) and x vertical y (D ) directions in the image plane. Assembling these data as a contour plot of D , D or a vector sum of D y x y x and D . The gray level represents the displacement in pixels and results in an “image” that is analogous to y conventional schlieren imagery. A plot of D is equivalent to a vertical knife orientation and a plot of the vertical x (D ) would be a horizontal knife orientation.
y Figure 3 is an example of wind tunnel imaging using Retroreflective Background Oriented Schlieren applied to the NASA Ames Unitary Plan 11-foot Transonic Tunnel. The background was made by careful sputtering of retroreflective paint on the floor of the tunnel and imaging that background with a digital camera. The first image (A) was acquired without the presence of the model, and acts as the reference, the second image (B) is the “wind - o n” or data image and the third (C) is the result of displacements computed by cross -correlation of the wind-off and wind-on images. This image is a launch abort tower in Mach= 1.3 flow, with the grayscale contour of the D x displacement. Here the compression waves are dark and the expansion fans are light.
Figure 2. The Principle of BOS . A light ray from the background (a speckle) passes through a density gradient and is deflected. The angle of deflection, ε, is proportional to the strength and optical thickness of the gradient. The apparent shift in the location of a speckle pattern is detected with image cross correlation . The position of the correlation peak defines the shift, D and D . The “schlieren image” is a contour plot of the disp lacement in x, y, x y some vector sum, or absolute magnitude.
A B C Figure 3 . Example of BOS imagery from wind tunnel testing . Reference (A), wind-on (B), and processed schlieren image (C) of the launch abort tower of Orion Space Capsule in Mach = 1.3 flow. The schlieren image is a contour plot of the horizontal deflection distances (in pixels.)
III. Flight Planning: Making AirBOS work The approach to applying this technique to flight was to observe and record the supersonic aircraft flying under a slower-moving aircraft. The sparse desert flora is used as the visually textured background. The optical system and flight planning were designed to optimize the parameters of the BOS technique.
Of particular importance was the size of the background speckles in the image of the camera. The optimal speckle size for image cross correlation is 2-5 pixels. Speckle size is dependent on the physical size of the flora, the lens focal length, the camera pixel size, and the altitude. The angular field of view is dependent on the pixel size and number of pixels. The dominant plant that can be seen from high altitude in the Mojave Desert is the creosote bush, interspersed with Joshua trees. The Joshua trees were not in sufficient abundance to factor in the average size of the flora so the attention was focused on the creosote bush. There are three special characteristics of the creosote plant in this environment. The first is that is it grows to approximately 10-12 feet in diameter (3-4 meters). Second, there is a natural mechanism that maintains separation of 5-10 feet (1.5 to 2 meters) between plants to prevent over population in an arid landscape. Third, from the air the creosote plants appear much darker than the desert floor at selected wavelengths, providing very good contrast. Taken together, these characteristics make creosote bushes the ideal background for implementing the AirBOS technique (see the desert floor in Figure 4).
The flight configuration consisted of a slow-moving-observing aircraft, the NASA Super King Air, flying at one altitude with the supersonic test aircraft flying at a lower altitude directly below the observer. The flight plan sought to have the target aircraft as high as practical. While the maximum altitude of the King Air is 35,000 feet MSL, operationally 30,000 was more time efficient and still allowed for higher altitude supersonic passes. The camera in the observing aircraft is mounted in a nadir-view port. So with altitude, sensor and speckle dimensions defined, a lens was chosen that rendered the background at the desired spatial frequency (2-5 pixels/bush). Reference images were acquired by the observer aircraft prior to the test aircraft entering the camera field-of-view, thus continuous recording was necessary as the test aircraft approached.
Figure 4 . Beechcraft B - 200 Super King Air observing aircraft. Note the sparse vegetation on the desert floor.
The imaging system consisted of two Phantom V641 cameras, whose sensor has 2560 x 1600 pixels with a 10 micron pitch. A #25 Red glass filter was attached to each lens to improve contrast and mitigate the effects of atmospheric haze. Their maximum full-frame recording frequency is 1470 frames per second. This frame rate was changed for each pass depending on separation distance. The resolution of the bushes is a very important parameter to optimize, so this was established by the altitude and the approximate average diameter of a creosote bush. An Excel worksheet was built that calculated the resolution and angular fields of view for different combinations of camera and lens parameters and aircraft altitudes. Since the average bush diameter was determined to be 10-12 feet and the observer altitude was 30,000 feet, the appropriate lens focal length was deterministic. The lens that provides this resolution is 180 mm. The angular field of view is 8.14 degrees. Table 1 is a sample of the resolution calculator used for the flight planning. Note that the resolution on the ground was calculated to be approximately ½ pixel per foot, with a bush approximately 10 feet that placed the bush resolution at approximately 5 pixels.
Table 1 . Sample results from the Excel spreadsheet used for determining magnification of both the target aircraft and the bushes used for the speckle pattern.
A legacy camera from the 2011 test campaign was mounted in the forward port. This was a Goodrich SUI SU640- SDWHVis-1.7RT InGaS camera with a 640 x 512 pixel sensor, 25 micrometer pixel pitch, and fitted with a 105 mm lens. It was used as a real-time spotting camera.
Two nadir camera ports are available in the King Air. Figure 5 is a drawing of the cabin layout, showing the Goodrich camera located in the forward port. These cameras were mounted to a common plate and bracketed to a flange plate that was adapted to the window structure. Figure 6 is a diagram of the flight pattern each aircraft followed over the supersonic corridor. The data acquisition began at the beginning of the straight run of the King Air over the target area of the supersonic corridor. The KingAir pilot used a Garmin 496 GPS unit to establish the course line. The Supersonic aircraft used military grade GPS for navigation.
Figure 5. Layout of imaging system in the Beechcraft B-200 Super King Air observing aircraft A reconnaissance flight was made over the area of the corridor that appeared to be the best location for the BOS background. The area was quickly evaluated in Google Maps using satellite view and visited during the flight.
Imagery was recorded and evaluated. Hargather and Settles offered a method to evaluate natural backgrounds for the purpose of BOS applications. This method was not as useful as simply determining the minimum interrogation size that produced a signal to noise ratio of 4 or greater in the correlation function between two sequential images. As the interrogation area decreases, more detail can be gleaned. However as the interrogation are decreases the SNR decreases. The SNR lower threshold for these data is four. In the case of the images from figure 6 below, a 16 pixel window size gave a SNR of 5.24. Subsequent data was processed with this parameter.
Figure 6 . Flight course line and re-entry pattern of the King Air and the supersonic aircraft in the Black Mountain Supersonic Corridor IV. Image Processing Image processing algorithms were used to determine the pixel shift of the background. Pixel shift is correlated with change of index of refraction and consequently the change in density and shock wave location. These data were processed using an in-house program called BOS_ETS.
Several image processing steps were required to produce the schlieren image. The initial step was determining which of the numerous passes were the most appropriate to process: on some passes the target aircraft did not enter the camera view; others were over areas of the desert with sparse plant density. A sequence, such as shown in Figure 7, was extracted from each pass for processing. The first image of each sequence was the last “clear” frame.
The stack of subsequent images was registered to the first using software developed at NASA Ames. This eliminated the effects of movement of the observer aircraft relative to the desert floor and assured that the displacements calculated were not offset by any apparent movement in the background.
Figure 7. Sequence of raw images s elected from the series of 300+ acquired from a pass where the observer flew at 28300 feet AGL and the test aircraft flew at 23300 feet AGL.
A. Reference image alignment The displacement of the background in each image relative to the first image was measured by image cross- correlation in a large interrogation window far removed from where the test aircraft appeared. The first time the images were analyzed during the 2011 campaign, they were registered using only a single reference point, resulting in simple x and y displacements of each image relative to the first. It was discovered, however, that this correction did not account for small variations in attitude of the observer aircraft. Therefore, as an alternative, each image was registered to the first image based on displacements measured at four reference points located near the corners of the images, and a first-order projective transformation was computed: 𝑎 𝑥 +𝑎 𝑦 +𝑎 1 0 2 0 3 A. 𝑥 = 𝑐 𝑥 +𝑐 𝑦 +1 1 0 2 0 𝑏 𝑥 +𝑏 𝑦 +𝑏 1 0 2 0 3 B. 𝑦 = (1) 𝑐 𝑥 +𝑐 𝑦 +1 1 0 2 0 where (x , y ) and (x = x + D , y = y + D ) are the image coordinates of the reference points in the reference and 0 0 0 x 0 y the current image respectively. The eight coefficients of the transformation (a , b , c ) were determined by 1-3 1-3 1-2 stacking equations for x and y at the four reference points and solving the resulting set of eight linear equations for the unknown coefficients. Then the transformed pixel coordinates of each point in the current image were computed from Eqn. 1A and Eqn. 1B. This eliminated differences in the images that were not caused by the disturbances created by the flow features of the test aircraft. These differences include differences due to pitch, roll or yaw of the observer airplane, and camera and lens vibration.
B. Reference-to-data image correlation Once the images are registered, local displacements of the background due to disturbances created by the test aircraft are determined by defining an interrogation grid and computing the displacement of each node by image cross correlation. The interrogation window size – 16 pixels – was determined by trial and error to be the smallest window that resulted in a signal-to-noise ratio (SNR) of the cross correlation function of at least four. A SNR threshold of four is conservative for obtaining a resolvable peak in the correlation that was measurable. That is, with a lower SNR, it was difficult to measure the subpixel shifts. The grid density was set to have a node every three pixels to achieve as highest resolution schlieren image as possible. The processing of the correlations used the multi-grid method, starting with 32x32 pixel windows and then refined to 16x16 pixels. A high-pass filter of 5x5 was applied prior to the correlation pass. A five-point Gaussian peak finder was applied in each direction to encompass the size of the plants after the high-pass filtering. Finally, a conservative outlier detection was performed, which permitted interpolation from the nearest neighbors.
The data from this correlation step produces an array dataset of i, j, D , D , and SNR for each correlation. The x y displacements of D were made into a gray scale contour plot and exported as bitmaps to produce the final single- y instance schlieren image. Figure 8 shows the results of cross correlation of the aligned reference image to the single images from the above sequence. There is an accumulated band of noise on the side and bottom of the images as the background from the data image moves further down-course from the reference image. If the observer were perfectly stationary this noise band would not be seen.
Figure 8. BOS images of single frames from the sequence. Each image is a grayscale contour plot of the vertical deflection, Dy. Note the slight change in tone as the sequence progressed. Target is flying at M = 1.05 and is 5000 feet below observer.
C. Averaging of schlieren data .
Random correlation noise has always limited the BOS image quality. Wind tunnel data can be simply averaged since data images are registered to the reference and the model tends to be stationary. In the AirBOS case, the target aircraft moves through the field of view, such that a simple average of the sequence as it is processed is not applicable. A second tracking algorithm was written to account for the inter-frame motion of the target aircraft after the data image is registered to the reference image. This algorithm shifted the data grid to match the pixel displacement of the aircraft. The shift was determined by cross-correlating features on the aircraft, such as a specular highlight, after the registration step. The displacement data from each sequence is averaged, not the grayscale images output from the individual sequence plots. This permits plotting the averaged data in any knife edge orientation, as well as a magnitude of displacement and eventually a calculated density. The averaged image in Figure 9 clearly shows the major shock structure generated by the aircraft, including the expansion shock around the canopy. One can see where the weaker shocks coalesce with the stronger shock in the far field.
Figure 9. A simple average of the resulting D data is shown. An aircraft tracking utility in the BOS program allows y for registering the correlation grid from each instance.
VI. Effects of Separation Distance Increasing the details in the shock structure was attempted by closing the separation distance between the observer and target aircraft. While this was desirable from an imaging perspective, the imaging cone narrows with decreasing separation. This makes the navigation more difficult. The closest separation attempt was at 2000 feet, which forced the pilot of the target plane to fly through a 200-foot, cross-track window. Also, the transit time of the aircraft in the camera field of view is much shorter, requiring higher frame rates and careful trigger timing. Figure 10 demonstrates the improved detail for the shock structure resulting from increasing the magnification of the area of interest by reducing the separation distance. Note that changing the lens to a longer focal length would increase the size of the images of the bushes, so separation distance is the only parameter available for increasing magnification.
Figure 10, Average of D in grayscale contour of the shock structures around the T-38 at M=1.05. The image of the x aircraft was added after the grayscale contour was generated from the data.
Figure 11 is a similarly processed image on a pass where the pilot attempted a 90 roll. The intent is to view the downward propagating shocks that a person on the ground would experience. Unfortunately the roll was only 45 , but the downward propagating shocks are visible. Figure 11A is a plot of the deflections in the D direction and 11B, y produced from the same dataset, is a contour plot of the lateral deflection D . Note that the wing tip vortices have x been rendered in this view.
11 A 11 B Figure 11 , Schlieren image with aircraft of the T-38 in a 45 roll revealing shocks emanating from structures on the underside of the aircraft. A, grayscale plot of the D value and B, grayscale plot of the D value.
y x VII. Conclusions A new method for obtaining high resolution imaging of density gradients in air around a full-scale airplane in flight has been demonstrated. The utility of this technique goes beyond the capturing of shockwaves; it can be used to study subsonic flows from tip vortices to propeller and helicopter blade tip vortices. The ability to average the flow field solutions of many instantaneous images improves the signal to noise such that the calculation of density from these data becomes more feasible. Future work includes additional flights with multiple aircraft formations, improving the image processing using optical flow algorithms, improving work flow with parallelized image processing code, and faster data transfer hardware for the imaging system.
VIII. Acknowledgements The authors wish to thank Peter Coen, Program Manager of the NASA Commercial Supersonics Technology Program for the funding and support, and Brett Pauer of NASA Armstrong Flight Research Center for his capable management of the project locally, and to Don Durston at NASA Ames Research Center for his continued advocacy for this work. Also, Air Force Test Pilot School for providing the T38 aircraft and pilots Maj. Jonathan Orso and Maj. Jeremy Vanderhal. Finally, we also thank the NASA Armstrong Chief Pilot Nils Larson and the flight crews for their professionalism, masterful piloting skills, and sage advice.
IX. References th 1. Dalziel, S.B., Hughes, G.O., Sutherland, B.R., “Synthetic Schlieren” 8 Int. Symp. On Flow Visualization, Sorrento, Italy, Sept 1-4, 1998 2. Raffel M, Tung C, Richard H, Yu Y, Meier G “Background oriented stereoscopic schlieren (boss) for full scale th helicopter vortex characterization”, 9 Int. Symp. on Flow Visualization, Heriot-Watt University, Edinburgh, G.B. August 24-29, 2000 3. Richard, H., Raffel, M., “Principle and applications of the background oriented schlieren (BOS) method” Measurement Science and Technology 12(9): pp 1576 – 1585, 2001 4. Weinstein, L.M., “An optical technique for examining aircraft shock wave structures in flight” NASA CP 3279, p 1-17, 1994 5. Banks, D. W., Heineck, J.T., Bean, P.S., Martin, B.J., Larson, D.N., Schairer, E.T., Walker, L.A., “Flight Validation of an Air-to- Air Background Oriented Schlieren Technique” NASA/TM -2014-218323, (ITAR) Oct.
6. Kindler, K. Goldhahn, E., Leopold, F., Raffel, M., “ Recent developments in background oriented schlieren methods for rotor blade tip vortex measurements” Experiments in Fluids , 43:233 – 240, 2007 7. Bauknecht, A., Landolt, A., Meier, A. H., Merz, C.B., Raffel, M., “Blade Tip Vortex Detection in Maneuvering th Flight Using the Background Oriented Schlieren (BOS) Technique” 69 AHS Forum, Phoenix, AZ, May 21-23, 8. Raffel, M., Heineck, J.T., Schairer, E.T., Leopold, F., Kindler, K. “ Background oriented schlieren imaging for full-scale and in- flight testing” Journal of the American Helicopter Society , 2013.
9. Heineck, J.T., Schairer, E.T, Walker, L.A., Kushner, L.K., “Retroreflective Background Oriented Schlieren th (RBOS)” 14 Int. Symp. On Flow Visualization, Daegu, Korea, June 21-24, 2010 10. USDA Plant Website: http://plants.usda.gov/core/profile?symbol=LATR2 11. Hargather, M.J., Settles, G.S., “ Natural-background-oriented schlieren imaging ” Experiments in Fluids, 48:59- 68, 2010