Document
Neural Networks in Nonlinear Aircraft Contro l
Dennis J. Linse Depar t men t o f Mechanical and Aerospace Engineering Princeton University Princeton, New Jersey Recent researc h indicates t ha t Artificial Neural Networks offer in t eresting learning or adaptive capabilities. The curren t research focuses on the po t en t ial for application of neural networks in a nonlinear aircraf t control law. The current work has been to determine which networks are suitable for such an application and how they will fit into a nonlinear control law.
1 51 Parameter Estimation i n Nonlinear Control The equa t ions o f mo t ion of an aircraf t can be cast in t o a set of nonlinear ordinary differential equa t ions t ha t are linear in the control input, u.
Included in the state and output equations is a parameter vector, p.
Several methods, including Nonlinear Inverse Dynamics [Lane and Stengel, 1988] and decoupling [Singh and Rugh, 1972], are available for genera t ing nonlinear feedback laws if the system is known. Using these techniques, the feedback laws are determined as functions of the parame t er v ector, p. p itself is a functi o n of the curren t sta t e and possibly ex t erna l s t a t es and con t rols not included in the system dynam i cs. The ma i n difficulty is to provide an estimate of the parameters tha t are possible complex nonl i near funct i ons of the states.
Given a No nl i ne a r Dynam ic System (e .g . an air c ra[ _ ) i = f(x, p ) + G( x , p )u y --- h (x , p ) and a No nlinear Feedb a c k L a w u = a(x, p ) + B(x, p)v wb e_ p = p(x, Xex t , Uext ) h o w d o yo u e _t lm a te t h e p ar a meter R , p ?
Nonlinear Function Approximation Whi l e many t ec h n iq ues exis t for non li near function approximation, three have been chosen for fur t her i nves ti gation. A B-spline (bas i s spl i ne) t echnique wi t h coefficients upda t ed using Recursive Leas t -Squares es t ima t ion represents a classic function approximation me t hod. Two neural ne t works methods are also investigated. The Back-Propagation Feedforward Network [Rumelhart, Hinton, and Williams, 1986] is a popular, widely investigated, model in the neural ne t work community.
The Cerebellar Model Articu l ation Controller (CMAC) [Albus, 1975] is less well known, but very useful and powerful in funct i on approximation implementations.
T h re e a pp ea l in g m e thod s • C1 AR_Ie m e thod - R ecu r si v e L ea s t S qtm r e_ E _t i n m t lon uMn g B -Spl i neR • N e ur a l me th odA - B ac k-Pr op ag a t i o n F ee dfor wa rd N e twn r k - C er c b c1|a r Model A rt ic u l nt i o n Co nt r _ ll e r (C MAC ) Netwo r k 1 53 Comments on Operation and Learning Eac h of the t h ree tec h niques has different adaptation (or learning) capabilities. The B-spline method requires a matrix inversion that may be very slow, especial l y for high input dimension sys t ems. The least-squares estimation scheme effectively extracts information from the input data, making adaptation fas t when measured in terms of number of points presen t ed to the estimat o r. The feedforward network, on the other hand, needs many presen t ations to accura t ely approxima t e the nonlinear function. Once trained, it can be extremely fast in operation, especially if fully implemented on a VLSI chip. The CMAC quickly approximates the desired function in the neighborhood of the training points and provides good operation speed on a traditional computer architecture.
• Re c ur si ve L ea s t ._ luare a B - Spl i ne a - S l o w l e a rning Ma t r i x i n v e rR ion - Effective lea r n ing • Back-Propagat l on Feedf o rw a rd Ne t w or ks - Rel at i v el y I ne ffici ent le a rnin g M an y tr a ining p cfi n hq n ee d e d - FA .q T nperat l on • C M AC - R e aa o na ble lea r nin g and opera t ion Approximation by Neural Networks The approximation capabilities of tradi t ional spline techniques are relatively well-known. The capabilities of the neural networks are less well-known. The CMAC approximates by a generalized table look-up.
Using overlapping, quantized inputs, the CMAC output is a piece-wise continuous approximation of the input function. The approximation accuracy is limited by the size of the table used in the look-up scheme.
The feedforward network provides a continuous approximation of the desired function with the accuracy determined by the number of layers and nodes in the network architecture. There has been much recent i nterest i n exactly deter mi n i ng the app r o x i mat i on capabi li t i es o["such networks.
• CM AC 'a ap p roxi ma t e by ge n e ral ized t a b l e I c_ k- u p - Approxi mat i on limlte d by t a b le size - St a l r ._ tep output • F ee d forward network _ - Ca p a city dete r m ined b y nc d e a and lnyers - Cnn t in u o um nut.put - Muc h re c ent in te r e s t in app r _ r x lma t lon a b ilities Back Propagation Neural Networks A v ery simple, 2-input / 2-output back-propaga t ion netw o rk is g i ven here.
The operation of the network can be described by a simple recursive rela t ionsh i p between the outpu t s of each layer. At each node, a weighted sum of the outpu t s from all of the nodes of previous layers is acted on by a simple nonlinear func t ion to provide the output for the node. A fixed, unity input is provided to each node to act as a threshold or bias. When the number of layers and nodes in each layer is chosen, the overall nonlinear function calculated by the network is determined by the values of the weigh t s in the interconnections.
W (0) W O) x1 (o) x 1 x ' ;' x _ z ) Thr _h o l d 1. 0 x Ck ) = s[W C k- Z) x C k- _) ] y -- x ( N ) = f[ x ¢o ) ] 1 56
Exact Representation using Neural Networks
While investigating the limits of neural networks, Hecht-Nielsen [I9871 was able to reinterpret a theorem by Kolmogorov related to the exact representation of a multi-input nonlinear function i n terms of simple single-input functions. While back-propagation neural networks can be interpreted in terms of this theorem, the nonlinear function a t each node is fixed by a limiting process which depends explicitly on tllc function to bc represented. Most neural networks, on the other hand, have a common, simple function a t t h e network node, and use t h e weighted interconnections to adjust the output of the network. There are many further difficulties with this theorem that limit its usefulness except a s the most basic of existence proofs.
Hccht-Nidncnk Inlcrpmbtion oTKolmqomv'n Tt~norcrii (Solution to Hilbert'n 13th P r A l c m ) - Exact Rcprencntnlion 2n " p - l f(x) = C g( C h. v ( x p t c q ) + q ) q = o p = l - W can be oblained an a uniform limit n f a ncqurncc v r ofconlinuoun nondccrcaning piecewise linear functions - W i n dillicult Lo computc - W in difrcmnt for each Rx) Approximation Representation for Feedfor w ard Networks In most instances an appr o ximate represen t a t ion of a non l inear funct i on i s all t ha t is needed. Many researchers have been investigating the approxima t ion capabilities of feedforward networks of the type described.
These theorems are usually based on results from functional analysis and give sufficient cond i tions for the approximation of any cont i nuous function to any desirable degree of accuracy. In general, they are not based on the Kolmogorov's Theorem for exac t represen t ation.
• Ca n f ee df o r w ard netw o rk a a pp roximate non i l n e ar fu n ct inn g ?
• Recent theoretic al r e ml l ts by Fun a h as hi, C yb e n ko , tlecht- N ie l sen, Stin c he o m b e an d White , and many other _ • B a sed on re s ults f r o m f uncti o nal analy s i R Cybenko's Theorem As an exam pl e o f these recent t h e o rems, t he vers i on due to C y benko [1989] is outlined here. Defining a s i gmoidal funct i on i n the most general form, Cybenko's Theorem shows t hat fini t e sums of s i gmo i dal functions, exactly the form of a single hidden layer feedforward neural network, are dense in the cont i nuous functions on the uni t hypercube. In s i mpler t erms, any continuous funct i on wi t h inpu t s be t ween 0 and 1 can be arbi t rarily closely approxima t ed by a neural network with one hidden layer. The other researchers have developed similar results. The differences are usually technical deta i ls related to how smooth the approx i ma t ed funct io n is and w h a t t ype of funct i ons are a ll owed in t he nodes of the hidden l ayer.
While t his is a very promis i ng resul t , it is only an existence proof. The number of nodes necessary in the hidden layer is on l y spec i fied as fin i te, and the we i ght vec t or is left unspecified.
_ : W e s ay that G in sigm o ida l if o(t)-- * 0 a s t- _ - * o las t _ ,oo _ : L et o b e any eo ntin u o un Rigmoid a l f u n c ti o n. Th e n finite s um s of the f o rm N g( X) = j ___i_t j fl ( w j rx + 0j) are d en n e in C ( [n).
I n oth er word s , g iv en f_ C(In) and E >O, the r e i s n _u m , g _ x), of t h e abov e f o r m , fi,r w hi ch
x )
for all x _ I n.
Example Functions The sigmoida l f unct i ons , as defined for Cybenko's Theorem, can have many different shapes, including those shown here. The back-propaga t ion algorithm requ i res a d i fferentiable nonlinear function at each node. In most implementations the function is similar to the firs t one shown.
As mentioned previously , the other approximation theorems have different requirements on the nonlinearity allowed in the node. Two are shown here tha t fail to meet the requirements for Cybenko's Theorem, but are shown to be su ffi cient for function approximation by others.
• Sa t i sfyi n g C yb enko 'M T h eorem I I • O t her Theo _ r nR 1 6 0 What's Ahead Future work i ncludes i nvest i ga ti ng the approx i mation capab i lities of reasonable sized ne t works for mul t i-input functions of the type that w i ll be found in a nonl i near con t rol law. This w i ll allow for a complete implementation of a nonl i near con t rol law for the 737 aircraft us i ng neural networks.
• M ulti d im ens iona l A pp roxim a ti o ns • C o mplete No nlinear Inverse D yn a m ics implement a ti o n f or 737 • I m pl eme nt Neu r a l N etw o r k sin Ni l} c ont ro l la w • H ope fo rc o ns tr uc tiv e r esu lts re g a rdi n g Ne u r al N e two r kf unc ti on approxi m a ti on