Abstract: The standard approach to control and optimization of stochastic systems is based on dynamic programming. This approach works backward in time and it treats the infinite-horizon problems as the limiting case with the backward time going to infinity. Optimality equations are first derived and then it is proved that the solutions to the optimality equations indeed lead to optimal policies. When the value functions are not differentiable, the concept of viscosity solutions is introduced.
A sensitivity-based approach has been developed recently to stochastic learning and optimization. The approach was first developed for discrete event dynamic systems and is being extended to continuous-time and continuous state systems. The basic idea is: fundamentally, one can only compare the performance of two policies at a time; and therefore, when developing optimization theories and methodologies, one has to first study the different of the performance of any two polices. It has been observed that for many stochastic optimization problems the information about the difference of the performance of any two policies can be decomposed into two factors, each of them is associated with one of the policies only. This feature allows us to find a better policy than the policy we are evaluating without analyzing any other policies. We found that this “information decomposition” is essential for optimization.
This “information decomposition” approach has some advantages over the dynamic programming approach: It is intuitive clear because it is based on a direct comparison of any two policies. Thus, it is easy to verify that the solution to the optimality equation is indeed optimal; viscosity solution is not needed. This approach applies in the same way to different performance criteria, including finite and infinite-horizon problems. Furthermore, the approach brings some new insights that leads to new methods and results in control and optimization, including the event-based optimization and gradient-based learning.

Brief Biography of the Speaker:
Xi-Ren Cao received the M.S. and Ph.D. degrees from Harvard University, in 1981 and 1984, respectively, where he was a research fellow from 1984 to 1986. He then worked as a consultant engineer/engineering manager at Digital Equipment Corporation, Massachusetts, U.S.A, until October 1993. Then he joined the Hong Kong University of Science and Technology (HKUST), where he is currently chair professor. He held visiting positions at Harvard University, University of Maryland at College Park, AT&T Labs, Tsinghua University, and other universities.
Dr. Cao owns three patents in data- and tele- communications and published three books in the area of stochastic learning and optimization and discrete event dynamic systems: "Stochastic Learning and Optimization - A Sensitivity-Based Approach," Springer, 2007, “Realization Probabilities - the Dynamics of Queuing Systems,” Springer Verlag, 1994, and “Perturbation Analysis of Discrete-Event Dynamic Systems,” Kluwer Academic Publishers, 1991 (co-authored with Y. C. Ho). He received the Outstanding Transactions Paper Award from the IEEE Control System Society in 1987, the Outstanding Publication Award from the Institution of Management Science in 1990, and the Outstanding Service Award from IFAC in 2008. He was elected as a Fellow of IEEE in 1995, and as a Fellow of IFAC in 2008. He is Editor-in-Chief of Discrete Event Dynamic Systems: Theory and Applications, Associate Editor at Large of IEEE Transactions of Automatic Control. He served as the Chairman of IEEE Fellow Evaluation Committee of IEEE Control System Society (2005-2007), and member on the Board of Governors of IEEE Control Systems Society. He is the chairman of IFAC Coordinating Committee on Systems and Signals (2206-2011) and on the Technical Board of IFAC, He is/was associate editor of a number of international journals and chairman of a few technical committees of international professional societies. His current research areas include discrete event dynamic systems, stochastic learning and optimisation, performance analysis of communication systems, signal processing, and financial engineering.