Stochastic control models; development of control laws by dynamic programming; separation of estimation and control; Kalman filtering; self-tuning regulators; dual
controllers; decentralized control.
 

Introduction: decision-making under uncertainty

State space models: state, observation and control processes

Properties of linear stochastic systems: linear Gaussian systems; asymptotic properties, Gauss-Markov processes; quadratic costs

Controlled Markov chain models: finite state systems; Markov and stationary policies; cost of Markov policy; infinite state systems

Input-output models: elimination of state variables; impulse response and frequency response models

Dynamic programming: optimal control laws; complete and partial information; information state, dual control

Estimation and control of linear stochastic systems: linear Gaussian systems; Kalman filter; optimal linear-quadradic control; minimum variance control for input-output models

Identification and adaptive control: Bayesian and non-Bayesian approaches; maximum likelihood estimate; least-squares, prediction error and instrumental variable methods; recursive identification; ODE method; self-tuning regulator

P.R. Kumar and P. Varaiya, Stochastic Systems, Prentice-Hall, 1986.

ECE 415 and ECE 434

1 unit.