Lecture Plan

IIA2217 System Identification and Optimal Estimation

Week nr. Date Topics
2 10.1-12.1 Introduction to system identification. Notations, state space models, realization theory, hankel matrix, impulse response matrices. Identification of impulse responses. Hankel matrices and system order.

Lecture 1: Lecture notes Ch.1, Section 2.2.5,
Exercise 1Exercise 1.        Solution 1.      Matlab script for solution of the numerical part of exercise

Lecture notes on Finite Impulse Response (FIR) models

Video Lectures:  Introduction

prbs1.m for experiment design (download here or on exercise page)

MATLAB example main_ex1_lecture1.m

3 17.1-19.1 Realization theory. Singular value decomposition (SVD). Different state space model realizations; output normal, input normal and balanced realizations. System identification of autonomeous systems.

Lecture 2: Lecture notes: Ch. 1, Sec. 2.2.5, Sec. 4.10
Exercise 2: Taken as Task 1 in exercises.

Video Lectures:      Lecture2a  Lecture2b

                           2019 Lecture2: Problem description

Lecture notes on Finite Impulse Response (FIR) models

Note: A link between impulse responses and discrete z-plane (or q-plane) transfer function models mentioned, as in in Ch. 2.3.7 in MPC_lect_notes

MATLAB example: Realization theory

4 24.1-26.1 PCA, PCR og SVD. System identification of steady state systems. The Least squares (LS) method. Partial Least Squares (PLS) regression. PLS only mentioned and not in details.

Lecture 3: OLS, PCA and SVD, PCR
Exercise 3: Exercises 8 and 7 in Note

Video Lectures: Lecture3a  Lecture3b


5 31.1-2.2 Identification of deterministic systems. Identification of system order, the extended observability matrix of the system and the dynamic properties, i.e. identification of, n, O_L, A and D. Orthogonal projection matrices.

Lecture 4: SID of deterministic systems. Autonomeous systems intro
Exercise 4: Exercises 11 and 12 in  Note Solution proposals for Exercise11/Task 11 here: losn_oppg10.m

Video Lectures: Lecture 4a  Lecture4b
6 7.2-9.2 Identification of combined deterministic and stochastic systems.

Computation of B and E from tilde(B) matrix. How to hande noisy data. Subspace system identification. Little about the Kalman filter.

Lecture5: SID of the general problem, i.e. Combined deterministic and Stochastic Systems. Lecture notes Ch. 3, Ch. 4,  Ch. 8.3.2, 8.3.4

Video Lectures: Lecture 5a   Lecture5b
                          Lecture_combined  Lecture_Kalman_intro



Subspace identification of combined deterministic and stochastic systems, and identification of the innovations process and closed loop systems.
Also theory as described previous week.

1) Closed and Open Loop SID of Kalman Filter. Ch. 6

2) Paper, "On system identification of the Kalman filter..", Lemma 3.9, page 15 and Ch 5 in item 5 in syllabus list and mic-journal paper: http://www.mic-journal.no/ABS/MIC-2009-2-3.asp

Video Lecture: Lecture6a
 MATLAB m-file example, main_clopid_ex1,m

8-9  21.2-23.2  (28.2-1.3) State estimation and the kalman filter for linear systems. Innovations formulations and apriori-aposteriori formulation of the kalman filter for discrete time linear systems.

Lecture notes: State estimation and Kalman filter, Ch. 2.2, 2.6.2, 2.6.3

Video Lectures: Lecture7a  Lecture7b
                          Lecture7_2019 (Continuous Kalman filter, Ch.2.2)

Example: main_oppg13_ext.m

8-9 28.2-1.3


Prof of equation for Kalman gain matrix.

Video Lecture: Lecture8

Exercise: Calculating the Kalman gain matrix. (MATLAB m-file)

Simple proof of Kalman gain matrix.

Exam 2014: Tasks 4 kalman filter. Task 5 and 6. Susbspace SID, shift invariance principle and feedback in data. Details in Ch. 6 in mic-journal paper

MATLAB: Kalman filter example: main_ex_march6.m

6.3-8.3 Prof of some Kalman filter equations. Lecture notes: State estimation and Kalman filter, Ch. 2.6.2, 2.6.3.

State estimation and the Kalman filter for non linear systems, the Extended Kalman Filter (EKF). Lecture notes: Ch. 3

Video Lectures: Lecture9 (The EKF)
11 13.3-15.3 Kalman filter and introduction to prediction error methods for system identification.

Video Lecture: Lecture10 (Introduction to PEM

Exercise : Kalman filter exercise. Work through parts of Kalman filter exercise 2 as described in Exercise plan, Week 12-13, with m-file solution proposals.

Demo Advanced Control IIAV3017: demo1_advanced_control.m dlqdu_pi.m

11 15.3-20.3

Prediction error methods for system identification, parameter estimation and Kalman filter. Polynomial, ARMAX and state space models.

The prediction error method and linear regression models. Ch. 2,3, 2.3.2. (Syllabus from Item 4 in syllabus page. )

Work out the proof of Eq. (55) on page 11.

Video Lecture: Lecture11

Exercise: Curve fit example using MATLAB polyfit.m and polyval.m functions to fit a polynomial to data X and Y, and MATLAB plotting facilities. main_ex_polyfit.m

12  20.3-22.3 TOPICS
1. Prediction error methods. More about polynomial models, ARX, OE, BOX-Jenkins etc. as in Sec 3 in lecture notes. This is also the topic in Week 15.

2. SS-PEM Toolbox for MATLAB, Section 6.1 in lecture notes.

3. A MIMO (m=2 and r=2) system with n=3 states syntetic example. MATLAB m-files linex2n2.m utype.m

4. Estimate State Space (SS) model using D-SR Toolbox for MATLAB and the MIMO example mentioned above.

5. EXAMPLE: SISO 1st order general linear model. State Space model, Linear Regression model, ARMAX and ARX models). Ch3 in syllabus item4
 13  27.3-29.3  The prediction error method and the Ordinary Least Squares (OLS) method. ARX models and the OLS method. Statistical analysis of the  OLS estimate. The Best Linear Unbiased Estimator (BLUE).

The recursive OLS method, Section 7, p. 26 

Video Lecture: Lecture12

Exercise: Illustration of ROLS method Ex. 7.2 with m-file: main_rols_ex.m , prbs1.m


13  Easter
14 3.4-5.4

Input and Output Model structures (polynomial models), lecture notes Ch.3 p. 14. ARX, ARMAX models etc.

Also same as Lecture12 Week13, the Recursive OLS, the ROLS method.

Quote from Ljung (1999): "A high order ARX model is capable of approximating any linear system arbitrarily well."

A view on closed loop subspace system identification, the DSR_e algorithm and MIC paper.

15 10.4-12.4

Topics for Lecture14:

1. How to handle trends.Ch.9

2. Model validation.Ch.10

3. Input experiment design. Ch. 11. MATLAB function prbs1.m to generate Pseudo Random Binary (PRBS) input signal experiments.

The above chapters is in the "subspace" lecture notes.


Exercises: As in ExercisePlan

16 17.4-19.4 EXERCISE
1) Work with earlier exercises
2) System identification by first identification of a higer order ARX model and following by model reduction. MATLAB function: harxmr.m This function is using m-file: hank_m.m

Quote from Ljung (1999): "A high order ARX model is capable of approximating any linear system arbitrarily well."

3) Identification of non-linear systems. Reformulating as a linear regression problem. Lecture notes.

17 25.4-27.4 Summing up the main topics:

1) Realisation theory.

2) Subspace based methods for system identification.

3) Optimal state estimation and the Kalman Filter.

4) Prediction error methods for system identification.

Summary on Web


Teacher: Dr. ing., 1. amanuensis David Di Ruscio                             

Oppdatert: david.di.ruscio@usn.no