MSML604

Prerequisite: undergraduate courses in calculus and basic linear algebra. The course focuses on recognizing, solving, and analyzing optimization problems. Linear algebra overview: vector spaces and matrices, linear transformations, matrix algebra, projections, similarity transformations, norms, eigen-decomposition and SVD. Convex sets, convex functions, duality theory and optimality conditions. Unconstrained optimization: 1D search, steepest descent, Newton's method, conjugate gradient method, DFP and BFGS methods, stochastic gradient descent. Constrained optimization: projected gradient methods, linear programming, quadratic programming, penalty functions, and interior-point methods. Global search methods: simulated annealing, genetic algorithms, particle swarm optimization.