A Trust Region Method for the Optimization of Noisy Functions .
Proposed and implemented a new trust region algorithm under noise; proved for global convergence.
My research focuses on the creation of new algorithms for solving complex optimization problems. Over the years, this research has been motivated by applications as diverse as weather forecasting, engineering design and machine learning. There are always new challenges as scientist and engineers create models of increasing nonlinearity and dimensionality, amid uncertainty.
To test the power of our algorithms and to make them widely available, my group has developed several software packages (some open source and some commercial) that are used in a wide range of applications. They include L-BFGS, KNITRO and L-BFGS-B. My view is that theory, algorithm design, and software are equally important in the creation of new algorithms. This is reflected in the textbook “Numerical Optimization”, which I co-authored with Steve Wright.
In recent years, nonlinear optimization has gained prominence, becoming a foundational element of machine learning and AI. At the heart of machine learning are three fundamental components: a statistical model, typically a neural network; a dataset; and a training phase. During this training phase, an optimization algorithm defines the shape of the neural network to enable the most accurate predictions. The computational cost of the optimization process can be enormous, involving the adjustment of up to a billion parameters. It is imperative to substantially lower costs and decrease energy demands of this process.
Short Bio
Jorge Nocedal is a Professor in the Department of Industrial Engineering and Management Sciences at Northwestern University. He obtained his B.S. degree from UNAM, Mexico, and a PhD from Rice University. His research is in optimization, both deterministic and stochastic, and with emphasis on large-scale problems. He served as editor-in-chief of the SIAM Journal on Optimization, is a SIAM Fellow, was awarded the 2012 George B. Dantzig Prize as well as the 2017 Von Neumann Theory Prize for contributions to theory and algorithms of nonlinear optimization. He is a member of the US National Academy of Engineering.Recent Work
Constrained optimization in the presence of noise.
Devised a stochastic line search algorithm for equality constrained optimization under inaccuracies.
Current Students
Shigeng Sun
Yuchen Lou (co-advised)
Former Students
Melody Xuan, Ph.D. [2023]
Michael Shi, Ph.D. [2022]
Yuchen Xie, Ph.D. [2022]
Raghu (Vijaya Raghavendra) Bollapragada, Ph.D. [2018]
Albert S. Berahas, Ph.D. [2018]
Nitish Shirish Keskar, Ph.D. [2017]
Stefan Solntsev, Ph.D. [2015]
Samantha Hansen, Ph.D. [2014]
Gillian Chin, Ph.D. [2013]
Yuchen Wu, Ph.D. [2010]
Frank Curtis, Ph.D. [2007]
Long Hei, Ph.D. [2007]
Richard Waltz, Ph.D. [2002]
Marcelo Marazzi, Ph.D. [2001]
Marcel Good, M.S. [1999]
Guanghui Liu, Ph.D. [1999]
Mary Beth Hribar, Ph.D. [1995]
Todd Plantega, Ph.D. [1994]
Paul Sally, M.S. [1992]
Marucha Lalee, Ph.D. [1992]
Peihuang Lu, Ph.D. [1992]
James Lee, M.S. [1989]
Dong C. Liu, Ph.D. [1987]
Kathryn Conolly, M.S. [1987]
Susan Mulvey, M.S.[1987]
Denise Skelton, M.S. [1986]