Reports from the Heart of the Machine
EDIT: Boats
The first edition of Introduction to Online Convex Optimization, published in 2016, has become a classic book in the field.
Convex optimization refers to the optimization method of the objective function being convex under the optimization requirement of minimizing (maximizing), and the set of feasible fields formed by the constraint is a convex set optimization method. Due to the ability to sequentially query external data sources, online convex optimization has become the best solution for convex functions and has gained widespread popularity due to its scalability in large-scale optimization and machine learning.
Recently, the classic book Introduction to Online Convex Optimization, written by computer scientist Elad Hazan, has published its second edition, which was published in 2016.
Second edition book link: https://arxiv.org/pdf/1909.05207.pdf
Similar to the first edition, this book treats optimization as a process. In many applications, the actual environment is very complex, and to build a comprehensive theoretical model, it is not feasible to use classical algorithmic theory and mathematical optimization. Therefore, it is necessary to apply optimization methods and learn while observing more aspects of the problem, and adopt a robust approach. This view of optimization as a process has been widely accepted in various fields, with great success in modeling and multiple systems.
With more and more research in machine learning, statistics, decision science, and mathematical optimization, the boundaries between deterministic modeling, stochastic modeling, and optimization methods are blurring. Following this real-world trend, the book uses the online convex optimization (OCO) framework as an example to explain the real-world problems modeled and solved by OCO, covering rigorous definitions, backgrounds, and algorithms.
The book contains 13 chapters:
Chapters 1 and 2 introduce the basics and basic concepts of online convex optimization;
Chapters 3 and 4 systematically introduce two types of online convex optimization methods;
Chapter 5 describes the content of regularization;
Chapter 6 details the classic framework Bandit Convex Optimization (BCO);
Chapter 7 explains the contents of the projectionless algorithm;
Chapter 8 explains online convex optimization theory from the perspective of game theory;
Chapter 9 explains the statistical learning theory related to online convex optimization;
Chapter 10 introduces the practical application of online convex optimization in a realistic and volatile environment;
Chapter 11 mainly introduces the measurement indicators of machine learning algorithm boosting and online convex optimization algorithms;
Chapter 12 explains the online boosting method and its uses;
Chapter 13 introduces the Blackwell accessibility theorem.
Below is a partial table of contents for the book.
About the author
The author of the book Introduction to Online Convex Optimization is Elad Hazan, an American computer scientist and professor in the Department of Computer Science at Princeton University.
Professor Hazan is also one of the authors of the adaptive gradient algorithm AdaGrad. He is mainly engaged in machine learning and mathematical optimization, especially the automation of learning mechanisms and the implementation of effective algorithms, and holds several patents.