Functional Analysis 3 -- Continuous Functions

This is course note from Siran Li-Shanghai Jiao Tong University.

Functional Analysis 2 -- Bounded Linear Operator

This is course note from Siran Li-Shanghai Jiao Tong University.

Functional Analysis 1 -- Banach Space

This is course note from Siran Li-Shanghai Jiao Tong University.

Dynamic Programming Classification

The key to solve DP:

View temporary status as the "sum" of previous status.

Reimplementation of Math Operations

This post records math operation between numbers and stirngs reimplemented on Leetcode.

Perspectives on Natural Gradient Descent

In our article Optimization I - Gradient Descent, we talk about how to view the gradient descent as the approximation of Riemannian gradient descent. However, is there a descent method more loylal to the Riemannian gradient descent? Or in other word, better than gradient descent in some cases. In this artile, we will start our journey to the beautiful and attracting information geometry world. In that abstract and fancy world, the secrete equivalence between several gradient descent methods are discovered.

Perspectives on Accelerated Gradient Descents

Last post we talk about several different perspectives on gradient descent. It seems that gradient descent(GD) is a fairly useful algorithm to numerically solve optimization problem. However, things may become impractical when it comes to reality. One major problem is the convergence rate.

Perspectives on Gradient Descent

When talking about numerical solution to optimization problems, nobody can avoid the gradient descent method created by Cauchy in 1847. In this blog, we try to understand this first order algorithm through different perspectives. \[ x_{k+1} = x_{k} - \alpha \nabla f(x_k) \]

This note records projects finished during learning Rust and valuable notes.