机器学习基石 - Logistic Regression

机器学习基石下 (Machine Learning Foundations)—Mathematical Foundations
Hsuan-Tien Lin, 林轩田，副教授 (Associate Professor)，资讯工程学系 (Computer Science and Information Engineering)

Logistic Regression Problem

• logistic hypothesis: h(x)=θ(wTx)$h(\mathbf{x})=\theta ({\mathbf{w}}^T\mathbf{x})$
• logistic function θ(s)=11+e−s$\theta (s)=\frac{1}{1+e^{-s}}$
• sign(h(x)−0.5)=sign(wTx)$sign(h(\mathbf{x})-0.5)=sign({\mathbf{w}}^T\mathbf{x})$

Logistic Regression Error

• Likelihood of Logistic Hypothesis

  

  极大似然

• cross-entropy error

  

  在极大似然估计下，logistic方程的误差函数

Gradient of Logistic Regression Error

• Minimizing Ein(w)$E_{in}(\mathbf{w})$

  

• PLA Revisited: Iterative Optimization

  

Gradient Descent

• Linear Approximation

  

• Simple Heuristic for Changing η$\eta$

  

• Logistic Regression Algorithm