【机器学习笔记】吴恩达公开课·第七章 Logistic 回归

假设陈述

• h(x) : estimated probability that y = 1 on input x
  $\large h_\theta(x) = P(y=1\,|\,x;\theta)= g(\theta^Tx)$hθ​(x)=P(y=1∣x;θ)=g(θTx)
• sigmod 函数
  $\large g(z) = \frac{1}{1+e^{-z}}$g(z)=1+e−z1​
  

决策界限 Decision Boundary

$\large y= \begin{cases} 1, &amp; { h_\theta (x) \geq 0.5 \, or \, h_\theta(x) \geq 0 } \\\ 0, &amp; h_\theta (x) &lt; 0.5 \,or \, h_\theta(x) &lt; 0 \end{cases}$y=⎩⎨⎧​1, 0,​hθ​(x)≥0.5orhθ​(x)≥0hθ​(x)<0.5orhθ​(x)<0​

• Non-Linear Decision Boundary
  

Logistic regression cost function

• cost
  $\large Cost(h_\theta(x^{(i)},y^{(i)})) = \begin{cases} -\log(h_\theta(x))\quad \text{if y = 1} \\ -\log(1-h_\theta(x))\quad \text{if y = 0} \end{cases}$Cost(hθ​(x(i),y(i)))=⎩⎨⎧​−log(hθ​(x))if y = 1−log(1−hθ​(x))if y = 0​
• simplify version

$\large Cost(h_\theta(x^{(i)},y^{(i)})) = -y\log(h_\theta(x))-(1-y)\log(1-h_\theta(x))$Cost(hθ​(x(i),y(i)))=−ylog(hθ​(x))−(1−y)log(1−hθ​(x))

• $J(\theta)$J(θ)

$\large J(\theta)= -\frac{1}{m}\sum_{i=1}^m Cost(h_\theta(x^{(i)},y^{(i)})) \\ = -\frac{1}{m}[\sum_{i=1}^my^{(i)} \log h_\theta (x^{(i)}) + (1-y^{(i)})\log (1-h_\theta (x^{(i)}))]$J(θ)=−m1​i=1∑m​Cost(hθ​(x(i),y(i)))=−m1​[i=1∑m​y(i)loghθ​(x(i))+(1−y(i))log(1−hθ​(x(i)))]

• to fie parameters $\theta$θ

$\large \min_\theta J(\theta)$θmin​J(θ)

高级优化

Optimization algorithms:

• Gradient descent
• Conjugate gradient
• BFGS
• L-BFGS

Advantages

• No need to manually pick $\alpha$α
• Often faster than gradient descent

Disadvantage

• More complex

Multiclass classification

Train a Logistic regression classifier $h_\theta ^{(i)}(x)$hθ(i)​(x) for each class $i$i to predict the probability that $y=i$y=i.
On a new input $x$x, to maake a prediction, pick the class $i$i that maximizes
$\large \max_i h_\theta^{(i)}(x)$imax​hθ(i)​(x)

$h_\theta ^{(i)}(x) = P(y=i \,|\, x; \theta) \quad (i = 1, 2, 3)$hθ(i)​(x)=P(y=i∣x;θ)(i=1,2,3)