目录

1. Multiple Features

1.1 Multiple features

1.2 Gradient descent for multiple varibales

1.3 Gradient descent in practice I：Feature scaling

1.4 Gradient descent in practice II: Learning rate

1.5 Summary

1.6 Features and polynomial regression

2. Computing Parameters Analytically

2.1 Normal Equation

2.2 Normal Equation Nonivertibility

---

1. Multiple Features

1.1 Multiple features

Size	Number of bedrooms	Number of floors	Age of home	Price
2104	5	1	45	460
1416	3	2	40	232
1534	3	2	30	315
852	2	1	36	178
X1	X2	X3	X4	y

Notation:

n = number of features

x^(i) = ith training example

x^(i)_(j) = value of feature j in ith training example

E.g:

x^(2) = [1416; 3; 2; 40]

Hypothesis:

vectorization:

令x0 = 1

这就叫做多元线性回归

 

1.2 Gradient descent for multiple varibales

Hypothesis: 

parameters: 

cost function:

Gradient descent:

simultaneously updata for every j=0,1...n

即

 

1.3 Gradient descent in practice I：Feature scaling

Feature scaling 特征缩放：

Idea: Make sure features are on a similar scalar

这样梯度下降就能收敛更快

E.g. x1 = size (0~2000)

        x2 = number of bedrooms (1~5)

引自coursera machine learning week 2 Gradient descent in practice I: Feature scaling

feature scaling:

More generally:

frature scaling:

get every feature into approximately a

x0 = 1可以

0 <= x1 <= 3 非常接近 可以

-100 <= x2 <= 100 须scaling

-0.0001 <= x3 <= 0.0001 须scaling

一般地，一个特征在

-3 to 3

- 1/3 to 1/3 都是可以的

另一种缩放叫归一化

 

1.4 Gradient descent in practice II: Learning rate

Gradient descent:

- "Debuggung": How to make sure gradient descent is working correctly

- How to choose learning rate

 

making sure gradient descent is working correctly

plot J(θ) - No. of iterations

引自coursera machine learning week 2 Gradient descent in practice II: Learning rate

J(θ) should decrease after every iteration

还能判断是否收敛

Example automatic converage test:

declare convergence if J(θ) decreases by less than 10^(-3) in one iteration

最好看图判断，因为阈值很难选取

如果收敛图是下面这样：

引自coursera machine learning week 2 Gradient descent in practice II: Learning rate

- For sufficiently small α, J(θ) should decrease on every iteration ------ hold true for linear regression

- But if α is too small, gradient descent can be slow converge

 

1.5 Summary

- If  α is too small: slow convergence

- If  α is too large: J(θ) may not decrease on every iteration; may not converge. (slow converge also possible)

to choose  α, try:

引自coursera machine learning week 2 Gradient descent in practice II: Learning rate

 

1.6 Features and polynomial regression

2. Computing Parameters Analytically

2.1 Normal Equation

A method to solve for θ analytically.(求解析解)

E.g.

J(θ) = aθ^2 + bθ +c, θ belongs to R

J(min) = J(-2a/b)

引自coursera machine learning week 2: Normal Equation

 

Normal Equation:

note: 不需要feature scaling

m training examples, n features

 

Gradient descent	Normal Equation
Need to choose α	No need to choose α
Need many iteration	Don’t need to iteration
works well even when n is large O(n^2)	Need to compute inv(a) O(n^3)
	slow if n is very large

n > 10000, 开始用梯度下降

 

2.2 Normal Equation Nonivertibility

如果不可逆？？？

当1. redundant feature(linearly dependent)

E.g.

x1 = size in feet^2

x2 = size in m^2

而1m = 3.28feet

2. too many features(e.g. m<=n)

时会出现不可逆，实际上当且仅当X的行向量线性无关时，才可逆。

解决方法是删除redundant features， 多余features，or use regularization.

到这里， 第二周的内容已经复习完，其中1.6多项式回归笔者以后会通过专题补充。如果想更多的了解正规方程，请看传送门：MIT线性代数

https://www.bilibili.com/video/av6951511/?p=16