机器学习基石 - Linear Models for Classification

机器学习基石下 (Machine Learning Foundations)—Mathematical Foundations
Hsuan-Tien Lin, 林轩田，副教授 (Associate Professor)，资讯工程学系 (Computer Science and Information Engineering)

Linear Models for Binary Classification

• ys$ys$: classification correctness score 分类正确性分数

• Visualizing Error Functions

  

• small ECEin(w)⇒small E0/1out(w)$small{E}_{in}^{CE}(\mathbf{w})\Rightarrow small{E}_{out}^{0/1}(\mathbf{w})$

  logistic/linear reg. for linear classification

• linear regression sometimes used to set w0${\mathbf{w}}_0$ for PLA/pocket/logistic regression

• logistic regression often preferred over pocket

Stochastic Gradient Descent

• stochastic gradient = true gradient + zero-mean ‘noise’ directions
  
• two practical rule-of-thumb:
  
  • stopping condition? t large enough
  • η$\eta$? 0.1 when x in proper range

Multiclass via Logistic Regression

• One Class at a Time Softly 属于某个类别的概率

• One-Versus-All (OVA) Decomposition

  

Multiclass via Binary Classification

• One-versus-one (OVO) Decomposition

  

  voting of classifiers