机器学习基石 - The Learning Problem

机器学习基石上 (Machine Learning Foundations)—Mathematical Foundations
Hsuan-Tien Lin, 林轩田，副教授 (Associate Professor)，资讯工程学系 (Computer Science and Information Engineering)

The Learning Problem

What Is Machine Learning

• observation → learning → skill
• data → ML → skill
• skill ↔ improve some performance measure
• machine learning: improve some performance measure with experience computed from data
• ML: an alternative route to build complicated system
  
  • some use scenarios
    

Key Essence of Machine Learning

• 具有潜在的模式
• 不能简单的编写出来程序
• 大量的数据

Examples

Applications of Machine Learning

• Food, Housing, Transport. Clothes. Education, Entertainment
• Learn our performances
• Examples
  

Components of Machine Learning

• input: x∈X$x\in \mathcal{X}$
• output: y∈Y$y\in \mathcal{Y}$
• unknown pattern to be learnt (target function): f:X→Y$f:\mathcal{X}\to \mathcal{Y}$
• data (training examples): D={(x1,y1),(x2,y2),...,(xN,yN)}$\mathcal{D}=\{(x_1,y_1),(x_2,y_2),...,(x_N,y_N)\}$
• hypothesis (skill with hopefully good performance): g:X→Y$g:\mathcal{X}\to \mathcal{Y}$
• {(xn,yn)} from f→ML→g$\{(x_n,y_n)\}fromf\to ML\to g$ , f$f$ 是真正的模式，但学不到，g$g$ 是学习之后给出的模式
• 
  
  • f$f$ is unknown (no programmable definition)
  • 希望 g$g$ 和 f$f$ 尽量接近（假设函数和目标函数）
  • assume g∈H={hk}$g\in \mathcal{H}=\{h_k\}$, hypothesis set H$\mathcal{H}$ can contain good or bad hypotheses
  • learning algorithm A$\mathcal{A}$ to pick the ‘best’ one as g$g$
• machine learning: use data to compute hypothesis g$g$ that approximates target f$f$

Machine Learning and Other Fields

Machine Learning and Data Mining

• 资料勘探即数据挖掘 (Data Mining)

  use huge data to find property that is interesting

• if ‘interesting property’ same as ‘hypothesis that approximate target’ —— ML = DM

• if ‘interesting property’ related to ‘hypothesis that approximate target’ —— DM can help ML, and vice versa (but not always)

• traditional DM also focuses on efficient computation in large database

Machine Learning and Artificial Intelligence

• 人工智能 (Artificial Intelligence)

  compute something that shows intelligent behavior

• ML is one possible route to realize AI (机器学习是实现人工智能的一种方法)

• g≈f$g\approx f$ is something that shows intelligent behavior

Machine Learning and Statistics

• 统计学 (Statistics)

  use data to make inference(推断) about an unknown process

• g$g$ is an inference outcome while f$f$ is something unknown

• statistics can be used to achieve ML (统计是实现机器学习的一种方法)

• traditional statistics also focus on provable results with math assumptions (侧重数学上的推论), and care less about computation

• In statistics, there are many useful tools for ML