AI-009: 吴恩达教授（Andrew Ng）的机器学习课程学习笔记38-47
本文是学习Andrew Ng的机器学习系列教程的学习笔记。教学视频地址：
https://study.163.com/course/introduction.htm?courseId=1004570029#/courseDetail?tab=1
38. Neural Networks - Representation - Non-linear hypotheses
Why neural networks?
Simple linear or logistic regression together with adding in maybe the quadratic or the cubic features, that is not the good way to learn complex nonlinear hypotheses when n is large, because just end up with too many features.
for example: Visual recognition
The computation would be very expensive to find and represent all of these features.
39. Neural Networks - Representation - neurons and the brain
mimic 模仿 大脑可以根据接入的信号不同产生不同的功能，比如将听觉区域接入视觉信号，就能'看'见；
cut ear or hand neural and connect eye neural, this part of brain will learn to see.(neuro-rewiring experiments神经重接实验)
try to find out brain’s learning algorithm!
you can plug in almost any sensor to the brain and the brain’s learning algorithm will just figure out how to learn from that data and deal with that data.
40. Neural networks: Representation - model representation I
Nucleus 核
Dendrite 树突 input wires
Cell body细胞体
Node of Ranvier 兰氏节
Axon 轴突 output wire
Myelin sheath 髓鞘
Schwann cell 神经膜细胞
Axon terminal 轴突末梢
one neurons send a little pulse of electricity via its axon to some different neuron’s dendrite.
next show the computational steps that are represented by this diagram.
forward propagation 前向传播
41. Neural Networks - Representation - examples and intuitions
just put a large negative weight in front of the variable you want to negate.
end up with a nonlinear decision boundary 得到一个非线性的决策边界
each layer compute even more complex functions, then the neural networks can deal with complex question.
41. Neural Networks - Representation - Mult-class classification
42. Neural Networks - Learning- Cost function
42. Neural Networks - Learning - back propagation algorithm
back propagation algorithm 反向传播算法
the key is how to compute these partial derivative terms.
关键是如何计算这些偏导项。
43. Neural Networks - Learning - Implementation note: unrolling parameters
How to onvert back and forth between the matrix representation of the parameters versus the vector representation of the parameters.
The advantage of the matrix representation is that when your parameters are stored as matrices it’s more convenient when you’re doing forward propagation and back propagation and it’s easier when your parameters are stored as matrices to take advantage of the sort of, vectorized implementations.
The advantage of the vector representation when you have like thetaVec or DVec is that when you are using the advanced optimization algorithms. Those algorithms tend to assume that you have all of your parameters unrolled into a big long vector.
44. Neural Networks - Learning - Gradient checking
45. Neural Networks - Learning - Random Initialization
all choose 0 will not work
we will only get one features
the epsilon here has no relationship with the epsilon in gradient checking.
46. Neural Network - Learning - Putting it together
47. Neural Networks - Learning - Autonomouse driving example
left top is man and neural network result.
Left bottom is image of road.
ALVINN
see human drive and after 2 minutes will auto drive.
can auto switch one-lade road and two lane-road weight. 单车道、双车道