[NN]前向神经网络总结 Feedforward Neural Network

前言

由于我时间实在不充裕，没能提供中文翻译，给观众老爷造成的不便本人深感抱歉，那就给观众老爷表演个大石碎胸口！

正文

思维导图原址请点这里

1. **函数Activation Function:

• h = ϕ ( W x ) h = \phi(Wx) h=ϕ(Wx)
• f ( h ) = W T h f(h)=W^{T}h f(h)=WTh

2. Mini-Batch:

j ( θ ) ≈ j ~ ( θ ) = 1 M ∑ j ∈ B ( y i − f ( x i ) ) 2 j(\theta) \approx \tilde{j}(\theta)= \frac{1}{M} \sum_{j \in B}(y_{i}-f(x_{i}))^2 j(θ)≈j~​(θ)=M1​j∈B∑​(yi​−f(xi​))2

3. 计算图 Computational Graph:

4. 链式法则 Chain Rule:

d z d θ = ∑ j d z d u j ⋅ d u j d θ i \frac{dz}{d\theta}=\sum_{j}\frac{dz}{du_{j}}·\frac{du_{j}}{d\theta_{i}} dθdz​=j∑​duj​dz​⋅dθi​duj​​

5. ReLU**函数：

ϕ ( s ) = M a x ( 0 , s ) \phi(s)=Max(0, s) ϕ(s)=Max(0,s)

各位观众老爷，给个赞再走吧~~