Skip to content

Commit

Permalink
修复部分 公式/图片 无法显示 (#768)
Browse files Browse the repository at this point in the history
* Update 04.1-最小二乘法.md

公式修复

* Update 04.2-梯度下降法.md

公式修复

* Update 04.4-多样本计算.md

公式修复

* Update 04.5-梯度下降的三种形式.md

* Update 04.5-梯度下降的三种形式.md

* Update 05.2-神经网络法.md

公式修复

* Update 05.4-还原参数值.md

公式修复

* Update 04.5-梯度下降的三种形式.md

复原

* Update 06.2-线性二分类实现.md

公式修复

* Update 01.3-神经网络的基本工作原理.md

公式修复

* Update 02.1-线性反向传播.md

公式修复

* Update 17.3-卷积的反向传播原理.md

* Update 17.3-卷积的反向传播原理.md

公式修复

* Update README.md

可能解决 #751

* Update 05.1-正规方程法.md

* Update 04.0-单入单出单层-单变量线性回归.md

* Update Deploy_website.yml
  • Loading branch information
sxjeru authored Mar 24, 2023
1 parent d545dc5 commit 76e2ae1
Show file tree
Hide file tree
Showing 14 changed files with 54 additions and 21 deletions.
9 changes: 5 additions & 4 deletions .github/workflows/Deploy_website.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,11 @@ name: CI
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ master ]
pull_request:
branches: [ master ]
# push:
# branches: [ master ]
# pull_request:
# branches: [ master ]
workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
Expand Down
32 changes: 16 additions & 16 deletions 基础教程/A2-神经网络基本原理/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,22 +56,22 @@

|网络结构名称|网络结构图|应用领域|
|---|----|----|
|单入<br>单出<br>一层|<img src="img/setup1.png"/>|一元线性回归|
|多入<br>单出<br>一层|<img src="img/setup2.png"/>|多元线性回归|
|多入<br>单出<br>一层|<img src="img/BinaryClassifierNN.png"/>|线性二分类<br>|
|多入<br>多出<br>一层|<img src="img/MultipleClassifierNN.png"/>|线性多分类<br>|
|单入<br>单出<br>两层|<img src="img/nn.png"/>|一元非线性回归/拟合<br>可以拟合任意复杂函数|
|多入<br>单出<br>两层|<img src="img/xor_nn.png"/>|非线性二分类|
|多入<br>多出<br>两层|<img src="img/nn11.png"/>|非线性多分类|
|多入<br>多出<br>三层|<img src="img/nn3.png"/>|非线性多分类|
|多层全连接网络|<img src="img/mnist_net.png"/>|非线性多分类|
|带批归一化层的多层全连接网络|<img src="img/bn_mnist.png"/>|非线性多分类|
|带丢弃层的多层全连接网络|<img src="img/dropout_net.png"/>|非线性多分类|
|简单的卷积神经网络|<img src="img/conv_net.png"/>|非线性多分类|
|复杂的卷积神经网络|<img src="img/mnist_net18.png"/>|非线性多分类|
|单向循环神经网络|<img src="img/bptt_simple.png"/>|非线性多分类|
|双向循环神经网络|<img src="img/bi_rnn_net_right.png"/>|非线性多分类|
|深度循环神经网络|<img src="img/deep_rnn_net.png"/>|非线性多分类|
|单入<br>单出<br>一层|<img src="./img/setup1.png"/>|一元线性回归|
|多入<br>单出<br>一层|<img src="./img/setup2.png"/>|多元线性回归|
|多入<br>单出<br>一层|<img src="./img/BinaryClassifierNN.png"/>|线性二分类<br>|
|多入<br>多出<br>一层|<img src="./img/MultipleClassifierNN.png"/>|线性多分类<br>|
|单入<br>单出<br>两层|<img src="./img/nn.png"/>|一元非线性回归/拟合<br>可以拟合任意复杂函数|
|多入<br>单出<br>两层|<img src="./img/xor_nn.png"/>|非线性二分类|
|多入<br>多出<br>两层|<img src="./img/nn11.png"/>|非线性多分类|
|多入<br>多出<br>三层|<img src="./img/nn3.png"/>|非线性多分类|
|多层全连接网络|<img src="./img/mnist_net.png"/>|非线性多分类|
|带批归一化层的多层全连接网络|<img src="./img/bn_mnist.png"/>|非线性多分类|
|带丢弃层的多层全连接网络|<img src="./img/dropout_net.png"/>|非线性多分类|
|简单的卷积神经网络|<img src="./img/conv_net.png"/>|非线性多分类|
|复杂的卷积神经网络|<img src="./img/mnist_net18.png"/>|非线性多分类|
|单向循环神经网络|<img src="./img/bptt_simple.png"/>|非线性多分类|
|双向循环神经网络|<img src="./img/bi_rnn_net_right.png"/>|非线性多分类|
|深度循环神经网络|<img src="./img/deep_rnn_net.png"/>|非线性多分类|

## 写在后面

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -138,9 +138,11 @@ $$A=\sigma{(Z)}$$
$$
z1_1 = x_1 \cdot w1_{1,1}+ x_2 \cdot w1_{2,1}+b1_1
$$

$$
z1_2 = x_1 \cdot w1_{1,2}+ x_2 \cdot w1_{2,2}+b1_2
$$

$$
z1_3 = x_1 \cdot w1_{1,3}+ x_2 \cdot w1_{2,3}+b1_3
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -139,12 +139,15 @@ $$
$$
\frac{\partial{z}}{\partial{x}}=\frac{\partial{}}{\partial{x}}(x \cdot y)=y=9
$$

$$
\frac{\partial{z}}{\partial{y}}=\frac{\partial{}}{\partial{y}}(x \cdot y)=x=18
$$

$$
\frac{\partial{x}}{\partial{b}}=\frac{\partial{}}{\partial{b}}(2w+3b)=3
$$

$$
\frac{\partial{y}}{\partial{b}}=\frac{\partial{}}{\partial{b}}(2b+1)=2
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ W=
w_{1} \\\\ w_{2} \\\\ w_{3}
\end{pmatrix}
$$

$$
Y=W^{\top}X+B=
\begin{pmatrix}
Expand All @@ -126,6 +127,7 @@ x_{3}
\end{pmatrix}
+b
$$

$$
=w_1 \cdot x_1 + w_2 \cdot x_2 + w_3 \cdot x_3 + b \tag{4}
$$
Expand Down Expand Up @@ -168,6 +170,7 @@ x_{3}
\end{pmatrix}
+b
$$

$$
=w_1 \cdot x_1 + w_2 \cdot x_2 + w_3 \cdot x_3 + b \tag{5}
$$
Expand Down Expand Up @@ -206,6 +209,7 @@ w_{3}
\end{pmatrix}
+b
$$

$$
=x_1 \cdot w_1 + x_2 \cdot w_2 + x_3 \cdot w_3 + b \tag{6}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ $$z_i \simeq y_i \tag{2}$$
其中,$x_i$ 是样本特征值,$y_i$ 是样本标签值,$z_i$ 是模型预测值。

如何学得 $w$ 和 $b$ 呢?均方差(MSE - mean squared error)是回归任务中常用的手段:

$$
J = \frac{1}{2m}\sum_{i=1}^m(z_i-y_i)^2 = \frac{1}{2m}\sum_{i=1}^m(y_i-wx_i-b)^2 \tag{3}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ $$
#### 计算z的梯度

根据公式2:

$$
\frac{\partial loss}{\partial z_i}=z_i - y_i \tag{3}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ $$
$$

其中:

$$
X =
\begin{pmatrix}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,6 @@ $repeat \lbrace \\\\
\quad \quad b=b-\eta \cdot db \\\\
\quad \rbrace \\\\
\rbrace$

***

#### 特点
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,9 +108,11 @@ $$
$$
\frac{\partial J}{\partial W}=2X^{\top}XW - 2X^{\top}Y=0 \tag{14}
$$

$$
X^{\top}XW = X^{\top}Y \tag{15}
$$

$$
W=(X^{\top}X)^{-1}X^{\top}Y \tag{16}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ $B$ 是个单值,因为输出层只有一个神经元,所以只有一个bias
#### 输出层

由于我们只想完成一个回归(拟合)任务,所以输出层只有一个神经元。由于是线性的,所以没有用激活函数。

$$
\begin{aligned}
Z&=
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ z = x_1' w_1' + x_2' w_2' + b' \tag{z是预测值}
$$

由于训练时标签值(房价)并没有做标准化,意味着我们是用真实的房价做的训练,所以预测值和标签值应该相等,所以:

$$
y = z
$$
Expand All @@ -86,6 +87,7 @@ x_1 w_1 + x_2 w_2 + b = x_1' w_1' + x_2' w_2' + b' \tag{1}
$$

标准化的公式是:

$$
x' = \frac{x - x_{min}}{x_{max}-x_{min}} \tag{2}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ $$
\frac{\partial loss}{\partial w_2}
\end{pmatrix}
$$

$$
=\begin{pmatrix}
\frac{\partial loss}{\partial z}\frac{\partial z}{\partial w_1} \\\\
Expand All @@ -95,6 +96,7 @@ $$
(a-y)x_2
\end{pmatrix}
$$

$$
=(x_1 \ x_2)^{\top} (a-y) \tag{4}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,11 @@ $$Z = W*A+b \tag{0}$$
分解到每一项就是下列公式:

$$z_{11} = w_{11} \cdot a_{11} + w_{12} \cdot a_{12} + w_{21} \cdot a_{21} + w_{22} \cdot a_{22} + b \tag{1}$$

$$z_{12} = w_{11} \cdot a_{12} + w_{12} \cdot a_{13} + w_{21} \cdot a_{22} + w_{22} \cdot a_{23} + b \tag{2}$$

$$z_{21} = w_{11} \cdot a_{21} + w_{12} \cdot a_{22} + w_{21} \cdot a_{31} + w_{22} \cdot a_{32} + b \tag{3}$$

$$z_{22} = w_{11} \cdot a_{22} + w_{12} \cdot a_{23} + w_{21} \cdot a_{32} + w_{22} \cdot a_{33} + b \tag{4}$$

求损失函数$J$对$a_{11}$的梯度:
Expand All @@ -50,6 +53,7 @@ $$
$$
\frac{\partial J}{\partial a_{22}}=\frac{\partial J}{\partial z_{11}} \frac{\partial z_{11}}{\partial a_{22}}+\frac{\partial J}{\partial z_{12}} \frac{\partial z_{12}}{\partial a_{22}}+\frac{\partial J}{\partial z_{21}} \frac{\partial z_{21}}{\partial a_{22}}+\frac{\partial J}{\partial z_{22}} \frac{\partial z_{22}}{\partial a_{22}}
$$

$$
=\delta_{z11} \cdot w_{22} + \delta_{z12} \cdot w_{21} + \delta_{z21} \cdot w_{12} + \delta_{z22} \cdot w_{11} \tag{7}
$$
Expand Down Expand Up @@ -132,13 +136,19 @@ $$
正向公式:

$$z_{111} = w_{111} \cdot a_{11} + w_{112} \cdot a_{12} + w_{121} \cdot a_{21} + w_{122} \cdot a_{22}$$

$$z_{112} = w_{111} \cdot a_{12} + w_{112} \cdot a_{13} + w_{121} \cdot a_{22} + w_{122} \cdot a_{23}$$

$$z_{121} = w_{111} \cdot a_{21} + w_{112} \cdot a_{22} + w_{121} \cdot a_{31} + w_{122} \cdot a_{32}$$

$$z_{122} = w_{111} \cdot a_{22} + w_{112} \cdot a_{23} + w_{121} \cdot a_{32} + w_{122} \cdot a_{33}$$

$$z_{211} = w_{211} \cdot a_{11} + w_{212} \cdot a_{12} + w_{221} \cdot a_{21} + w_{222} \cdot a_{22}$$

$$z_{212} = w_{211} \cdot a_{12} + w_{212} \cdot a_{13} + w_{221} \cdot a_{22} + w_{222} \cdot a_{23}$$

$$z_{221} = w_{211} \cdot a_{21} + w_{212} \cdot a_{22} + w_{221} \cdot a_{31} + w_{222} \cdot a_{32}$$

$$z_{222} = w_{211} \cdot a_{22} + w_{212} \cdot a_{23} + w_{221} \cdot a_{32} + w_{222} \cdot a_{33}$$

求$J$对$a_{22}$的梯度:
Expand Down Expand Up @@ -176,18 +186,21 @@ z_{11} &= w_{111} \cdot a_{111} + w_{112} \cdot a_{112} + w_{121} \cdot a_{121}
\end{aligned}
\tag{10}
$$

$$
\begin{aligned}
z_{12} &= w_{111} \cdot a_{112} + w_{112} \cdot a_{113} + w_{121} \cdot a_{122} + w_{122} \cdot a_{123} \\\\
&+ w_{211} \cdot a_{212} + w_{212} \cdot a_{213} + w_{221} \cdot a_{222} + w_{222} \cdot a_{223}
\end{aligned}\tag{11}
$$

$$
\begin{aligned}
z_{21} &= w_{111} \cdot a_{121} + w_{112} \cdot a_{122} + w_{121} \cdot a_{131} + w_{122} \cdot a_{132} \\\\
&+ w_{211} \cdot a_{221} + w_{212} \cdot a_{222} + w_{221} \cdot a_{231} + w_{222} \cdot a_{232}
\end{aligned}\tag{12}
$$

$$
\begin{aligned}
z_{22} &= w_{111} \cdot a_{122} + w_{112} \cdot a_{123} + w_{121} \cdot a_{132} + w_{122} \cdot a_{133} \\\\
Expand Down Expand Up @@ -321,6 +334,7 @@ $$
$$
z = x * w
$$

$$
loss = \frac{1}{2}(z-y)^2
$$
Expand Down

0 comments on commit 76e2ae1

Please sign in to comment.