Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

修复部分 公式/图片 无法显示 #768

Merged
merged 17 commits into from
Mar 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .github/workflows/Deploy_website.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,11 @@ name: CI
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ master ]
pull_request:
branches: [ master ]
# push:
# branches: [ master ]
# pull_request:
# branches: [ master ]
workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
Expand Down Expand Up @@ -36,7 +37,7 @@ jobs:

# Installation dependency
- run: pip install git+https://github.com/IMSUVEN/mkdocs.git
- run: pip install mkdocs-material==7.1.8 mkdocs-material-extensions==1.0.1 pymdown-extensions==8.2 jieba==0.42.1
- run: pip install mkdocs-material==7.1.8 mkdocs-material-extensions==1.0.1 pymdown-extensions==8.2 jieba==0.42.1 pygments==2.11.0

# Deploy website
- run: mkdocs gh-deploy --force
32 changes: 16 additions & 16 deletions 基础教程/A2-神经网络基本原理/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,22 +56,22 @@

|网络结构名称|网络结构图|应用领域|
|---|----|----|
|单入<br>单出<br>一层|<img src="img/setup1.png"/>|一元线性回归|
|多入<br>单出<br>一层|<img src="img/setup2.png"/>|多元线性回归|
|多入<br>单出<br>一层|<img src="img/BinaryClassifierNN.png"/>|线性二分类<br>|
|多入<br>多出<br>一层|<img src="img/MultipleClassifierNN.png"/>|线性多分类<br>|
|单入<br>单出<br>两层|<img src="img/nn.png"/>|一元非线性回归/拟合<br>可以拟合任意复杂函数|
|多入<br>单出<br>两层|<img src="img/xor_nn.png"/>|非线性二分类|
|多入<br>多出<br>两层|<img src="img/nn11.png"/>|非线性多分类|
|多入<br>多出<br>三层|<img src="img/nn3.png"/>|非线性多分类|
|多层全连接网络|<img src="img/mnist_net.png"/>|非线性多分类|
|带批归一化层的多层全连接网络|<img src="img/bn_mnist.png"/>|非线性多分类|
|带丢弃层的多层全连接网络|<img src="img/dropout_net.png"/>|非线性多分类|
|简单的卷积神经网络|<img src="img/conv_net.png"/>|非线性多分类|
|复杂的卷积神经网络|<img src="img/mnist_net18.png"/>|非线性多分类|
|单向循环神经网络|<img src="img/bptt_simple.png"/>|非线性多分类|
|双向循环神经网络|<img src="img/bi_rnn_net_right.png"/>|非线性多分类|
|深度循环神经网络|<img src="img/deep_rnn_net.png"/>|非线性多分类|
|单入<br>单出<br>一层|<img src="./img/setup1.png"/>|一元线性回归|
|多入<br>单出<br>一层|<img src="./img/setup2.png"/>|多元线性回归|
|多入<br>单出<br>一层|<img src="./img/BinaryClassifierNN.png"/>|线性二分类<br>|
|多入<br>多出<br>一层|<img src="./img/MultipleClassifierNN.png"/>|线性多分类<br>|
|单入<br>单出<br>两层|<img src="./img/nn.png"/>|一元非线性回归/拟合<br>可以拟合任意复杂函数|
|多入<br>单出<br>两层|<img src="./img/xor_nn.png"/>|非线性二分类|
|多入<br>多出<br>两层|<img src="./img/nn11.png"/>|非线性多分类|
|多入<br>多出<br>三层|<img src="./img/nn3.png"/>|非线性多分类|
|多层全连接网络|<img src="./img/mnist_net.png"/>|非线性多分类|
|带批归一化层的多层全连接网络|<img src="./img/bn_mnist.png"/>|非线性多分类|
|带丢弃层的多层全连接网络|<img src="./img/dropout_net.png"/>|非线性多分类|
|简单的卷积神经网络|<img src="./img/conv_net.png"/>|非线性多分类|
|复杂的卷积神经网络|<img src="./img/mnist_net18.png"/>|非线性多分类|
|单向循环神经网络|<img src="./img/bptt_simple.png"/>|非线性多分类|
|双向循环神经网络|<img src="./img/bi_rnn_net_right.png"/>|非线性多分类|
|深度循环神经网络|<img src="./img/deep_rnn_net.png"/>|非线性多分类|

## 写在后面

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -138,9 +138,11 @@ $$A=\sigma{(Z)}$$
$$
z1_1 = x_1 \cdot w1_{1,1}+ x_2 \cdot w1_{2,1}+b1_1
$$

$$
z1_2 = x_1 \cdot w1_{1,2}+ x_2 \cdot w1_{2,2}+b1_2
$$

$$
z1_3 = x_1 \cdot w1_{1,3}+ x_2 \cdot w1_{2,3}+b1_3
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -139,12 +139,15 @@ $$
$$
\frac{\partial{z}}{\partial{x}}=\frac{\partial{}}{\partial{x}}(x \cdot y)=y=9
$$

$$
\frac{\partial{z}}{\partial{y}}=\frac{\partial{}}{\partial{y}}(x \cdot y)=x=18
$$

$$
\frac{\partial{x}}{\partial{b}}=\frac{\partial{}}{\partial{b}}(2w+3b)=3
$$

$$
\frac{\partial{y}}{\partial{b}}=\frac{\partial{}}{\partial{b}}(2b+1)=2
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ W=
w_{1} \\\\ w_{2} \\\\ w_{3}
\end{pmatrix}
$$

$$
Y=W^{\top}X+B=
\begin{pmatrix}
Expand All @@ -126,6 +127,7 @@ x_{3}
\end{pmatrix}
+b
$$

$$
=w_1 \cdot x_1 + w_2 \cdot x_2 + w_3 \cdot x_3 + b \tag{4}
$$
Expand Down Expand Up @@ -168,6 +170,7 @@ x_{3}
\end{pmatrix}
+b
$$

$$
=w_1 \cdot x_1 + w_2 \cdot x_2 + w_3 \cdot x_3 + b \tag{5}
$$
Expand Down Expand Up @@ -206,6 +209,7 @@ w_{3}
\end{pmatrix}
+b
$$

$$
=x_1 \cdot w_1 + x_2 \cdot w_2 + x_3 \cdot w_3 + b \tag{6}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ $$z_i \simeq y_i \tag{2}$$
其中,$x_i$ 是样本特征值,$y_i$ 是样本标签值,$z_i$ 是模型预测值。

如何学得 $w$ 和 $b$ 呢?均方差(MSE - mean squared error)是回归任务中常用的手段:

$$
J = \frac{1}{2m}\sum_{i=1}^m(z_i-y_i)^2 = \frac{1}{2m}\sum_{i=1}^m(y_i-wx_i-b)^2 \tag{3}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ $$
#### 计算z的梯度

根据公式2:

$$
\frac{\partial loss}{\partial z_i}=z_i - y_i \tag{3}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ $$
$$

其中:

$$
X =
\begin{pmatrix}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,6 @@ $repeat \lbrace \\\\
\quad \quad b=b-\eta \cdot db \\\\
\quad \rbrace \\\\
\rbrace$

***

#### 特点
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,9 +108,11 @@ $$
$$
\frac{\partial J}{\partial W}=2X^{\top}XW - 2X^{\top}Y=0 \tag{14}
$$

$$
X^{\top}XW = X^{\top}Y \tag{15}
$$

$$
W=(X^{\top}X)^{-1}X^{\top}Y \tag{16}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ $B$ 是个单值,因为输出层只有一个神经元,所以只有一个bias
#### 输出层

由于我们只想完成一个回归(拟合)任务,所以输出层只有一个神经元。由于是线性的,所以没有用激活函数。

$$
\begin{aligned}
Z&=
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ z = x_1' w_1' + x_2' w_2' + b' \tag{z是预测值}
$$

由于训练时标签值(房价)并没有做标准化,意味着我们是用真实的房价做的训练,所以预测值和标签值应该相等,所以:

$$
y = z
$$
Expand All @@ -86,6 +87,7 @@ x_1 w_1 + x_2 w_2 + b = x_1' w_1' + x_2' w_2' + b' \tag{1}
$$

标准化的公式是:

$$
x' = \frac{x - x_{min}}{x_{max}-x_{min}} \tag{2}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ $$
\frac{\partial loss}{\partial w_2}
\end{pmatrix}
$$

$$
=\begin{pmatrix}
\frac{\partial loss}{\partial z}\frac{\partial z}{\partial w_1} \\\\
Expand All @@ -95,6 +96,7 @@ $$
(a-y)x_2
\end{pmatrix}
$$

$$
=(x_1 \ x_2)^{\top} (a-y) \tag{4}
$$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,11 @@ $$Z = W*A+b \tag{0}$$
分解到每一项就是下列公式:

$$z_{11} = w_{11} \cdot a_{11} + w_{12} \cdot a_{12} + w_{21} \cdot a_{21} + w_{22} \cdot a_{22} + b \tag{1}$$

$$z_{12} = w_{11} \cdot a_{12} + w_{12} \cdot a_{13} + w_{21} \cdot a_{22} + w_{22} \cdot a_{23} + b \tag{2}$$

$$z_{21} = w_{11} \cdot a_{21} + w_{12} \cdot a_{22} + w_{21} \cdot a_{31} + w_{22} \cdot a_{32} + b \tag{3}$$

$$z_{22} = w_{11} \cdot a_{22} + w_{12} \cdot a_{23} + w_{21} \cdot a_{32} + w_{22} \cdot a_{33} + b \tag{4}$$

求损失函数$J$对$a_{11}$的梯度:
Expand All @@ -50,6 +53,7 @@ $$
$$
\frac{\partial J}{\partial a_{22}}=\frac{\partial J}{\partial z_{11}} \frac{\partial z_{11}}{\partial a_{22}}+\frac{\partial J}{\partial z_{12}} \frac{\partial z_{12}}{\partial a_{22}}+\frac{\partial J}{\partial z_{21}} \frac{\partial z_{21}}{\partial a_{22}}+\frac{\partial J}{\partial z_{22}} \frac{\partial z_{22}}{\partial a_{22}}
$$

$$
=\delta_{z11} \cdot w_{22} + \delta_{z12} \cdot w_{21} + \delta_{z21} \cdot w_{12} + \delta_{z22} \cdot w_{11} \tag{7}
$$
Expand Down Expand Up @@ -132,13 +136,19 @@ $$
正向公式:

$$z_{111} = w_{111} \cdot a_{11} + w_{112} \cdot a_{12} + w_{121} \cdot a_{21} + w_{122} \cdot a_{22}$$

$$z_{112} = w_{111} \cdot a_{12} + w_{112} \cdot a_{13} + w_{121} \cdot a_{22} + w_{122} \cdot a_{23}$$

$$z_{121} = w_{111} \cdot a_{21} + w_{112} \cdot a_{22} + w_{121} \cdot a_{31} + w_{122} \cdot a_{32}$$

$$z_{122} = w_{111} \cdot a_{22} + w_{112} \cdot a_{23} + w_{121} \cdot a_{32} + w_{122} \cdot a_{33}$$

$$z_{211} = w_{211} \cdot a_{11} + w_{212} \cdot a_{12} + w_{221} \cdot a_{21} + w_{222} \cdot a_{22}$$

$$z_{212} = w_{211} \cdot a_{12} + w_{212} \cdot a_{13} + w_{221} \cdot a_{22} + w_{222} \cdot a_{23}$$

$$z_{221} = w_{211} \cdot a_{21} + w_{212} \cdot a_{22} + w_{221} \cdot a_{31} + w_{222} \cdot a_{32}$$

$$z_{222} = w_{211} \cdot a_{22} + w_{212} \cdot a_{23} + w_{221} \cdot a_{32} + w_{222} \cdot a_{33}$$

求$J$对$a_{22}$的梯度:
Expand Down Expand Up @@ -176,18 +186,21 @@ z_{11} &= w_{111} \cdot a_{111} + w_{112} \cdot a_{112} + w_{121} \cdot a_{121}
\end{aligned}
\tag{10}
$$

$$
\begin{aligned}
z_{12} &= w_{111} \cdot a_{112} + w_{112} \cdot a_{113} + w_{121} \cdot a_{122} + w_{122} \cdot a_{123} \\\\
&+ w_{211} \cdot a_{212} + w_{212} \cdot a_{213} + w_{221} \cdot a_{222} + w_{222} \cdot a_{223}
\end{aligned}\tag{11}
$$

$$
\begin{aligned}
z_{21} &= w_{111} \cdot a_{121} + w_{112} \cdot a_{122} + w_{121} \cdot a_{131} + w_{122} \cdot a_{132} \\\\
&+ w_{211} \cdot a_{221} + w_{212} \cdot a_{222} + w_{221} \cdot a_{231} + w_{222} \cdot a_{232}
\end{aligned}\tag{12}
$$

$$
\begin{aligned}
z_{22} &= w_{111} \cdot a_{122} + w_{112} \cdot a_{123} + w_{121} \cdot a_{132} + w_{122} \cdot a_{133} \\\\
Expand Down Expand Up @@ -321,6 +334,7 @@ $$
$$
z = x * w
$$

$$
loss = \frac{1}{2}(z-y)^2
$$
Expand Down