-
Notifications
You must be signed in to change notification settings - Fork 0
/
search.xml
364 lines (207 loc) · 274 KB
/
search.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>docker1-入门</title>
<link href="/2018/08/31/docker1-%E5%85%A5%E9%97%A8/"/>
<url>/2018/08/31/docker1-%E5%85%A5%E9%97%A8/</url>
<content type="html"><![CDATA[<h2 id="docker组成"><a href="#docker组成" class="headerlink" title="docker组成"></a>docker组成</h2><p>docker 分为<strong>镜像</strong>，<strong>容器</strong>，<strong>仓库</strong>三部分。</p><ul><li>仓库：可以说是类似是github一样的东西，github是存放代码的地方，dockerhub是存放docker镜像的地方</li><li>镜像：可以认为是我们的源代码，可以根据源代码编译成执行文件(容器)</li><li>容器：可以类比是各类软件(软件由代码编译，容器由镜像编译成)，由docker镜像生成。</li></ul><h2 id="docker安装"><a href="#docker安装" class="headerlink" title="docker安装"></a>docker安装</h2><p>docker现在分为ce和ee版本</p><p>ce(community-edition)版本是社区办，ee(enterprise-edition)是企业版</p><table><thead><tr><th>社区版</th><th>企业版基本</th><th>企业版标准</th><th>企业版高级</th><th></th></tr></thead><tbody><tr><td>容器引擎,built in orchestration, 网络, 安全</td><td><code>yes</code></td><td><code>yes</code></td><td><code>yes</code></td><td><code>yes</code></td></tr><tr><td>Docker认证的基础架构，插件和ISV容器</td><td></td><td><code>yes</code></td><td><code>yes</code></td><td><code>yes</code></td></tr><tr><td>image管理(私有docker registry, caching)</td><td>Cloud hosted repos</td><td></td><td><code>yes</code></td><td><code>yes</code></td></tr><tr><td>Docker数据中心(集成容器应用程序管理)</td><td></td><td></td><td><code>yes</code></td><td><code>yes</code></td></tr><tr><td>Docker数据中心(RBAC, LDAP/AD support)</td><td></td><td></td><td><code>yes</code></td><td><code>yes</code></td></tr><tr><td>集成加密管理，镜像签名策略(Integrated secrets mgmt, image signing policy)</td><td></td><td></td><td><code>yes</code></td><td><code>yes</code></td></tr><tr><td>镜像安全扫描(Image security scanning)</td><td>Preview</td><td></td><td></td><td><code>yes</code></td></tr><tr><td>Support</td><td>Community Support</td><td>Business Day or Business Critical</td><td>Business Day or Business Critical</td><td>Business Day or Business Critical</td></tr><tr><td></td><td>免费</td><td>750$/年</td><td>1500$/年</td><td>2000$/年</td></tr></tbody></table><h3 id="Ubuntu-安装-Docker-CE"><a href="#Ubuntu-安装-Docker-CE" class="headerlink" title="Ubuntu 安装 Docker CE"></a>Ubuntu 安装 Docker CE</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo apt-get install docker</span><br></pre></td></tr></table></figure><h3 id="启动-Docker-CE"><a href="#启动-Docker-CE" class="headerlink" title="启动 Docker CE"></a>启动 Docker CE</h3><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> sudo systemctl <span class="built_in">enable</span> docker</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> sudo systemctl start docker</span></span><br></pre></td></tr></table></figure><h3 id="建立-docker-用户组"><a href="#建立-docker-用户组" class="headerlink" title="建立 docker 用户组"></a>建立 docker 用户组</h3><p>默认情况下，<code>docker</code> 命令会使用 <a href="https://en.wikipedia.org/wiki/Unix_domain_socket" target="_blank" rel="noopener">Unix socket</a> 与 Docker 引擎通讯。而只有 <code>root</code> 用户和 <code>docker</code> 组的用户才可以访问 Docker 引擎的 Unix socket。出于安全考虑，一般 Linux 系统上不会直接使用 <code>root</code> 用户。因此，更好地做法是将需要使用 <code>docker</code> 的用户加入 <code>docker</code> 用户组。</p><p>建立 <code>docker</code> 组：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo groupadd docker</span><br></pre></td></tr></table></figure><p>将当前用户加入 <code>docker</code> 组：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo usermod -aG docker $USER</span><br></pre></td></tr></table></figure><h3 id="镜像加速"><a href="#镜像加速" class="headerlink" title="镜像加速"></a>镜像加速</h3><p>鉴于国内网络问题，后续拉取 Docker 镜像十分缓慢，强烈建议安装 Docker 之后配置 <a href="https://yeasy.gitbooks.io/docker_practice/install/mirror.html" target="_blank" rel="noopener">国内镜像加速</a>。</p><p><a href="https://dev.aliyun.com/search.html" target="_blank" rel="noopener">阿里云镜像仓库</a> </p><h2 id="镜像"><a href="#镜像" class="headerlink" title="镜像"></a>镜像</h2><h3 id="获取镜像"><a href="#获取镜像" class="headerlink" title="获取镜像"></a>获取镜像</h3><p>从 Docker 镜像仓库获取镜像的命令是 <code>docker pull</code>。其命令格式为：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">docker pull [选项] [Docker Registry 地址[:端口号]/]仓库名[:标签]</span><br></pre></td></tr></table></figure><p>具体的选项可以通过 <code>docker pull --help</code> 命令看到。</p><ul><li>Docker 镜像仓库地址：地址的格式一般是 <code><域名/IP>[:端口号]</code>。默认地址是 Docker Hub。</li><li>仓库名：如之前所说，这里的仓库名是两段式名称，即 <code><用户名>/<软件名></code>。对于 Docker Hub，如果不给出用户名，则默认为 <code>library</code>，也就是官方镜像。</li></ul><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">sudo docker pull redis</span><br><span class="line">Using default tag: latest</span><br><span class="line">latest: Pulling from library/redis</span><br><span class="line">be8881be8156: Already exists </span><br><span class="line">d6f5ea773ca3: Pull complete </span><br><span class="line">735cc65c0db4: Pull complete </span><br><span class="line">787dddf99946: Pull complete </span><br><span class="line">0733799a7c0a: Pull complete </span><br><span class="line">6d250f04811a: Pull complete </span><br><span class="line">Digest: sha256:858b1677143e9f8455821881115e276f6177221de1c663d0abef9b2fda02d065</span><br><span class="line">Status: Downloaded newer image for redis:latest</span><br></pre></td></tr></table></figure><p>如果docker pull没有给出镜像地址，就会默认从docker hub下下载镜像</p><h3 id="列出镜像"><a href="#列出镜像" class="headerlink" title="列出镜像"></a>列出镜像</h3><p>可以通过<code>docker image</code>来查看本地有哪些镜像</p><blockquote><p>~$ sudo docker images<br>REPOSITORY TAG IMAGE ID CREATED SIZE<br>wordpress latest 3745a1731caf 40 hours ago 408MB<br>mysql latest 29e0ae3b69b9 2 weeks ago 484MB<br>redis latest 4e8db158f18d 3 weeks ago 83.4MB<br>列表包含了 <code>仓库名</code>、<code>标签</code>、<code>镜像 ID</code>、<code>创建时间</code> 以及 <code>所占用的空间</code>。</p></blockquote><h4 id="虚悬镜像"><a href="#虚悬镜像" class="headerlink" title="虚悬镜像"></a>虚悬镜像</h4><p>有时候有些镜像显示的是<code><none></code>。：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><none> <none> 00285df0df87 5 days ago 342 MB</span><br></pre></td></tr></table></figure><p>这个镜像原本是有镜像名和标签的，原来为 <code>mongo:3.2</code>，随着官方镜像维护，发布了新版本后，重新 <code>docker pull mongo:3.2</code> 时，<code>mongo:3.2</code> 这个镜像名被转移到了新下载的镜像身上，而旧的镜像上的这个名称则被取消，从而成为了 <code><none></code>。除了 <code>docker pull</code> 可能导致这种情况，<code>docker build</code> 也同样可以导致这种现象。由于新旧镜像同名，旧镜像名称被取消，从而出现仓库名、标签均为 <code><none></code> 的镜像。这类无标签镜像也被称为 <strong>虚悬镜像(dangling image)</strong> ，可以用下面的命令专门显示这类镜像：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ docker images -f dangling=true</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line"><none> <none> 00285df0df87 5 days ago 342 MB</span><br></pre></td></tr></table></figure><p>一般来说，虚悬镜像已经失去了存在的价值，是可以随意删除的，可以用下面的命令删除。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker image prune</span><br></pre></td></tr></table></figure><h4 id="中间层镜像"><a href="#中间层镜像" class="headerlink" title="中间层镜像"></a>中间层镜像</h4><p>为了加速镜像构建、重复利用资源，Docker 会利用 <strong>中间层镜像</strong>。所以在使用一段时间后，可能会看到一些依赖的中间层镜像。默认的 <code>docker image ls</code> 列表中只会显示顶层镜像，如果希望显示包括中间层镜像在内的所有镜像的话，需要加 <code>-a</code> 参数。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker image ls -a</span><br></pre></td></tr></table></figure><p>这样会看到很多无标签的镜像，与之前的虚悬镜像不同，这些无标签的镜像很多都是中间层镜像，是其它镜像所依赖的镜像。这些无标签镜像不应该删除，否则会导致上层镜像因为依赖丢失而出错。实际上，这些镜像也没必要删除，因为之前说过，相同的层只会存一遍，而这些镜像是别的镜像的依赖，因此并不会因为它们被列出来而多存了一份，无论如何你也会需要它们。只要删除那些依赖它们的镜像后，这些依赖的中间层镜像也会被连带删除。 </p><h3 id="删除本地镜像"><a href="#删除本地镜像" class="headerlink" title="删除本地镜像"></a>删除本地镜像</h3><p>如果要删除本地的镜像，可以使用 <code>docker image rm</code> 命令或者<code>docker rmi</code>，其格式为：</p> <figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker image rm [选项] <镜像1> [<镜像2> ...]</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker rmi [选项] <镜像1> [<镜像2> ...]</span><br></pre></td></tr></table></figure><h4 id="用-ID、镜像名、摘要删除镜像"><a href="#用-ID、镜像名、摘要删除镜像" class="headerlink" title="用 ID、镜像名、摘要删除镜像"></a>用 ID、镜像名、摘要删除镜像</h4><p>其中，<code><镜像></code> 可以是 <code>镜像短 ID</code>、<code>镜像长 ID</code>、<code>镜像名</code> 或者 <code>镜像摘要</code>。</p><p><code>sudo docker rmi 3745a1731caf</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">Untagged: centos:latest</span><br><span class="line">Untagged: centos@sha256:b2f9d1c0ff5f87a4743104d099a3d561002ac500db1b9bfa02a783a46e0d366c</span><br><span class="line">Deleted: sha256:0584b3d2cf6d235ee310cf14b54667d889887b838d3f3d3033acd70fc3c48b8a</span><br><span class="line">Deleted: sha256:97ca462ad9eeae25941546209454496e1d66749d53dfa2ee32bf1faabd239d38</span><br></pre></td></tr></table></figure><h2 id="容器"><a href="#容器" class="headerlink" title="容器"></a>容器</h2><h3 id="运行容器"><a href="#运行容器" class="headerlink" title="运行容器"></a>运行容器</h3><p>有了镜像后，我们就能够以这个镜像为基础启动并运行一个容器。</p><p><code>docker run --name redis_test -p 6379:6379 -d redis</code></p><blockquote><p>~$ sudo docker run –name redis_test -p 6379:6379 -d redis<br>5d0fde30bedc6192e77b75bbb8a5a9894b593f7eb58411f0b96a2270872445da</p></blockquote><blockquote><p>~$ sudo docker ps<br>CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES<br>5d0fde30bedc redis “docker-entrypoint.s…” 45 seconds ago Up 38 seconds 0.0.0.0:6379->6379/tcp redis_test</p></blockquote><p><code>docker run</code>就是运行容器的命令,运行他的时候可以带一些参数，详细可以通过<code>docke run --help</code></p><ul><li><code>--rm</code>：这个参数是说容器退出后随之将其删除。默认情况下，为了排障需求，退出的容器并不会立即删除，除非手动 <code>docker rm</code>。</li><li><code>redis</code>：这是指用 <code>redis</code> 镜像为基础来启动容器。</li><li><code>--name</code>:为容器指定名称。</li><li><code>-p</code>:将容器的端口发布到主机.这里就是讲reids容器的6379端口映射到主机上的6379端口(<strong>ip:hostPort:containerPort</strong>)</li><li><code>-d</code>:在后台运行容器并打印容器ID</li></ul><h3 id="进入容器内部"><a href="#进入容器内部" class="headerlink" title="进入容器内部"></a>进入容器内部</h3><p>进入redis容器内部，运行redis-cli，来操作redis:</p><p><code>docker exec -it containerid /bin/bash</code></p><ul><li><p><code>exec</code>:在正在运行的容器中运行命令</p></li><li><p><code>-it</code>：这是两个参数，一个是 <code>-i</code>：交互式操作，保证容器中stdin(一般指键盘输入到缓冲区里的东西)是开放的，一个是 <code>-t</code> ：告诉docker为要创建的容器分配一个伪tty终端。这样，新创建的容器才能够提供一个交互式shell</p></li><li><p><code>bash</code>：放在容器id后的是<strong>命令</strong>，这里我们希望有个交互式 Shell，因此用的是 <code>bash</code>。</p></li></ul><blockquote><p>~$ sudo docker exec -it 5d0fde30bedc /bin/bash<br>root@5d0fde30bedc:/data# redis-cli<br>127.0.0.1:6379> set test value<br>OK<br>127.0.0.1:6379> get test<br>“value”<br>127.0.0.1:6379> </p></blockquote><p>想要退出就按<code>ctrl+d</code></p><h3 id="停止，删除容器"><a href="#停止，删除容器" class="headerlink" title="停止，删除容器"></a>停止，删除容器</h3><ol><li><strong>停止容器运行</strong>：<code>docker stop containerid</code></li><li><strong>删除容器</strong>：<code>docker rm containerid</code> 删除容器必须要先将容器停止之后.</li><li><strong>删除所有容器</strong>: <em>docker rm `sudo docker ps -a -q`</em></li></ol><h3 id="启动，重启容器"><a href="#启动，重启容器" class="headerlink" title="启动，重启容器"></a>启动，重启容器</h3><ol><li><p><strong>启动已经停止运行的容器</strong>：<code>docker start containername</code>或者通过id启动<code>docker start containerid</code></p></li><li><p><strong>重启容器</strong>:<code>docker restart containername</code>或者<code>docker restart containerid</code></p></li><li><p><strong>自动重启容器</strong>：</p><p>如果容器因为某种错误而导致了容器停止运行，还可以通过<code>--restart</code>标志，让docker自动重新启动该容器。<code>--restart</code>标志会检查容器的退出代码，并根据此来决定是否要重启容器。默认的行为是docker不会容器容器。</p><p><strong>–restart</strong>:参数：</p><p><code>always</code>:无论容器的退出代码是什么，docker 都会自动重启该容器，</p><p><code>on-failure</code>:只有当容器退出的代码是非0值的时候，才会自动重启，on-failure可以接受一个可选的重启次数参数，如<code>--restart=on-failure:5</code>.这样，当容器退出代码为非0的时候，docker会尝试自动重启该容器，最多重启5此</p></li></ol>]]></content>
<categories>
<category> docker </category>
</categories>
<tags>
<tag> docker </tag>
</tags>
</entry>
<entry>
<title>TCP-SYN</title>
<link href="/2018/08/27/TCP-SYN/"/>
<url>/2018/08/27/TCP-SYN/</url>
<content type="html"><![CDATA[<h1 id="简介"><a href="#简介" class="headerlink" title="简介"></a>简介</h1><p>传输控制协议中的TCP三次握手（也称为TCP握手;三次消息握手或SYN-SYN-ACK）是TCP在基于互联网协议的网络上建立TCP / IP连接所使用的方法。 TCP的三路握手技术通常被称为“SYN-SYN-ACK”（或者更准确地说是SYN，SYN-ACK，ACK），因为TCP传输三条消息来协商和启动两台计算机之间的TCP会话。 TCP握手机制被设计为使得两个尝试通信的计算机可以在传输诸如SSH和HTTP web浏览器请求之类的数据之前协商网络TCP套接字连接的参数。</p><p>此三次握手过程的设计也是为了使两端可以同时启动和协商单独的TCP套接字连接。能够同时在两个方向上协商多个TCP套接字连接允许多路复用单个物理网络接口（例如以太网）以同时传输多个TCP数据流。</p><h2 id="tcp帧格式"><a href="#tcp帧格式" class="headerlink" title="tcp帧格式"></a>tcp帧格式</h2><p>tcp数据包的格式如下：<br><img src="/2018/08/27/TCP-SYN/tcp-header.png" alt=""></p><p>源端口号和目的端口号与udp中类似，用于寻找发端和收端应用进程</p><p> <strong>源端口号(Source Port)和目的端口号(Destination port)</strong>与udp中类似，用于寻找发端和收端应用进程。这两个值加上IP首部中的源端IP地址和目的端IP地址唯一确定一个TCP连接，在网络编程中，一般一个IP地址和一个端口号组合称为一个套接字（socket）。 <strong>（源端口和目的端口各占2个字节，总共四个字节）</strong><br> <strong>序列号（SequenceNumber）</strong>：32位的序列号标识了TCP报文中第一个byte在对应方向的传输中对应的字节序号。当SYN出现，SN=ISN(随机值）单位是byte。比如发送端发送的一个TCP包净荷(不包含TCP头20byte)为12byte，SN为5，则发送端接着发送的下一个数据包的时候，SN应该设置为5+12=17。通过序列号，TCP接收端可以识别出重复接收到的TCP包，从而丢弃重复包，同时对于乱序数据包也可以依靠系列号进行重排序，进而对高层提供有序的数据流。另外如果接收的包中包含SYN或FIN标志位，逻辑上也占用1个byte，应答号需加1。 <strong>（四字节）</strong><br> <strong>确认序号(Acknowledgment Number简称ACK Number)</strong>：32位的ACK Number标识了报文发送端期望接收的字节序列。如果设置了ACK控制位，这个值表示一个准备接收的包的序列码，注意是准备接收的包，比如当前接收端接收到一个净荷为12byte的数据包，SN为5，则会回复一个确认收到的数据包，如果这个数据包之前的数据也都已经收到了，这个数据包中的ACK Number则设置为12+5=17，表示之前的数据都已经收到了，准备接受SN=17的数据包。 <strong>（四字节）</strong> </p><p> <strong>头长(Header Length)</strong>：4位包括TCP头大小，指示TCP头的长度，即数据从何处开始。</p><p> <strong>CWR(Congestion Window Reduce)</strong>：拥塞窗口减少标志set by sender，用来表明它接收到了设置ECE标志的TCP包。并且sender 在收到消息之后已经通过降低发送窗口的大小来降低发送速率。</p><p> <strong>ECE(ECN Echo)</strong>：ECN响应标志被用来在TCP3次握手时表明一个TCP端是具备ECN功能的。在数据传输过程中也用来表明接收到的TCP包的IP头部的ECN被设置为11。注：IP头部的ECN被设置为11表明网络线路拥堵。</p><p> <strong>URG</strong>： 紧急指针（ urgent pointer）有效。 <strong>（1bit）</strong><br> <strong>ACK</strong>： 取值1代表Acknowledgment Number字段有效，这是一个确认的TCP包，取值0则不是确认包。后续文章介绍中当ACK标志位有效的时候我们称呼这个包为ACK包，使用大写的ACK称呼。<br> <strong>PSH</strong>： 该标志置位时，一般是表示发送端缓存中已经没有待发送的数据，接收端不将该数据进行队列处理，而是尽可能快将数据转由应用处理。在处理 telnet 或 rlogin 等交互模式的连接时，该标志总是置位的。<br> <strong>RST</strong>： 重建连接。用于reset相应的TCP连接。通常在发生异常或者错误的时候会触发复位TCP连接。<br> <strong>SYN</strong>： 同步序列编号(Synchronize Sequence Numbers)有效。该标志仅在三次握手建立TCP连接时有效。）<br> <strong>FIN</strong>： 发端完成发送任务。 No more data from sender。当FIN标志有效的时候我们称呼这个包为FIN包。<br> <strong>窗口大小</strong>：16位，该值指示了从Ack Number开始还愿意接收多少byte的数据量，也即用来表示当前接收端的接收窗还有多少剩余空间，用于TCP的流量控制。<br> <strong>检验和</strong>：检16位TCP头。发送端基于数据内容计算一个数值，接收端要与发送端数值结果完全一样，才能证明数据的有效性。接收端checksum校验失败的时候会直接丢掉这个数据包。CheckSum是根据伪头+TCP头+TCP数据三部分进行计算的。</p><p><strong>优先指针（紧急,Urgent Pointer）</strong>：16位，指向后面是优先数据的字节，在URG标志设置了时才有效。如果URG标志没有被设置，紧急域作为填充。 </p><p><strong>选项(Option)</strong>：长度不定，但长度必须以是32bits的整数倍。常见的选项包括MSS、SACK、Timestamp等等。</p><h3 id="tcp报文段"><a href="#tcp报文段" class="headerlink" title="tcp报文段"></a>tcp报文段</h3><p><img src="/2018/08/27/TCP-SYN/tcp-segment.png" alt=""> </p><h3 id="ip数据报"><a href="#ip数据报" class="headerlink" title="ip数据报"></a>ip数据报</h3><p><img src="/2018/08/27/TCP-SYN/ip-segment.png" alt=""></p>]]></content>
<categories>
<category> tcp </category>
</categories>
<tags>
<tag> tcp ,syn </tag>
</tags>
</entry>
<entry>
<title>protobuf</title>
<link href="/2018/08/27/protobuf/"/>
<url>/2018/08/27/protobuf/</url>
<content type="html"><![CDATA[<h1 id="简介"><a href="#简介" class="headerlink" title="简介"></a>简介</h1><p>由于线上项目在做压力测试的时候，发现在数据量不是特别大的时候cpu资源消耗特别多，经过go pprof工具的cpu性能分析发现，因为在此项目中有大量使用gob序列化和发序列化的地方导致，因为gob序列化需要用到反射，性能很差，最后在做了各个序列化对比后，发现google的Protocol Buffers序列化与反序列化性能消耗差不多，性能是gob的十几倍。因此想将线上数据改造为protobuf格式。</p><p>Protocol Buffers 是一种轻便高效的结构化数据存储格式，可以用于结构化数据串行化，或者说序列化。它很适合做数据存储或 RPC 数据交换格式。可用于通讯协议、数据存储等领域的语言无关、平台无关、可扩展的序列化结构数据格式。</p><h2 id="安装-Google-Protocol-Buffer"><a href="#安装-Google-Protocol-Buffer" class="headerlink" title="安装 Google Protocol Buffer"></a>安装 Google Protocol Buffer</h2><ul><li><p>protoc可执行文件下载地址：<a href="https://github.com/google/protobuf/releases" target="_blank" rel="noopener">https://github.com/google/protobuf/releases</a></p></li><li><p>安装编译器插件protoc-gen-go (protoc-gen-go用于生成Go语言代码)</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">go get -u github.com/golang/protobuf/{proto,protoc-gen-go}</span><br></pre></td></tr></table></figure></li></ul><h2 id="编写-proto文件"><a href="#编写-proto文件" class="headerlink" title="编写.proto文件"></a>编写.proto文件</h2><p>首先我们需要编写一个 proto 文件，定义我们程序中需要处理的结构化数据，在 protobuf 的术语中，结构化数据被称为 Message。proto 文件非常类似 java 或者 C 语言的数据定义。代码清单 1 显示了例子应用中的 proto 文件内容。</p><h5 id="清单-1-proto-文件"><a href="#清单-1-proto-文件" class="headerlink" title="清单 1. proto 文件"></a>清单 1. proto 文件</h5><figure class="highlight protobuf"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">syntax = "proto3";</span><br><span class="line">package lm; </span><br><span class="line">message helloworld </span><br><span class="line">{ </span><br><span class="line"> required int32 id = 1; // ID </span><br><span class="line"> required string str = 2; // str </span><br><span class="line"> optional int32 opt = 3; //optional field </span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>一个比较好的习惯是认真对待 proto 文件的文件名。比如将命名规则定于如下：</p><p><code>packageName.MessageName.proto</code></p><p>在上例中，package 名字叫做 lm，定义了一个消息 helloworld，该消息有三个成员，类型为 int32 的 id，另一个为类型为 string 的成员 str。opt 是一个可选的成员，即消息中可以不包含该成员。</p><h3 id="编译-proto-文件"><a href="#编译-proto-文件" class="headerlink" title="编译 .proto 文件"></a>编译 .proto 文件</h3><p>写好 proto 文件之后就可以用 Protobuf 编译器将该文件编译成目标语言了。本例中我们将使用 Go。</p><p><strong>这里详细介绍golang的编译姿势:</strong></p><ul><li><code>-I</code> 参数：指定import路径，可以指定多个<code>-I</code>参数，编译时按顺序查找，不指定时默认查找当前目录</li><li><code>--go_out</code> ：golang编译支持，支持以下参数<ul><li><code>plugins=plugin1+plugin2</code> - 指定插件，目前只支持grpc，即：<code>plugins=grpc</code></li><li><code>M</code> 参数 - 指定导入的.proto文件路径编译后对应的golang包名(不指定本参数默认就是<code>.proto</code>文件中<code>import</code>语句的路径)</li><li><code>import_prefix=xxx</code> - 为所有<code>import</code>路径添加前缀，主要用于编译子目录内的多个proto文件，这个参数按理说很有用，尤其适用替代一些情况时的<code>M</code>参数，但是实际使用时有个蛋疼的问题导致并不能达到我们预想的效果，自己尝试看看吧</li><li><code>import_path=foo/bar</code> - 用于指定未声明<code>package</code>或<code>go_package</code>的文件的包名，最右面的斜线前的字符会被忽略</li><li>末尾 <code>:编译文件路径 .proto文件路径(支持通配符)</code></li></ul></li></ul><p>完整示例：</p><p>假设您的 proto 文件存放在 $SRC_DIR 下面，您也想把生成的文件放在同一个目录下，则可以使用如下命令：</p><p><code>protoc -I=$SRC_DIR --go_out=$DST_DIR $SRC_DIR/addressbook.proto</code></p><p>命令将生成一个文件：</p><p><code>addressbook.pb.go</code> </p>]]></content>
<categories>
<category> protobuf </category>
</categories>
<tags>
<tag> protobuf，google </tag>
</tags>
</entry>
<entry>
<title>redis-eval</title>
<link href="/2018/08/27/redis-eval/"/>
<url>/2018/08/27/redis-eval/</url>
<content type="html"><![CDATA[<h1 id="EVAL"><a href="#EVAL" class="headerlink" title="EVAL"></a>EVAL</h1><p><strong>EVAL script numkeys key [key …] arg [arg …]</strong></p><p>从 Redis 2.6.0 版本开始，通过内置的 Lua 解释器，可以使用 <a href="http://doc.redisfans.com/script/eval.html#eval" target="_blank" rel="noopener">EVAL</a> 命令对 Lua 脚本进行求值。</p><ul><li><code>script</code> 参数是一段 Lua 5.1 脚本程序，它会被运行在 Redis 服务器上下文中，这段脚本不必(也不应该)定义为一个 Lua 函数。</li><li><code>numkeys</code> 参数用于指定键名参数的个数。</li><li>键名参数 <code>key [key ...]</code> 从 <a href="http://doc.redisfans.com/script/eval.html#eval" target="_blank" rel="noopener">EVAL</a> 的第三个参数开始算起，表示在脚本中所用到的那些 Redis 键(key)，这些键名参数可以在 Lua 中通过全局变量 <code>KEYS</code> 数组，用 <code>1</code> 为基址的形式访问( <code>KEYS[1]</code> ， <code>KEYS[2]</code> ，以此类推)。</li><li>在命令的最后，那些不是键名参数的附加参数 <code>arg [arg ...]</code> ，可以在 Lua 中通过全局变量 <code>ARGV</code> 数组访问，访问的形式和 <code>KEYS</code> 变量类似( <code>ARGV[1]</code> 、 <code>ARGV[2]</code> ，诸如此类)。</li></ul><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">> eval <span class="string">"return {KEYS[1],KEYS[2],ARGV[1],ARGV[2]}"</span> <span class="number">2</span> key1 key2 first second</span><br><span class="line"><span class="number">1</span>) <span class="string">"key1"</span></span><br><span class="line"><span class="number">2</span>) <span class="string">"key2"</span></span><br><span class="line"><span class="number">3</span>) <span class="string">"first"</span></span><br><span class="line"><span class="number">4</span>) <span class="string">"second"</span></span><br></pre></td></tr></table></figure><p>通过docker 安装reids</p><figure class="highlight dockerfile"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">docker <span class="keyword">run</span><span class="bash"> --name redis-lua -p 6379:6379 -d redis</span></span><br></pre></td></tr></table></figure><blockquote><p>~ docker ps</p><p>CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES<br>939cd668f9d0 redis “docker-entrypoint.s…” 6 seconds ago Up 2 seconds 0.0.0.0:6379->6379/tcp redis-lua</p></blockquote><p>执行以下命令进入docker容器内部：<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">docker exec -it 939cd668f9d0 /bin/bash</span><br></pre></td></tr></table></figure></p><h2 id="执行eval命令"><a href="#执行eval命令" class="headerlink" title="执行eval命令"></a>执行eval命令</h2><p>在 Lua 脚本中，可以使用两个不同函数来执行 Redis 命令，它们分别是：</p><ul><li><code>redis.call()</code></li><li><code>redis.pcall()</code></li></ul><p>这两个函数的唯一区别在于它们使用不同的方式处理执行命令所产生的错误。</p><blockquote><p>当 <code>redis.call()</code> 在执行命令的过程中发生错误时，脚本会停止执行，并返回一个脚本错误，错误的输出信息会说明错误造成的原因：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">> redis> lpush foo a</span><br><span class="line">> (integer) 1</span><br><span class="line">> </span><br><span class="line">> redis> eval "return redis.call('get', 'foo')" 0</span><br><span class="line">> (error) ERR Error running script (call to f_282297a0228f48cd3fc6a55de6316f31422f5d17): ERR Operation against a key holding the wrong kind of value</span><br><span class="line">></span><br></pre></td></tr></table></figure></blockquote><blockquote><p>和 <code>redis.call()</code> 不同， <code>redis.pcall()</code> 出错时并不引发(raise)错误，而是返回一个带 <code>err</code> 域的 Lua 表(table)，用于表示错误：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">> redis 127.0.0.1:6379> EVAL "return redis.pcall('get', 'foo')" 0</span><br><span class="line">> (error) ERR Operation against a key holding the wrong kind of value</span><br><span class="line">></span><br></pre></td></tr></table></figure></blockquote><p><code>redis.call()</code> 和 <code>redis.pcall()</code> 两个函数的参数可以是任何格式良好(well formed)的 Redis 命令：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">> eval "return redis.call('set','foo','bar')" 0</span><br><span class="line">OK</span><br></pre></td></tr></table></figure><p>需要注意的是，上面这段脚本的确实现了将键 <code>foo</code> 的值设为 <code>bar</code> 的目的，但是，它违反了 <a href="http://doc.redisfans.com/script/eval.html#eval" target="_blank" rel="noopener">EVAL</a> 命令的语义，因为脚本里使用的所有键都应该由 <code>KEYS</code> 数组来传递，就像这样：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">> eval "return redis.call('set',KEYS[1],'bar')" 1 foo</span><br><span class="line">OK</span><br></pre></td></tr></table></figure><h2 id="golang中使用Lua脚本操作Redis"><a href="#golang中使用Lua脚本操作Redis" class="headerlink" title="golang中使用Lua脚本操作Redis"></a>golang中使用Lua脚本操作Redis</h2><p>为了在我的一个基本库中降低与Redis的通讯成本，我将一系列操作封装到LUA脚本中，借助Redis提供的EVAL命令来简化操作。<br> EVAL能够提供的特性：</p><ol><li>可以在LUA脚本中封装若干操作，如果有多条Redis指令，封装好之后只需向Redis一次性发送所有参数即可获得结果</li><li>Redis可以保证Lua脚本运行期间不会有其他命令插入执行，提供像数据库事务一样的原子性</li><li>Redis会根据脚本的SHA值缓存脚本，已经缓存过的脚本不需要再次传输Lua代码，减少了通信成本，此外在自己代码中改变Lua脚本，执行时Redis必定也会使用最新的代码。</li></ol><p>导入常见的Go库如 “github.com/go-redis/redis”，就可以实现以下代码。</p><p><strong>生成一段Lua脚本</strong></p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// KEYS: key for record</span></span><br><span class="line"><span class="comment">// ARGV: fieldName, currentUnixTimestamp, recordTTL</span></span><br><span class="line"><span class="comment">// Update expire field of record key to current timestamp, and renew key expiration</span></span><br><span class="line"><span class="keyword">var</span> updateRecordExpireScript = redis.NewScript(<span class="string">`</span></span><br><span class="line"><span class="string">redis.call("EXPIRE", KEYS[1], ARGV[3])</span></span><br><span class="line"><span class="string">redis.call("HSET", KEYS[1], ARGV[1], ARGV[2])</span></span><br><span class="line"><span class="string">return 1</span></span><br><span class="line"><span class="string">`</span>)</span><br></pre></td></tr></table></figure><p> 该变量创建时，Lua代码不会被执行，也不需要有已存的Redis连接。 Redis提供的Lua脚本支持，默认有KEYS、ARGV两个数组，KEYS代表脚本运行时传入的若干键值，ARGV代表传入的若干参数。由于Lua代码需要保持简洁，难免难以读懂，最好为这些参数写一些注释</p><p>注意：上面一段代码使用<code></code>跨行，`所在的行虽然空白回车，也会被认为是一行，报错时不要看错代码行号。</p><p><strong>运行一段Lua脚本</strong></p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">updateRecordExpireScript.Run(c.Client, []<span class="keyword">string</span>{recordKey(key)}, </span><br><span class="line"> expireField,</span><br><span class="line"> time.Now().UTC().UnixNano(), <span class="keyword">int64</span>(c.opt.RecordTTL/time.Second)).Err()</span><br></pre></td></tr></table></figure><p> 运行时，Run将会先通过EVALSHA尝试通过缓存运行脚本。如果没有缓存，则使用EVAL运行，这时Lua脚本才会被整个传入Redis。</p><h2 id="Lua脚本的限制"><a href="#Lua脚本的限制" class="headerlink" title="Lua脚本的限制"></a>Lua脚本的限制</h2><ol><li>Redis不提供引入额外的包，例如os等，只有redis这一个包可用。</li><li>Lua脚本将会在一个函数中运行，所有变量必须使用local声明</li><li>return返回多个值时，Redis将会只给你第一个</li></ol><h2 id="脚本中的类型限制"><a href="#脚本中的类型限制" class="headerlink" title="脚本中的类型限制"></a>脚本中的类型限制</h2><ol><li>脚本返回nil时，Go中得到的是err = redis.Nil（与Get找不到值相同）</li><li>脚本返回false时，Go中得到的是nil，脚本返回true时，Go中得到的是int64类型的1</li><li>脚本返回{“ok”: …}时，Go中得到的是redis的status类型（true/false)</li><li>脚本返回{“err”: …}时，Go中得到的是err值，也可以通过return redis.error_reply(“My Error”)达成</li><li>脚本返回number类型时，Go中得到的是int64类型</li><li><p>传入脚本的KEYS/ARGV中的值一律为string类型，要转换为数字类型应当使用to_number</p><p><strong>如果脚本运行了很久会发生什么？</strong></p></li></ol><p>Lua脚本运行期间，为了避免被其他操作污染数据，这期间将不能执行其它命令，一直等到执行完毕才可以继续执行其它请求。当Lua脚本执行时间超过了lua-time-limit时，其他请求将会收到Busy错误，除非这些请求是SCRIPT KILL（杀掉脚本）或者SHUTDOWN NOSAVE（不保存结果直接关闭Redis）</p><h1 id="参考："><a href="#参考：" class="headerlink" title="参考："></a>参考：</h1><p> <a href="https://www.jianshu.com/p/fec18f59ff8f" target="_blank" rel="noopener">Go中通过Lua脚本操作Redis</a></p><p><a href="http://doc.redisfans.com/script/eval.html" target="_blank" rel="noopener">redis命令参考</a></p>]]></content>
<categories>
<category> redis </category>
</categories>
<tags>
<tag> redis,lua </tag>
</tags>
</entry>
<entry>
<title>lua-1</title>
<link href="/2018/08/27/lua-1/"/>
<url>/2018/08/27/lua-1/</url>
<content type="html"><![CDATA[<h1 id="简介"><a href="#简介" class="headerlink" title="简介"></a>简介</h1><p>Lua 是一种轻量小巧的脚本语言，用标准C语言编写并以源代码形式开放， 其设计目的是为了嵌入应用程序中，从而为应用程序提供灵活的扩展和定制功能。。Lua是类C的，所以，他是大小写字符敏感的。</p><h1 id="Lua-环境安装"><a href="#Lua-环境安装" class="headerlink" title="Lua 环境安装"></a>Lua 环境安装</h1><h2 id="Linux-系统上安装"><a href="#Linux-系统上安装" class="headerlink" title="Linux 系统上安装"></a>Linux 系统上安装</h2><p>Linux & Mac上安装 Lua 安装非常简单，只需要下载源码包并在终端解压编译即可。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">wget https://www.lua.org/ftp/lua-5.3.5.tar.gz</span><br><span class="line">tar zxf lua-5.3.5.tar.gz</span><br><span class="line">cd lua-5.3.5/</span><br><span class="line">make linux test</span><br><span class="line">sudo make install</span><br></pre></td></tr></table></figure><h1 id="编写Lua脚本的helloworld"><a href="#编写Lua脚本的helloworld" class="headerlink" title="编写Lua脚本的helloworld"></a>编写Lua脚本的helloworld</h1><p> Lua脚本的语句的分号是可选的，这个和GO语言很类似。</p><ul><li>在hello.lua中编写lua代码</li></ul><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">vim hello.lua</span><br><span class="line"><span class="built_in">print</span>(<span class="string">"Hello World"</span>)</span><br><span class="line">lua hello.lua</span><br></pre></td></tr></table></figure><ul><li>我们也可以将代码修改为如下形式来执行脚本（在开头添加：#!/usr/local/bin/lua）：</li></ul><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">#!/usr/<span class="keyword">local</span>/bin/lua</span><br><span class="line"></span><br><span class="line"><span class="built_in">print</span>(<span class="string">"Hello World！"</span>)</span><br><span class="line"><span class="built_in">print</span>(<span class="string">"www.runoob.com"</span>)</span><br></pre></td></tr></table></figure><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">chmod 766 hell.lua</span><br><span class="line">./hello.lua</span><br></pre></td></tr></table></figure><ul><li>你可以像python一样，在命令行上运行lua命令后进入lua的shell中执行语句。</li></ul><p>Lua 交互式编程模式可以通过命令 lua -i 或 lua 来启用：</p><blockquote><p>~ lua<br>Lua 5.3.5 Copyright (C) 1994-2018 Lua.org, PUC-Rio</p><p>>print(“hello world”)</p><p>hello world</p></blockquote><h1 id="Lua-基本语法"><a href="#Lua-基本语法" class="headerlink" title="Lua 基本语法"></a>Lua 基本语法</h1><h2 id="注释"><a href="#注释" class="headerlink" title="注释"></a>注释</h2><h3 id="单行注释"><a href="#单行注释" class="headerlink" title="单行注释"></a>单行注释</h3><p>两个减号是单行注释:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">--</span><br></pre></td></tr></table></figure><h3 id="多行注释"><a href="#多行注释" class="headerlink" title="多行注释"></a>多行注释</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">--[[</span><br><span class="line"> 多行注释</span><br><span class="line"> 多行注释</span><br><span class="line"> --]]</span><br></pre></td></tr></table></figure><hr><h2 id="标示符"><a href="#标示符" class="headerlink" title="标示符"></a>标示符</h2><p>Lua 标示符用于定义一个变量，函数获取其他用户定义的项。标示符以一个字母 A 到 Z 或 a 到 z 或下划线 _ 开头后加上0个或多个字母，下划线，数字（0到9）。</p><h2 id="关键词"><a href="#关键词" class="headerlink" title="关键词"></a>关键词</h2><p>以下列出了 Lua 的保留关键字。保留关键字不能作为常量或变量或其他用户自定义标示符：</p><table><thead><tr><th>and</th><th>break</th><th>do</th><th>else</th></tr></thead><tbody><tr><td>elseif</td><td>end</td><td>false</td><td>for</td></tr><tr><td>function</td><td>if</td><td>in</td><td>local</td></tr><tr><td>nil</td><td>not</td><td>or</td><td>repeat</td></tr><tr><td>return</td><td>then</td><td>true</td><td>until</td></tr><tr><td>while</td><td></td><td></td></tr></tbody></table><p>一般约定，以下划线开头连接一串大写字母的名字（比如 _VERSION）被保留用于 Lua 内部全局变量。</p><h1 id="Lua-变量"><a href="#Lua-变量" class="headerlink" title="Lua 变量"></a>Lua 变量</h1><p><strong>Lua 变量有三种类型：全局变量、局部变量、表中的域。</strong></p><h2 id="全局变量"><a href="#全局变量" class="headerlink" title="全局变量"></a>全局变量</h2><p>需要注意的是：lua中的变量如果没有特殊说明，全是全局变量，那怕是语句块或是函数里。变量前加<strong>local</strong>关键字的是局部变量。</p><p>全局变量不需要声明，给一个变量赋值后即创建了这个全局变量，访问一个没有初始化的全局变量也不会出错，只不过得到的结果是：nil。</p><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">> <span class="built_in">print</span>(b)</span><br><span class="line"><span class="literal">nil</span></span><br><span class="line">> b=<span class="number">10</span></span><br><span class="line">> <span class="built_in">print</span>(b)</span><br><span class="line"><span class="number">10</span></span><br><span class="line">></span><br></pre></td></tr></table></figure><p>如果你想删除一个全局变量，只需要将变量赋值为nil。</p><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">b = <span class="literal">nil</span></span><br><span class="line"><span class="built_in">print</span>(b) <span class="comment">--> nil</span></span><br></pre></td></tr></table></figure><p>这样变量b就好像从没被使用过一样。换句话说, 当且仅当一个变量不等于nil时，这个变量即存在。</p><h2 id="控制语句"><a href="#控制语句" class="headerlink" title="控制语句"></a>控制语句</h2><h5 id="while循环"><a href="#while循环" class="headerlink" title="while循环"></a>while循环</h5><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">sum = <span class="number">0</span></span><br><span class="line">num = <span class="number">1</span></span><br><span class="line"><span class="keyword">while</span> num <= <span class="number">100</span> <span class="keyword">do</span></span><br><span class="line"> sum = sum + num</span><br><span class="line"> num = num + <span class="number">1</span></span><br><span class="line"><span class="keyword">end</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">"sum ="</span>,sum)</span><br></pre></td></tr></table></figure><h5 id="if-else分支"><a href="#if-else分支" class="headerlink" title="if-else分支"></a>if-else分支</h5><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">if</span> age == <span class="number">40</span> <span class="keyword">and</span> sex ==<span class="string">"Male"</span> <span class="keyword">then</span></span><br><span class="line"> <span class="built_in">print</span>(<span class="string">"男人四十一枝花"</span>)</span><br><span class="line"><span class="keyword">elseif</span> age > <span class="number">60</span> <span class="keyword">and</span> sex ~=<span class="string">"Female"</span> <span class="keyword">then</span></span><br><span class="line"> <span class="built_in">print</span>(<span class="string">"old man without country!"</span>)</span><br><span class="line"><span class="keyword">elseif</span> age < <span class="number">20</span> <span class="keyword">then</span></span><br><span class="line"> <span class="built_in">io</span>.<span class="built_in">write</span>(<span class="string">"too young, too naive!\n"</span>)</span><br><span class="line"><span class="keyword">else</span></span><br><span class="line"> <span class="keyword">local</span> age = <span class="built_in">io</span>.<span class="built_in">read</span>()</span><br><span class="line"> <span class="built_in">print</span>(<span class="string">"Your age is "</span>..age)</span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure><p>上面的语句不但展示了if-else语句，也展示了<br>1）“～=”是不等于，而不是!=<br>2）io库的分别从stdin和stdout读写的read和write函数<br>3）字符串的拼接操作符“..”</p><p> <strong>（注意：Lua没有++或是+=这样的操作）</strong></p><p>条件表达式中的与或非为分是：and, or, not关键字。</p><h5 id="for-循环"><a href="#for-循环" class="headerlink" title="for 循环"></a>for 循环</h5><p>从1加到100</p><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">sum = <span class="number">0</span></span><br><span class="line"><span class="keyword">for</span> i = <span class="number">1</span>, <span class="number">100</span> <span class="keyword">do</span></span><br><span class="line"> sum = sum + i</span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure><p>从1到100的奇数和</p><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">sum = <span class="number">0</span></span><br><span class="line"><span class="keyword">for</span> i=<span class="number">1</span>,<span class="number">100</span>,<span class="number">2</span> <span class="keyword">do</span></span><br><span class="line">sum = sum +i</span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure><p>从100到1的偶数和</p><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">sum = <span class="number">0</span></span><br><span class="line"><span class="keyword">for</span> i = <span class="number">100</span>, <span class="number">1</span>, <span class="number">-2</span> <span class="keyword">do</span></span><br><span class="line"> sum = sum + i</span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure><h5 id="until循环"><a href="#until循环" class="headerlink" title="until循环"></a>until循环</h5> <figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">sum = <span class="number">2</span></span><br><span class="line"><span class="keyword">repeat</span></span><br><span class="line"> sum = sum ^ <span class="number">2</span> <span class="comment">--幂操作</span></span><br><span class="line"> <span class="built_in">print</span>(sum)</span><br><span class="line"><span class="keyword">until</span> sum ><span class="number">1000</span></span><br></pre></td></tr></table></figure><h2 id="函数"><a href="#函数" class="headerlink" title="函数"></a>函数</h2><p>Lua的函数和Javascript的很像</p><h3 id="递归"><a href="#递归" class="headerlink" title="递归"></a>递归</h3><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">function</span> <span class="title">fib</span><span class="params">(n)</span></span></span><br><span class="line"> <span class="keyword">if</span> n < <span class="number">2</span> <span class="keyword">then</span> <span class="keyword">return</span> <span class="number">1</span> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">return</span> fib(n - <span class="number">2</span>) + fib(n - <span class="number">1</span>)</span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure><h3 id="闭包"><a href="#闭包" class="headerlink" title="闭包"></a>闭包</h3><figure class="highlight lua"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">function</span> <span class="title">newCounter</span><span class="params">()</span></span></span><br><span class="line"> <span class="keyword">local</span> i = <span class="number">0</span></span><br><span class="line"> <span class="keyword">return</span> <span class="function"><span class="keyword">function</span><span class="params">()</span></span> <span class="comment">-- anonymous function</span></span><br><span class="line"> i = i + <span class="number">1</span></span><br><span class="line"> <span class="keyword">return</span> i</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br><span class="line"> </span><br><span class="line">c1 = newCounter()</span><br><span class="line"><span class="built_in">print</span>(c1()) <span class="comment">--> 1</span></span><br><span class="line"><span class="built_in">print</span>(c1()) <span class="comment">--> 2</span></span><br></pre></td></tr></table></figure>]]></content>
<categories>
<category> lua </category>
</categories>
<tags>
<tag> lua </tag>
</tags>
</entry>
<entry>
<title>mysql深入</title>
<link href="/2018/08/23/mysql%E6%B7%B1%E5%85%A5/"/>
<url>/2018/08/23/mysql%E6%B7%B1%E5%85%A5/</url>
<content type="html"><![CDATA[<h2 id="Replace-into"><a href="#Replace-into" class="headerlink" title="Replace into"></a>Replace into</h2><p>Replace into是Insert into的增强版。在向表中插入数据时，我们经常会遇到这样的情况：1、首先判断数据是否存在；2、如果不存在，则插入；3、如果存在，则更新。</p><p>但是Replace也有坑，</p><h3 id="删除记录并且数据会变化"><a href="#删除记录并且数据会变化" class="headerlink" title="删除记录并且数据会变化"></a>删除记录并且数据会变化</h3><ol><li>Replace into的执行步骤其实就是，先去检查查询当前要插入的数据的唯一索引是否存在，如果存在，则将当前的数据删除，然后重新新增一条数据，新增的数据除了自己设置的值，其他的值都是，默认值，假设test表有三个字段</li></ol><blockquote><p><code>id</code> <code>key</code> <code>value</code>,id为自增主键，key为唯一索引，</p><p> 1 1 1 </p><p> 2 2 2 </p><p> 3 3 3 </p></blockquote><p>那么<code>replace into test (key,value) values (3,4);</code>的时候，则会去删除当前的记key=3的记录，并重新insert一条新的记录，此时表面看是将数据更新了，但是，自增id的值已经变了。</p><blockquote><p><code>id</code> <code>key</code> <code>value</code></p><p> 1 1 1 </p><p> 2 2 2 </p><p> 4 3 4 </p></blockquote><h3 id="删除多条记录"><a href="#删除多条记录" class="headerlink" title="删除多条记录"></a>删除多条记录</h3><p>在表中有超过一个的唯一索引。在这种情况下，REPLACE将考虑每一个唯一索引，并对每一个索引对应的重复记录都删除，然后插入这条新记录。</p><p>假设有一个table1表，有3个字段a, b, c。它们都有一个唯一索引。 </p><p>假设table1中已经有了3条记录 </p><blockquote><p>a b c </p><p>1 1 1 </p><p>2 2 2 </p><p>3 3 3 </p></blockquote><p>下面我们使用REPLACE语句向table1中插入一条记录。 </p><p><code>REPLACE INTO table1(a, b, c) VALUES(1,2,3);</code> </p><p>返回的结果如下 Query OK, 4 rows affected (0.00 sec) </p><p>在table1中的记录如下 </p><blockquote><p> a b c </p><p> 1 2 3 </p></blockquote><p>我们可以看到，REPLACE将原先的3条记录都删除了，然后将（1, 2, 3）插入。 </p><h2 id="替代方法"><a href="#替代方法" class="headerlink" title="替代方法"></a>替代方法</h2><ol><li>通过两条SQL语句</li></ol><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">SELECT COUNT(*) FROM xxx WHERE ID=xxx;</span><br><span class="line">if (x == 0)</span><br><span class="line"> INSERT INTO xxx VALUES;</span><br><span class="line">else</span><br><span class="line"> UPDATE xxx SET ;</span><br></pre></td></tr></table></figure><ol start="2"><li><p>如果在INSERT语句末尾指定了ON DUPLICATE KEY UPDATE，并且插入行后会导致在一个UNIQUE索引或PRIMARY KEY中出现重复值，则在出现重复值的行执行UPDATE；如果不会导致唯一值列重复的问题，则插入新行。<strong>不过（ 这语法效率超低。此语发是mysql独有）</strong></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">INSERT INTO TABLE (a,c) VALUES (1,3) ON DUPLICATE KEY UPDATE c=c+1;</span><br><span class="line">UPDATE TABLE SET c=c+1 WHERE a=1;</span><br></pre></td></tr></table></figure></li></ol><h2 id="详细可以参考："><a href="#详细可以参考：" class="headerlink" title="详细可以参考："></a>详细可以参考：</h2><p> <strong><a href="http://jjhpeopl.iteye.com/blog/2368927" target="_blank" rel="noopener">mysql的replace into“坑” </a></strong> </p>]]></content>
<categories>
<category> MySQL </category>
</categories>
<tags>
<tag> MySQL，SQL </tag>
</tags>
</entry>
<entry>
<title>Linux下性能分析工具</title>
<link href="/2018/08/23/Linux%E4%B8%8B%E6%80%A7%E8%83%BD%E5%88%86%E6%9E%90%E5%B7%A5%E5%85%B7/"/>
<url>/2018/08/23/Linux%E4%B8%8B%E6%80%A7%E8%83%BD%E5%88%86%E6%9E%90%E5%B7%A5%E5%85%B7/</url>
<content type="html"><![CDATA[<p>linux下性能分析工具还有很多，在平常检测系统性能的时候主要关注的无非是cpu,内存，磁盘io,网络带宽等。</p><h1 id="top命令"><a href="#top命令" class="headerlink" title="top命令"></a>top命令</h1><p>top命令工可以查看的状态就比较多了，他可以查看cpu，内存，平均负载，磁盘io等状况，使用也很简单，直接在命令行输入top。但是top对资源占用比较高，少用</p><p><img src="/2018/08/23/Linux下性能分析工具/top.png" alt=""></p><p>top的 -b 选项开启批处理模式，将每次刷新全部打印到stdout</p><p>top的 -n 选项指定退出top命令前刷新多少次信息。</p><p>top命令的输出：</p><p><strong>第1行：主要关注最后三列就是系统的平均负载：</strong><br> 即系统1分钟、5分钟、15分钟内的平均负载，判断一个系统负载是否偏高需要计算单核CPU的平均负载，这里显示的系统平均负载 / CPU核数，一般以0.7为比较合适的值。偏高说明有比较多的进程在等待使用CPU资源。<br> 如果1分钟平均负载很高，而15分钟平均负载很低，说明服务器正在命令高负载情况，需要进一步排查CPU资源都消耗在了哪里。反之，如果15分钟平均负载很高，1分钟平均负载较低，则有可能是CPU资源紧张时刻已经过去。</p><p>其他查看平均负载的命令还有</p><ul><li>tload: 能够绘制出负载变化的图形</li><li>uptime:显示平均负载的同时，还显示开机以来的时间，和top的第一行一样，如果想要持续观察平均负载，可以使用watch uptime,默认刷新时间是两秒</li><li>w: 显示uptime的信息以外，还同时显示已登录的用户</li></ul><p><strong>第3行：当前的CPU运行情况：</strong></p><ul><li><p>us：非nice用户进程占用CPU的比率</p></li><li><p>sy：内核、内核进程占用CPU的比率；</p></li><li><p>ni：如果一些用户进程修改过优先级，这里显示这些进程占用CPU时间的比率；</p></li><li><p>id：CPU空闲比率，如果系统缓慢而这个值很高，说明系统慢的原因不是CPU负载高；</p></li><li><p>wa：CPU等待执行I/O操作的时间比率，该指标可以用来排查磁盘I/O的问题，通常结合wa和id判断</p></li><li><p>hi：CPU处理硬件终端所占时间的比率；</p></li><li><p>si：CPU处理软件终端所占时间的比率；</p></li><li><p>st：流逝的时间，虚拟机中的其他任务所占CPU时间的比率；</p></li></ul><p> 用户进程占比高，wa低，说明系统缓慢的原因在于进程占用大量CPU，通常还会伴有教低的id，说明CPU空转时间很少；</p><p> wa低，id高，可以排除CPU资源瓶颈的可能。 </p><p> wa高，说明I/O占用了大量的CPU时间，需要检查交换空间的使用，交换空间位于磁盘上，性能远低于内存，当内存耗尽开始使用交换空间时，将会给性能带来严重影响，所以对于性能要求较高的服务器，一般建议关闭交换空间。另一方面，如果内存充足，但wa很高，说明需要检查哪个进程占用了大量的I/O资源。</p><h1 id="mpstat"><a href="#mpstat" class="headerlink" title="mpstat"></a>mpstat</h1><p>该命令可以显示每个CPU的占用情况，如果有一个CPU占用率特别高，那么有可能是一个单线程应用程序引起的。</p><p>mpstat -P ALL 1</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">mpstat [-P {|ALL}] [internal [count]]</span><br><span class="line">-P {|ALL} 表示监控哪个CPU， cpu在[0,cpu个数-1]中取值</span><br><span class="line">internal 相邻的两次采样的间隔时间、</span><br><span class="line">count 采样的次数，count只能和delay一起使用</span><br><span class="line">当没有参数时，mpstat则显示系统启动以后所有信息的平均值。有interval时，第一行的信息自系统启动以来的平均信息。从第二行开始，输出为前一个interval时间段的平均信息。</span><br></pre></td></tr></table></figure><p><img src="/2018/08/23/Linux下性能分析工具/mpstat.png" alt=""></p><p>字段的含义如下:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">%user 在internal时间段里，用户态的CPU时间(%)，不包含nice值为负进程 (usr/total)*100</span><br><span class="line">%nice 在internal时间段里，nice值为负进程的CPU时间(%) (nice/total)*100</span><br><span class="line">%sys 在internal时间段里，内核时间(%) (system/total)*100</span><br><span class="line">%iowait 在internal时间段里，硬盘IO等待时间(%) (iowait/total)*100</span><br><span class="line">%irq 在internal时间段里，硬中断时间(%) (irq/total)*100</span><br><span class="line">%soft 在internal时间段里，软中断时间(%) (softirq/total)*100</span><br><span class="line">%idle 在internal时间段里，CPU除去等待磁盘IO操作外的因为任何原因而空闲的时间闲置时间(%) (idle/total)*100</span><br></pre></td></tr></table></figure><p>计算公式如下</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">total_cur=user+system+nice+idle+iowait+irq+softirq</span><br><span class="line">total_pre=pre_user+ pre_system+ pre_nice+ pre_idle+ pre_iowait+ pre_irq+ pre_softirq</span><br><span class="line">user=user_cur – user_pre</span><br><span class="line">total=total_cur-total_pre</span><br><span class="line">其中_cur 表示当前值，_pre表示interval时间前的值。上表中的所有值可取到两位小数点。</span><br></pre></td></tr></table></figure>]]></content>
<categories>
<category> Linux </category>
</categories>
<tags>
<tag> Linux,系统分析 </tag>
</tags>
</entry>
<entry>
<title>goroutine scheduler</title>
<link href="/2018/08/14/goroutine-scheduler/"/>
<url>/2018/08/14/goroutine-scheduler/</url>
<content type="html"><![CDATA[<h2 id="一、Goroutine调度器"><a href="#一、Goroutine调度器" class="headerlink" title="一、Goroutine调度器"></a>一、Goroutine调度器</h2><p>goroutine是golang内置的协程，当我需要并发执行一些任务的时候，在go语言中可以使用go关键字来创建goroutine。</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span>{</span><br><span class="line"> <span class="comment">// dosth</span></span><br><span class="line">}()</span><br></pre></td></tr></table></figure><p>相比于c++和java等语言创建线程，go语言创建的goroutine是在进程和线程的基础上做更高层次的抽象，Go采用了<strong>用户层轻量级thread</strong>或者说是<strong>类coroutine</strong>的概念来解决这些问题，Go将之称为”<strong>goroutine</strong>“。goroutine占用的资源非常小(<a href="http://tonybai.com/2014/11/04/some-changes-in-go-1-4/" target="_blank" rel="noopener">Go 1.4</a>将每个goroutine stack的size默认设置为2k)，goroutine调度的切换也不用陷入(trap)操作系统内核层完成，代价很低。因此，一个Go程序中可以创建成千上万个并发的goroutine。所有的Go代码都在goroutine中执行，哪怕是go的runtime也不例外。将这些goroutines按照一定算法放到“<em>CPU</em>”上执行的程序就称为<strong>goroutine调度器</strong>或<strong>goroutine scheduler</strong>。</p><p>但是针对操作系统层面，操作系统是不知道goroutine的存在的，goroutine的调度全部是靠自己内部完成的，实现Go程序内goroutine之间“公平”的竞争“CPU”资源，这个任务就落到了Go runtime头上，要知道在一个Go程序中，除了用户代码，剩下的就是go runtime了。</p><h2 id="二、Go调度器模型与演化过程"><a href="#二、Go调度器模型与演化过程" class="headerlink" title="二、Go调度器模型与演化过程"></a>二、Go调度器模型与演化过程</h2><p>goroutine是通过三种基本对象互相协作(GMP)，来实现在用户空间管理和调度并发任务。</p><p>基本关系是</p><p><img src="/2018/08/14/goroutine-scheduler/sketch.png" alt="来自go语言学习笔记"></p><p><strong>此图来自雨痕的go语言学习笔记</strong></p><blockquote><ol><li><p>首先是Processor(简称P):</p><p> 他的作用类似于CPU核，用来控制可同时并发执行的任务数，每个工作线程都必须绑定一个有效P才被允许执行任务。否则只能休眠。直到有空闲P时被唤醒，P还为线程提供执行资源，比如<code>内存分配</code>，<code>本地任务队列</code>等。线程独享所绑定的P资源，可以在无锁状态下执行高效操作。</p></li><li><p>其次是Goroutine(简称G):</p><p> 进程内一切都在以goroutine方式运行，包括运行时相关的服务，以及mani.main入口函数。需要指出，G并非执行体，他仅仅保存并发任务状态，为任务执行提供所需的栈内存空间。G任务创建后被放置在P本地队列火全局队列，等待工作线程调度执行</p></li><li><p>最后是系统线程machine(简称M):</p><p> 实际执行体是系统线程和p绑定，以调度循环方式不停执行G并发任务。M通过修改寄存器，将执行栈指向G自带的栈内存，并在此空间内分配堆栈帧，执行任务函数。当需要中途切换的时候，只要将相关寄存器值保存回G空间即可维持状态，任何M都可以据此恢复执行。线程仅负责执行。不在持有状态，这是并发任务跨线程调度，实现多路复用的根本所在。</p></li></ol></blockquote><p>虽然<strong>P/M</strong>构成执行组合体，但两者数量不是一一对应的。通常情况下，p的数量相对恒定，默认是cpu的核心数。但是也可以更多或者更少，可以通过<code>runtime.GOMAXPROCS()</code>函数来设置。但是<strong>M</strong>则是由调度器按需创建的，举例来说，当M因陷入系统调用而长时间阻塞时，p会被健康线程抢回去，去新建（或唤醒）一个去执行其他任务，这样M的数量就会增长。</p><p><strong>优点：</strong></p><p> 因为G初始栈仅有2KB,且创建操作只是在用户空间简单的分配对象，远比要进入内核态分配线程要简单的多。</p><p>调度器让多个M进入调度循环，不停获取并执行任务，这样就可以创建成千上万个并发任务。</p><p><strong>疑问：</strong></p><p> 其实按照上面的描述，goroutine调度队列只需要有M和g就可以了，用户呢创建goroutine，go运行时机制去创建线程来调度goroutine就可以，为什么要增加一个p来作为中间层呢？</p><p><strong>解析：</strong></p><p> 查了资料之后，发现原来<a href="https://blog.golang.org/go-version-1-is-released" target="_blank" rel="noopener">Go 1.0正式发布</a>的时候确实是实现的G-M模型，并没有P的存在，但是此模型存在一系列不足，前Intel blackbelt工程师、现Google工程师<a href="https://github.com/dvyukov" target="_blank" rel="noopener">Dmitry Vyukov</a>在其《<a href="https://docs.google.com/document/d/1TTj4T2JO42uD5ID9e89oa0sLKhJYD0Y_kqxDv3I3XMw/edit#!" target="_blank" rel="noopener">Scalable Go Scheduler Design</a>》一文中指出了<strong>G-M模型</strong>的一个重要不足： 限制了Go并发程序的伸缩性，尤其是对那些有高吞吐或并行计算需求的服务程序。主要体现在如下几个方面：</p><blockquote><ol><li>单一全局互斥锁(Sched.Lock)和集中状态存储的存在导致所有goroutine相关操作，比如：创建、重新调度等都要上锁；</li><li>goroutine传递问题：M经常在M之间传递”可运行”的goroutine，这导致调度延迟增大以及额外的性能损耗；</li><li>每个M做内存缓存，导致内存占用过高，数据局部性较差；</li><li>由于syscall调用而形成的剧烈的worker thread阻塞和解除阻塞，导致额外的性能损耗</li></ol></blockquote><p>于是Dmitry Vyukov亲自操刀改进Go scheduler，在<a href="https://golang.org/doc/go1.1" target="_blank" rel="noopener">Go 1.1</a>中实现了<strong>G-P-M调度模型</strong>和<a href="http://supertech.csail.mit.edu/papers/steal.pdf" target="_blank" rel="noopener">work stealing算法</a>，这个模型一直沿用至今</p><blockquote><ul><li>G: 表示goroutine，存储了goroutine的执行stack信息、goroutine状态以及goroutine的任务函数等；另外G对象是可以重用的。</li><li>P: 表示逻辑processor，P的数量决定了系统内最大可<strong>并行</strong>的G的数量（前提：系统的物理cpu核数>=P的数量）；P的最大作用还是其拥有的各种G对象队列、链表、一些cache和状态。</li><li>M: M代表着真正的执行计算资源。在绑定有效的p后，进入schedule循环；而schedule循环的机制大致是从各种队列、p的本地队列中获取G，切换到G的执行栈上并执行G的函数，调用goexit做清理工作并回到m，如此反复。M并不保留G状态，这是G可以跨M调度的基础。</li></ul></blockquote><p><img src="/2018/08/14/goroutine-scheduler/scheduler.jpg" alt="preview"></p><ul><li><p>当一个OS线程M0陷入阻塞时(一般是channel阻塞或network I/O阻塞或者system call阻塞)，P转而在OS线程M1上运行。调度器保证有足够的线程来运行所以的context P。图中的M1可能是被创建，或者从线程缓存中取出。当M0返回时，它必须尝试取得一个context P来运行goroutine，一般情况下，它会从其他的OS线程那里steal偷一个context P过来，如果没有偷到的话，它就把goroutine放在一个global runqueue里，然后自己就去睡大觉了（放入线程缓存里）。Contexts P们也会周期性的检查global runqueue，否则global runqueue上的goroutine永远无法执行</p></li><li><p>另一种情况是P所分配的任务G很快就执行完了（分配不均），这就导致了一个上下文P闲着没事儿干而系统却任然忙碌。但是如果global runqueue没有任务G了，那么P就不得不从其他的上下文P那里拿一些G来执行。一般来说，如果上下文P从其他的上下文P那里要偷一个任务的话，一般就‘偷’run queue的一半，这就确保了每个OS线程都能充分的使用。</p></li></ul><h2 id="三、调度器状态的查看方法"><a href="#三、调度器状态的查看方法" class="headerlink" title="三、调度器状态的查看方法"></a>三、调度器状态的查看方法</h2><p>Go提供了调度器当前状态的查看方法：使用Go运行时环境变量GODEBUG。 </p><p>GODEBUG这个Go运行时环境变量很是强大，通过给其传入不同的key1=value1,key2=value2… 组合，Go的runtime会输出不同的调试信息，比如在这里我们给GODEBUG传入了”schedtrace=1000″，其含义就是每1000ms，打印输出一次goroutine scheduler的状态，每次一行。每一行各字段含义如下：</p><blockquote><p>SCHED 6016ms: gomaxprocs=4 idleprocs=0 threads=26 spinningthreads=0 idlethreads=20 runqueue=1 [3 4 0 10] </p><p>SCHED：调试信息输出标志字符串，代表本行是goroutine scheduler的输出；</p><p> 6016ms：即从程序启动到输出这行日志的时间； </p><p>gomaxprocs: P的数量；</p><p> idleprocs: 处于idle状态的P的数量；通过gomaxprocs和idleprocs的差值，我们就可知道执行go代码的P的数量； </p><p>threads: os threads的数量，包含scheduler使用的m数量，加上runtime自用的类似sysmon这样的thread的数量； </p><p>spinningthreads: 处于自旋状态的os thread数量； </p><p>idlethread: 处于idle状态的os thread的数量；</p><p> runqueue=1： go scheduler全局队列中G的数量；</p><p> [3 4 0 10]: 分别为4个P的local queue中的G的数量。 </p></blockquote><h1 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h1><p><a href="https://tonybai.com/2017/06/23/an-intro-about-goroutine-scheduler/" target="_blank" rel="noopener">也谈goroutine调度器</a></p><p>Go语言学习笔记<br><a href="https://www.zhihu.com/question/20862617" target="_blank" rel="noopener">Golang 的 goroutine 是如何实现的？</a></p>]]></content>
<categories>
<category> Golang </category>
</categories>
<tags>
<tag> golang </tag>
</tags>
</entry>
<entry>
<title>Go里面的堆栈跟踪</title>
<link href="/2018/07/18/Go%E9%87%8C%E9%9D%A2%E7%9A%84%E5%A0%86%E6%A0%88%E8%B7%9F%E8%B8%AA/"/>
<url>/2018/07/18/Go%E9%87%8C%E9%9D%A2%E7%9A%84%E5%A0%86%E6%A0%88%E8%B7%9F%E8%B8%AA/</url>
<content type="html"><![CDATA[<p>转载翻译，原文地址：<a href="https://www.ardanlabs.com/blog/2015/01/stack-traces-in-go.html" target="_blank" rel="noopener">Stack Traces In Go</a></p><h1 id="Go里面的堆栈跟踪"><a href="#Go里面的堆栈跟踪" class="headerlink" title="Go里面的堆栈跟踪"></a>Go里面的堆栈跟踪</h1><p>在Go语言中有一些调试技巧能帮助我们快速找到问题，有时候你想尽可能多的记录异常但仍觉得不够，搞清楚堆栈的意义有助于定位Bug或者记录更完整的信息。</p><p>本文将讨论堆栈跟踪信息以及如何在堆栈中识别函数所传递的参数。</p><h2 id="Functions-函数的情况"><a href="#Functions-函数的情况" class="headerlink" title="Functions (函数的情况)"></a>Functions (函数的情况)</h2><p>先从这段代码开始：</p><h3 id="清单1"><a href="#清单1" class="headerlink" title="清单1"></a>清单1</h3><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> slice := <span class="built_in">make</span>([]<span class="keyword">string</span>, <span class="number">2</span>, <span class="number">4</span>)</span><br><span class="line"> Example(slice, <span class="string">"hello"</span>, <span class="number">10</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">Example</span><span class="params">(slice []<span class="keyword">string</span>, str <span class="keyword">string</span>, i <span class="keyword">int</span>)</span></span> {</span><br><span class="line"> <span class="built_in">panic</span>(<span class="string">"Want stack trace"</span>)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><strong>清单1</strong>显示了一个程序，其中main函数在第05行调用Example函数.Example函数在第08行声明并接受三个参数，1个string类型的slice, 1个string和1个integer, 。 Example执行的唯一代码是调用第09行的内置函数panic，它会立即生成堆栈跟踪：</p><h3 id="清单2"><a href="#清单2" class="headerlink" title="清单2"></a>清单2</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line">Panic: Want stack trace</span><br><span class="line"></span><br><span class="line">goroutine 1 [running]:</span><br><span class="line">main.Example(0x2080c3f50, 0x2, 0x4, 0x425c0, 0x5, 0xa)</span><br><span class="line"> /Users/bill/Spaces/Go/Projects/src/github.com/goinaction/code/</span><br><span class="line"> temp/main.go:9 +0x64</span><br><span class="line">main.main()</span><br><span class="line"> /Users/bill/Spaces/Go/Projects/src/github.com/goinaction/code/</span><br><span class="line"> temp/main.go:5 +0x85</span><br><span class="line"></span><br><span class="line">goroutine 2 [runnable]:</span><br><span class="line">runtime.forcegchelper()</span><br><span class="line"> /Users/bill/go/src/runtime/proc.go:90</span><br><span class="line">runtime.goexit()</span><br><span class="line"> /Users/bill/go/src/runtime/asm_amd64.s:2232 +0x1</span><br><span class="line"></span><br><span class="line">goroutine 3 [runnable]:</span><br><span class="line">runtime.bgsweep()</span><br><span class="line"> /Users/bill/go/src/runtime/mgc0.go:82</span><br><span class="line">runtime.goexit()</span><br><span class="line"> /Users/bill/go/src/runtime/asm_amd64.s:2232 +0x1</span><br></pre></td></tr></table></figure><p><strong>清单2</strong>中的堆栈跟踪显示了panic是存在的所有goroutine,每个程序的状态以及相应goroutine下的调用堆栈。</p><p>正在运行的goroutine和导致堆栈跟踪的goroutine将位于顶部。让我们关注报了panic的goroutine.</p><h3 id="清单3"><a href="#清单3" class="headerlink" title="清单3"></a>清单3</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">01 goroutine 1 [running]:</span><br><span class="line">02 main.Example(0x2080c3f50, 0x2, 0x4, 0x425c0, 0x5, 0xa)</span><br><span class="line"> /Users/bill/Spaces/Go/Projects/src/github.com/goinaction/code/</span><br><span class="line"> temp/main.go:9 +0x64</span><br><span class="line">03 main.main()</span><br><span class="line"> /Users/bill/Spaces/Go/Projects/src/github.com/goinaction/code/</span><br><span class="line"> temp/main.go:5 +0x85</span><br></pre></td></tr></table></figure><p>清单三中地 01 行的的堆栈跟踪显示goroutine 1 在panic之前运行，在第 02 行，我们看到panic的代码在package main中的Example函数中。缩进的行显示了次函数所在的代码文件和路径以及正在执行的代码行。在这种情况下，第 09 行的代码正在运行，这是对panic的调用。</p><p>第 03 行显示调用Example的函数的名称，这是main包中的主要功能，在函数名称下面，缩进的行显示了对Example进行调用的代码文件的路径和代码行</p><p>堆栈工资显示goroutine范围内的函数调用链，直到发生panic发生，现在让我们关注传递给Example函数的每个参数的值：</p><h3 id="清单4"><a href="#清单4" class="headerlink" title="清单4"></a>清单4</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">// Declaration</span><br><span class="line">main.Example(slice []string, str string, i int)</span><br><span class="line"></span><br><span class="line">// Call to Example by main.</span><br><span class="line">slice := make([]string, 2, 4)</span><br><span class="line">Example(slice, "hello", 10)</span><br><span class="line"></span><br><span class="line">// Stack trace</span><br><span class="line">main.Example(0x2080c3f50, 0x2, 0x4, 0x425c0, 0x5, 0xa)</span><br></pre></td></tr></table></figure><p><strong>清单4</strong> 这里展示了在main中带参数调用Example函数时的堆栈信息 。 将堆栈跟踪中的值与函数声明进行比较时，它似乎不匹配。 Example函数的声明接受三个参数，但堆栈跟踪显示六个十六进制值。 要理解值如何与参数匹配的关键需要知道每个参数类型的实现。</p><p>让我们从第一个参数开始，它是一个1个string类型的slice， slice是Go中的引用类型。 这意味着slice的值是一个标题值，其中包含指向某些基础数据的指针。 在slice的情况下，标头值是三字结构，其包含指向底层阵列的指针，slice的长度和容量。 与切片标头关联的值由堆栈跟踪中的前三个值表示：</p><h3 id="清单5"><a href="#清单5" class="headerlink" title="清单5"></a>清单5</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">// Slice parameter value</span><br><span class="line">slice := make([]string, 2, 4)</span><br><span class="line"></span><br><span class="line">// Slice header values</span><br><span class="line">Pointer: 0x2080c3f50</span><br><span class="line">Length: 0x2</span><br><span class="line">Capacity: 0x4</span><br><span class="line"></span><br><span class="line">// Declaration</span><br><span class="line">main.Example(slice []string, str string, i int)</span><br><span class="line"></span><br><span class="line">// Stack trace</span><br><span class="line">main.Example(0x2080c3f50, 0x2, 0x4, 0x425c0, 0x5, 0xa)</span><br></pre></td></tr></table></figure><p><strong>清单5</strong> 显示了堆栈跟踪中的前三个值如何与slice参数匹配。 第一个值表示指向底层字符串数组的指针。 用于初始化slice的长度和容量的数字与第二个和第三个值匹配 。这三个值表示切片标头的每个值，即Example函数的第一个参数。</p><p><strong>Figure 1</strong></p><p><img src="/2018/07/18/Go里面的堆栈跟踪/Figure3.png" alt=""></p><p>现在让我们看一下第二个参数，它是一个string。 string也是引用类型，但此标头值是不可变的。 字符串的标头值被声明为两部分，包含指向底层字节数组的指针和字符串的长度：</p><h3 id="清单6"><a href="#清单6" class="headerlink" title="清单6"></a>清单6</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">// String parameter value</span><br><span class="line">"hello"</span><br><span class="line"></span><br><span class="line">// String header values</span><br><span class="line">Pointer: 0x425c0</span><br><span class="line">Length: 0x5</span><br><span class="line"></span><br><span class="line">// Declaration</span><br><span class="line">main.Example(slice []string, str string, i int)</span><br><span class="line"></span><br><span class="line">// Stack trace</span><br><span class="line">main.Example(0x2080c3f50, 0x2, 0x4, 0x425c0, 0x5, 0xa)</span><br></pre></td></tr></table></figure><p><strong>清单6</strong>显示了堆栈跟踪中的第四个和第五个值如何与string参数匹配。 第四个值表示指向底层字节数组的指针，第五个值表示字符串的长度为5。字符串</p><p>“hello”</p><p>需要5个字节。 这两个值表示字符串标题的每个值，即Example函数的第二个参数。</p><p><strong>Figure 2</strong></p><p><img src="/2018/07/18/Go里面的堆栈跟踪/Figure2.png" alt=""></p><p>第三个参数是一个整数，它是一个单值：</p><h3 id="清单7"><a href="#清单7" class="headerlink" title="清单7"></a>清单7</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">// Integer parameter value</span><br><span class="line">10</span><br><span class="line"></span><br><span class="line">// Integer value</span><br><span class="line">Base 16: 0xa</span><br><span class="line"></span><br><span class="line">// Declaration</span><br><span class="line">main.Example(slice []string, str string, i int)</span><br><span class="line"></span><br><span class="line">// Stack trace</span><br><span class="line">main.Example(0x2080c3f50, 0x2, 0x4, 0x425c0, 0x5, 0xa)</span><br></pre></td></tr></table></figure><p><strong>清单7</strong>显示了堆栈跟踪中的最后一个值如何与int类型的参数匹配。 trace中的最后一个值是十六进制数0xa，它的值是10.与该参数传递的值相同。 该值代表Example函数中的第三个参数。</p><p><strong>Figure 3</strong></p><p><img src="/2018/07/18/Go里面的堆栈跟踪/Figure3.png" alt=""></p><h2 id="Methods-方法的情况"><a href="#Methods-方法的情况" class="headerlink" title="Methods(方法的情况)"></a>Methods(方法的情况)</h2><p>如果我们将Example作为结构体的方法会怎么样呢?</p><h3 id="清单8"><a href="#清单8" class="headerlink" title="清单8"></a>清单8</h3><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="number">01</span> <span class="keyword">package</span> main</span><br><span class="line"><span class="number">02</span></span><br><span class="line"><span class="number">03</span> <span class="keyword">import</span> <span class="string">"fmt"</span></span><br><span class="line"><span class="number">04</span></span><br><span class="line"><span class="number">05</span> <span class="keyword">type</span> trace <span class="keyword">struct</span>{}</span><br><span class="line"><span class="number">06</span></span><br><span class="line"><span class="number">07</span> <span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"><span class="number">08</span> slice := <span class="built_in">make</span>([]<span class="keyword">string</span>, <span class="number">2</span>, <span class="number">4</span>)</span><br><span class="line"><span class="number">09</span></span><br><span class="line"><span class="number">10</span> <span class="keyword">var</span> t trace</span><br><span class="line"><span class="number">11</span> t.Example(slice, <span class="string">"hello"</span>, <span class="number">10</span>)</span><br><span class="line"><span class="number">12</span> }</span><br><span class="line"><span class="number">13</span></span><br><span class="line"><span class="number">14</span> <span class="function"><span class="keyword">func</span> <span class="params">(t *trace)</span> <span class="title">Example</span><span class="params">(slice []<span class="keyword">string</span>, str <span class="keyword">string</span>, i <span class="keyword">int</span>)</span></span> {</span><br><span class="line"><span class="number">15</span> fmt.Printf(<span class="string">"Receiver Address: %p\n"</span>, t)</span><br><span class="line"><span class="number">16</span> <span class="built_in">panic</span>(<span class="string">"Want stack trace"</span>)</span><br><span class="line"><span class="number">17</span> }</span><br></pre></td></tr></table></figure><p><strong>清单8</strong>通过在第05行声明一个名为trace的新类型，并更改程序，将Example申明为trace类型的方法。通过使用trace类型的指针接收器重新声明该函数来完成转换。 然后在第10行，将变量t申明为trace类型，并且在第11行进行方法调用。</p><p>由于该方法是使用指针声明的，因此Go将获取变量t的地址来支持接收者类型，即使方法调用是使用值来完成的。 这次运行程序时，堆栈跟踪有点不同：</p><h3 id="清单9"><a href="#清单9" class="headerlink" title="清单9"></a>清单9</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">Receiver Address: 0x1553a8</span><br><span class="line">panic: Want stack trace</span><br><span class="line"></span><br><span class="line">01 goroutine 1 [running]:</span><br><span class="line">02 main.(*trace).Example(0x1553a8, 0x2081b7f50, 0x2, 0x4, 0xdc1d0, 0x5, 0xa)</span><br><span class="line"> /Users/bill/Spaces/Go/Projects/src/github.com/goinaction/code/</span><br><span class="line"> temp/main.go:16 +0x116</span><br><span class="line"></span><br><span class="line">03 main.main()</span><br><span class="line"> /Users/bill/Spaces/Go/Projects/src/github.com/goinaction/code/</span><br><span class="line"> temp/main.go:11 +0xae</span><br></pre></td></tr></table></figure><p>在<strong>清单9</strong>中你应该注意的第一件事是第02行的堆栈跟踪清楚的显示这是一个使用指针接收器调用的方法。现在函数的名称显示的样子是： 在package名字和方法名之间多出了”*trace”字样 。 需要注意的第二件事是参数列表的第1个参数标明了结构体(t)地址。 我们从堆栈跟踪中看到了这个实现细节。</p><h2 id="Packing（打包）"><a href="#Packing（打包）" class="headerlink" title="Packing（打包）"></a>Packing（打包）</h2><p>如果有多个参数可以填充到一个single word, 那么堆栈跟踪中参数的值将打包在一起 ：</p><h3 id="清单10"><a href="#清单10" class="headerlink" title="清单10"></a>清单10</h3><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="number">01</span> <span class="keyword">package</span> main</span><br><span class="line"><span class="number">02</span></span><br><span class="line"><span class="number">03</span> <span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"><span class="number">04</span> Example(<span class="literal">true</span>, <span class="literal">false</span>, <span class="literal">true</span>, <span class="number">25</span>)</span><br><span class="line"><span class="number">05</span> }</span><br><span class="line"><span class="number">06</span> </span><br><span class="line"><span class="number">07</span> <span class="function"><span class="keyword">func</span> <span class="title">Example</span><span class="params">(b1, b2, b3 <span class="keyword">bool</span>, i <span class="keyword">uint8</span>)</span></span> {</span><br><span class="line"><span class="number">08</span> <span class="built_in">panic</span>(<span class="string">"Want stack trace"</span>)</span><br><span class="line"><span class="number">09</span> }</span><br></pre></td></tr></table></figure><p>这个例子修改Example函数改为接收4个参数：3个bool型和1个八位无符号整型。bool值也是用8个bit表示，所以在32位和64位架构下，4个参数可以合并为一个single word。 当程序运行时，它会产生一个有趣的堆栈跟踪 :</p><h3 id="清单11"><a href="#清单11" class="headerlink" title="清单11"></a>清单11</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">01 goroutine 1 [running]:</span><br><span class="line">02 main.Example(0x19010001)</span><br><span class="line"> /Users/bill/Spaces/Go/Projects/src/github.com/goinaction/code/</span><br><span class="line"> temp/main.go:8 +0x64</span><br><span class="line">03 main.main()</span><br><span class="line"> /Users/bill/Spaces/Go/Projects/src/github.com/goinaction/code/</span><br><span class="line"> temp/main.go:4 +0x32</span><br></pre></td></tr></table></figure><p>对于对Example的调用，堆栈跟踪中没有四个值，而是有一个值。所有四个单独的8位值都拼凑成一个单词：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">// Parameter values</span><br><span class="line">true, false, true, 25</span><br><span class="line"></span><br><span class="line">// Word value</span><br><span class="line">Bits Binary Hex Value</span><br><span class="line">00-07 0000 0001 01 true</span><br><span class="line">08-15 0000 0000 00 false</span><br><span class="line">16-23 0000 0001 01 true</span><br><span class="line">24-31 0001 1001 19 25</span><br><span class="line"></span><br><span class="line">// Declaration</span><br><span class="line">main.Example(b1, b2, b3 bool, i uint8)</span><br><span class="line"></span><br><span class="line">// Stack trace</span><br><span class="line">main.Example(0x19010001)</span><br></pre></td></tr></table></figure><p><strong>清单12</strong>显示了堆栈跟踪中的值是如何与传入的所有四个参数值匹配.true的值是一个8位值，用1表示，false的值是0.二进制25的值是11001，转换为十六进制是19。 现在，我们看到堆栈信息中包括十六进制值，需要知道这些值是如何传递的。</p><h1 id="结论"><a href="#结论" class="headerlink" title="结论"></a>结论</h1><p>Go运行时提供了大量信息来帮助我们调试程序。在这篇文章中，我们专注于堆栈跟踪。分析在整个调用堆栈中传递给每个函数的值的能力是很有用的。它不止一次帮助我很快识别我的错误。既然您已经知道如何读取堆栈跟踪，那么希望您可以在下次发生堆栈跟踪时利用这些知识。</p>]]></content>
<categories>
<category> Golang </category>
</categories>
<tags>
<tag> golang </tag>
</tags>
</entry>
<entry>
<title>Consul安装部署</title>
<link href="/2018/07/13/Consul%E5%AE%89%E8%A3%85%E9%83%A8%E7%BD%B2/"/>
<url>/2018/07/13/Consul%E5%AE%89%E8%A3%85%E9%83%A8%E7%BD%B2/</url>
<content type="html"><![CDATA[<h1 id="什么是Consul"><a href="#什么是Consul" class="headerlink" title="什么是Consul"></a>什么是Consul</h1><ul><li>是一个服务管理软件。</li><li>支持多数据中心下，分布式高可用的，服务发现和配置共享。</li><li>consul支持健康检查，允许存储键值对。</li><li>一致性协议采用 Raft 算法,用来保证服务的高可用.</li><li>成员管理和消息广播 采用GOSSIP协议，支持ACL访问控制。</li></ul><p><strong>ACL技术</strong></p><p>在路由器中被广泛采用，它是一种基于包过滤的流控制技术。控制列表通过把源地址、目的地址及端口号作为数据包检查的基本元素，并可以规定符合条件的数据包是否允许通过。</p><p><strong>gossip就是p2p协议。</strong></p><p>他主要要做的事情是，去中心化。<br>这个协议就是模拟人类中传播谣言的行为而来。首先要传播谣言就要有种子节点。种子节点每秒都会随机向其他节点发送自己所拥有的节点列表，以及需要传播的消息。任何新加入的节点，就在这种传播方式下很快地被全网所知道。</p><p><strong>什么是强一致性协议？</strong></p><p>按照某一顺序串行执行存储对象读写操作， 更新存储对象之后， 后续访问总是读到最新值。 假如进程A先更新了存储对象，存储系统保证后续A,B,C进程的读取操作都将返回最新值。强一致性模型有几种常见实现方法， 主从同步复制， 以及quorum复制等。</p><p>Consul is opinionated in its usage while Serf is a more flexible and general purpose tool. In CAP terms, Consul uses a CP architecture, favoring consistency over availability.</p><p><a href="https://www.consul.io/intro/vs/serf.html" target="_blank" rel="noopener">官方文档地址</a></p><p>说明consul是cp的，并不是网上有些文章说的是ca模式</p><p>下面表格对consul 、zookeeper、 etcd、 euerka做了对比</p><table><thead><tr><th>Feature</th><th>Consul</th><th>zookeeper</th><th>etcd</th><th>euerka</th></tr></thead><tbody><tr><td>服务健康检查</td><td>服务状态，内存，硬盘等</td><td>(弱)长连接，keepalive</td><td>连接心跳</td><td>可配支持</td></tr><tr><td>多数据中心</td><td>支持</td><td>—</td><td>—</td><td>—</td></tr><tr><td>kv存储服务</td><td>支持</td><td>支持</td><td>支持</td><td>—</td></tr><tr><td>一致性</td><td>raft</td><td>paxos</td><td>raft</td><td>—</td></tr><tr><td>cap</td><td>cp</td><td>cp</td><td>cp</td><td>ap</td></tr><tr><td>使用接口(多语言能力)</td><td>支持http和dns</td><td>客户端</td><td>http/grpc</td><td>http（sidecar）</td></tr><tr><td>watch支持</td><td>全量/支持long polling</td><td>支持</td><td>支持 long polling</td><td>支持 long polling/大部分增量</td></tr><tr><td>自身监控</td><td>metrics</td><td>—</td><td>metrics</td><td>metrics</td></tr><tr><td>安全</td><td>acl /https</td><td>acl</td><td>https支持（弱）</td><td>—</td></tr><tr><td>spring cloud集成</td><td>已支持</td><td>已支持</td><td>已支持</td><td>已支持</td></tr></tbody></table><h1 id="Consul安装"><a href="#Consul安装" class="headerlink" title="Consul安装"></a>Consul安装</h1><p>安装Consul,找到适合你系统的包下载他.Consul打包为一个’Zip’文件.<a href="https://www.consul.io/downloads.html" target="_blank" rel="noopener">前往下载</a></p><p>下载后解开压缩包.拷贝Consul到你的PATH路径中,在Unix系统中<code>/bin</code>和<code>/usr/local/bin</code>是通常的安装目录.根据你是想为单个用户安装还是给整个系统安装来选择.在Windows系统中有可以安装到<code>%PATH%</code>的路径中.</p><h2 id="验证安装"><a href="#验证安装" class="headerlink" title="验证安装"></a>验证安装</h2><p>完成安装后,通过打开一个新终端窗口检查<code>consul</code>安装是否成功.通过执行 <code>consul</code>你应该看到类似下面的输出</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line">[root@iZbp14ouog5ocoeakj39q1Z ~]# consul</span><br><span class="line">Usage: consul [--version] [--help] <command> [<args>]</span><br><span class="line"></span><br><span class="line">Available commands are:</span><br><span class="line"> agent Runs a Consul agent</span><br><span class="line"> catalog Interact with the catalog</span><br><span class="line"> connect Interact with Consul Connect</span><br><span class="line"> event Fire a new event</span><br><span class="line"> exec Executes a command on Consul nodes</span><br><span class="line"> force-leave Forces a member of the cluster to enter the "left" state</span><br><span class="line"> info Provides debugging information for operators.</span><br><span class="line"> intention Interact with Connect service intentions</span><br><span class="line"> join Tell Consul agent to join cluster</span><br><span class="line"> keygen Generates a new encryption key</span><br><span class="line"> keyring Manages gossip layer encryption keys</span><br><span class="line"> kv Interact with the key-value store</span><br><span class="line"> leave Gracefully leaves the Consul cluster and shuts down</span><br><span class="line"> lock Execute a command holding a lock</span><br><span class="line"> maint Controls node or service maintenance mode</span><br><span class="line"> members Lists the members of a Consul cluster</span><br><span class="line"> monitor Stream logs from a Consul agent</span><br><span class="line"> operator Provides cluster-level tools for Consul operators</span><br><span class="line"> reload Triggers the agent to reload configuration files</span><br><span class="line"> rtt Estimates network round trip time between nodes</span><br><span class="line"> snapshot Saves, restores and inspects snapshots of Consul server state</span><br><span class="line"> validate Validate config files/directories</span><br><span class="line"> version Prints the Consul version</span><br><span class="line"> watch Watch for changes in Consul</span><br></pre></td></tr></table></figure><p>如果你得到一个<code>consul not be found</code>的错误,你的<code>PATH</code>可能没有正确设置.请返回检查你的<code>consul</code>的安装路径是否包含在<code>PATH</code>中.</p><h2 id="consule参数的介绍"><a href="#consule参数的介绍" class="headerlink" title="consule参数的介绍"></a>consule参数的介绍</h2><h3 id="consul-术语"><a href="#consul-术语" class="headerlink" title="consul 术语"></a>consul 术语</h3><p>首先介绍下在 consul 中会经常见到的术语：</p><ul><li><code>node</code>：节点，需要 consul 注册发现或配置管理的服务器。在一个集群中必须是唯一的，默认是该节点的主机名</li><li><code>agent</code>：consul 中的核心程序，它将以守护进程的方式在各个节点运行，有 client 和 server 启动模式。每个 agent 维护一套服务和注册发现以及健康信息。</li><li><code>client</code>：agent 以 client 模式启动的节点。在该模式下，该节点会采集相关信息，通过 RPC 的方式向 server 发送。这个地址提供HTTP、DNS、RPC等服务，默认是127.0.0.1所以不对外提供服务，如果你要对外提供服务改成0.0.0.0</li><li><code>server</code>：agent 以 server 模式启动的节点。一个数据中心中至少包含 1 个 server 节点。不过官方建议使用 3 或 5 个 server 节点组建成集群，以保证高可用且不失效率。server 节点参与 Raft、维护会员信息、注册服务、健康检查等功能。</li><li><code>datacenter</code>：数据中心，私有的，低延迟的和高带宽的网络环境。一般的多个数据中心之间的数据是不会被复制的，但可用过 <a href="https://www.consul.io/docs/guides/acl.html#outages-and-acl-replication" target="_blank" rel="noopener">ACL replication</a> 或使用外部工具 <a href="https://github.com/hashicorp/consul-replicate" target="_blank" rel="noopener">onsul-replicate</a>。</li><li><code>Consensus</code>，<a href="https://www.consul.io/docs/internals/consensus.html" target="_blank" rel="noopener">共识协议</a>，使用它来协商选出 leader。</li><li><code>Gossip</code>：consul 是建立在 <a href="https://www.serf.io/" target="_blank" rel="noopener">Serf</a>，它提供完整的 <a href="https://www.consul.io/docs/internals/gossip.html" target="_blank" rel="noopener">gossip protocol</a>，<a href="https://en.wikipedia.org/wiki/Gossip_protocol" target="_blank" rel="noopener">维基百科</a>。</li><li><code>LAN Gossip</code>，Lan gossip 池，包含位于同一局域网或数据中心上的节点。</li><li><code>WAN Gossip</code>，只包含 server 的 WAN Gossip 池，这些服务器主要位于不同的数据中心，通常通过互联网或广域网进行通信。</li><li><code>members</code>：成员，对 consul 成员的称呼。提供会员资格，故障检测和事件广播。</li><li><code>-bootstrap-expect</code> ：在一个datacenter中期望提供的server节点数目，当该值提供的时候，consul一直等到达到指定sever数目的时候才会引导整个集群，该标记不能和bootstrap共用</li><li><code>-bind</code>：该地址用来在集群内部的通讯，集群内的所有节点到地址都必须是可达的，默认是0.0.0.0</li><li><code>-ui-dir</code>： 提供存放web ui资源的路径，该目录必须是可读的，<strong>1.2中是直接使用-ui参数就可以</strong></li><li><code>-rejoin</code>：使consul忽略先前的离开，在再次启动后仍旧尝试加入集群中。</li><li><code>-config-dir</code>：：配置文件目录，里面所有以.json结尾的文件都会被加载</li></ul><h3 id="consul-端口说明"><a href="#consul-端口说明" class="headerlink" title="consul 端口说明"></a>consul 端口说明</h3><p>consul 内使用了很多端口，理解这些端口的用处对你理解 consul 架构很有帮助：</p><table><thead><tr><th>端口</th><th>说明</th></tr></thead><tbody><tr><td>TCP/8300</td><td>8300 端口用于服务器节点。客户端通过该端口 RPC 协议调用服务端节点。服务器节点之间相互调用</td></tr><tr><td>TCP/UDP/8301</td><td>8301 端口用于单个数据中心所有节点之间的互相通信，即对 LAN 池信息的同步。它使得整个数据中心能够自动发现服务器地址，分布式检测节点故障，事件广播（如领导选举事件）。</td></tr><tr><td>TCP/UDP/8302</td><td>8302 端口用于单个或多个数据中心之间的服务器节点的信息同步，即对 WAN 池信息的同步。它针对互联网的高延迟进行了优化，能够实现跨数据中心请求。</td></tr><tr><td>8500</td><td>8500 端口基于 HTTP 协议，用于 API 接口或 WEB UI 访问。</td></tr><tr><td>8600</td><td>8600 端口作为 DNS 服务器，它使得我们可以通过节点名查询节点信息。</td></tr></tbody></table><h1 id="Consul运行"><a href="#Consul运行" class="headerlink" title="Consul运行"></a>Consul运行</h1><h2 id="开发模式运行consul"><a href="#开发模式运行consul" class="headerlink" title="开发模式运行consul"></a>开发模式运行consul</h2><p><img src="/2018/07/13/Consul安装部署/consul_dev.png" alt=""></p><ul><li>查看集群成员</li></ul><p>新开一个终端窗口运行<code>consul members</code>, 你可以看到Consul集群的成员.</p><p><img src="/2018/07/13/Consul安装部署/member_dev.png" alt=""></p><ul><li><p>浏览器查看webUI界面</p><p>浏览器中输出serverip:8500,会出现consul的管理webUI<br><img src="/home/zhaohq/blog/hexo/source/_posts/image/Consul安装部署/ui_dev.png" alt=""></p></li></ul><h2 id="生产环境运行consul"><a href="#生产环境运行consul" class="headerlink" title="生产环境运行consul"></a>生产环境运行consul</h2><h3 id="启动三台server服务器"><a href="#启动三台server服务器" class="headerlink" title="启动三台server服务器"></a>启动三台server服务器</h3><p> agent可以运行为server或client模式.每个数据中心至少必须拥有一台server . 建议在一个集群中有3或者5个server.部署单一的server,在出现失败时会不可避免的造成数据丢失.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">其他的agent运行为client模式.一个client是一个非常轻量级的进程.用于注册服务,运行健康检查和转发对server的查询.agent必须在集群中的每个主机上运行.</span><br></pre></td></tr></table></figure><p>这里启动三个agent server，三台机器的地址分别是</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">10.174.96.52 s1 </span><br><span class="line">10.173.224.146 s2</span><br><span class="line">10.173.224.74 s3</span><br></pre></td></tr></table></figure><p>必须有一个初始节点，且手动指定为leader，然后开启其它server节点，让它们加入集群。 最后初始节点下线，重新加入集群，参与选举。</p><p>我们手动指定10.174.96.52为leader,这种方式，<code>-bootstrap-expect 3</code> 期待三个 server 加入才能完成 consul 的引导。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">consul agent -server -bootstrap-expect 3 -data-dir /tmp/consul -node=s1 -bind=10.174.96.52 -ui -client 0.0.0.0</span><br></pre></td></tr></table></figure><p>继续添加 server2、server3：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">consul agent -server -data-dir /tmp/consul -node=s2 -bind=10.173.224.146 -ui -join 10.174.96.52</span><br><span class="line"></span><br><span class="line">consul agent -server -data-dir /tmp/consul -node=s3 -bind=10.173.224.74 -ui -join 10.174.96.52</span><br></pre></td></tr></table></figure><ul><li><code>-data-dir</code>：提供一个目录用来存放agent的状态，所有的agent允许都需要该目录，该目录必须是稳定的，系统重启后都继续存在</li><li><code>-join</code>：将agent加入到集群</li></ul><p><strong>此时10.174.96.52显示的信息是，当146主机和74主机加入到集群后，s1就被选举为leader</strong><br><img src="/2018/07/13/Consul安装部署/leader_52.png" alt=""></p><p><strong>10.173.224.146加入到集群中后显示的信息</strong>：<br><img src="/2018/07/13/Consul安装部署/candidate_146.png" alt=""></p><p><strong>查看webUI</strong><br><img src="/2018/07/13/Consul安装部署/ui_1.png" alt=""></p><p>查看三个server的名称<br><img src="/2018/07/13/Consul安装部署/ui_2.png" alt=""></p><h3 id="将新服务服务注册到consul"><a href="#将新服务服务注册到consul" class="headerlink" title="将新服务服务注册到consul"></a>将新服务服务注册到consul</h3><p>consul 支持两种服务发现的方式：</p><ol><li>通过 HTTP API 方式，这种方式需要额外编程，适用于不安装 consul agent 的情况，<a href="https://www.consul.io/api/catalog.html" target="_blank" rel="noopener">文档地址</a>。</li><li>通过 consul agent 配置的方式，agent 启动的时候会读取一个配置文件目录，通过配置进行服务的发现，<a href="https://www.consul.io/docs/agent/services.html" target="_blank" rel="noopener">文档地址</a>。</li></ol><p>这里介绍第二种方式，通过配置文件来进行服务发现。这里就需要用到我们的 client 服务器啦。</p><p>首先，用 Go 写一个简单的 HTTP 服务器：</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"> <span class="string">"fmt"</span></span><br><span class="line"> <span class="string">"net/http"</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">HandleExample</span><span class="params">(w http.ResponseWriter, r *http.Request)</span></span> {</span><br><span class="line"> w.Write([]<span class="keyword">byte</span>(<span class="string">"hello man"</span>))</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">HandleHealth</span><span class="params">(w http.ResponseWriter, r *http.Request)</span></span> {</span><br><span class="line"> fmt.Println(<span class="string">"health check!"</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> http.HandleFunc(<span class="string">"/"</span>, HandleExample)</span><br><span class="line"> http.HandleFunc(<span class="string">"/health"</span>, HandleHealth)</span><br><span class="line"></span><br><span class="line"> fmt.Println(<span class="string">"listen on :9000"</span>)</span><br><span class="line"> http.ListenAndServe(<span class="string">":9000"</span>, <span class="literal">nil</span>)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>然后编辑一个配置文件 <code>/etc/consul.d/web.json</code>：</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">{</span><br><span class="line"> <span class="attr">"service"</span>:</span><br><span class="line"> {</span><br><span class="line"> <span class="attr">"name"</span>: <span class="string">"web"</span>,</span><br><span class="line"> <span class="attr">"tags"</span>: [<span class="string">"primary"</span>],</span><br><span class="line"> <span class="attr">"address"</span>: <span class="string">"10.174.96.52"</span>,</span><br><span class="line"> <span class="attr">"port"</span>: <span class="number">9000</span>,</span><br><span class="line"> <span class="attr">"checks"</span>: [</span><br><span class="line"> {</span><br><span class="line"> <span class="attr">"http"</span>: <span class="string">"http://localhost:9000/health"</span>,</span><br><span class="line"> <span class="attr">"interval"</span>: <span class="string">"10s"</span></span><br><span class="line"> }]</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><img src="/2018/07/13/Consul安装部署/ui_3.png" alt=""></p><ul><li><code>-config-dir</code>：配置文件目录，里面所有以.json结尾的文件都会被加载</li></ul><h3 id="配置共享"><a href="#配置共享" class="headerlink" title="配置共享"></a>配置共享</h3><p>由与有了 agent client 和 server 模式的提供，配置共享也变得异常的简单。</p><p>在任意节点更新配置数据：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ consul kv put redis/config 192.168.99.133</span><br><span class="line">Success! Data written to: redis/config</span><br></pre></td></tr></table></figure><p>整个集群均会自动更新，在 s1 节点查看数据：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ consul kv get redis/config</span><br><span class="line">192.168.99.133</span><br></pre></td></tr></table></figure><h2 id="断开连接"><a href="#断开连接" class="headerlink" title="断开连接"></a>断开连接</h2><p>你可以使用<code>Ctrl-C</code> 优雅的关闭Agent. 中断Agent之后你可以看到他离开了集群并关闭.</p><p>在退出中,Consul提醒其他集群成员,这个节点离开了.如果你强行杀掉进程.集群的其他成员应该能检测到这个节点失效了.当一个成员离开,他的服务和检测也会从目录中移除.当一个成员失效了,他的健康状况被简单的标记为危险,但是不会从目录中移除.Consul会自动尝试对失效的节点进行重连.允许他从某些网络条件下恢复过来.离开的节点则不会再继续联系.</p><p>此外,如果一个agent作为一个服务器,一个优雅的离开是很重要的,可以避免引起潜在的可用性故障影响达成<a href="https://www.consul.io/docs/internals/consensus.html" target="_blank" rel="noopener">一致性协议</a>.</p><p>查看<a href="https://www.consul.io/docs/internals/consensus.html" target="_blank" rel="noopener">这里</a>了解添加和移除server.</p><h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p>一个consul agent就是一个独立的程序。一个长时间运行的守护进程，运行在concul集群中的每个节点上。</p><p>启动一个consul agent ，只是启动一个孤立的node，如果想知道集群中的其他节点，应该将consul agent加入到集群中去 cluster。</p><p><strong>agent有两种模式：server与client。</strong></p><ul><li>server模式包含了一致性的工作：保证一致性和可用性（在部分失败的情况下），响应RPC，同步数据到其他节点代理。</li><li>client 模式用于与server进行通信，转发RPC到服务的代理agent，它仅保存自身的少量一些状态，是非常轻量化的东西。本身是相对无状态的。</li></ul><p>agent除去设置server/client模式、数据路径之外，还最好设置node的名称和ip。</p><p><strong>一张经典的consul架构图片：</strong></p><p><img src="/2018/07/13/Consul安装部署/consul架构图.png" alt=""></p><ul><li>LAN gossip pool包含了同一局域网内所有节点，包括server与client。这基本上是位于同一个数据中心DC。</li><li>WAN gossip pool一般仅包含server，将跨越多个DC数据中心，通过互联网或广域网进行通信。</li><li>Leader服务器负责所有的RPC请求，查询并相应。所以其他服务器收到client的RPC请求时，会转发到leader服务器。</li></ul><h1 id="参考："><a href="#参考：" class="headerlink" title="参考："></a>参考：</h1><p><a href="https://deepzz.com/post/the-consul-of-discovery-and-configure-services.html" target="_blank" rel="noopener">consul 支持多数据中心的服务发现与配置共享工具</a></p><p><a href="https://blog.csdn.net/viewcode/article/details/45915179" target="_blank" rel="noopener">consul入门</a></p><p><a href="http://www.liangxiansen.cn/2017/04/06/consul" target="_blank" rel="noopener">Consul使用手册</a></p>]]></content>
<categories>
<category> 分布式 </category>
</categories>
<tags>
<tag> Consul </tag>
</tags>
</entry>
<entry>
<title>微服务初探</title>
<link href="/2018/07/12/%E5%BE%AE%E6%9C%8D%E5%8A%A1%E5%88%9D%E6%8E%A2/"/>
<url>/2018/07/12/%E5%BE%AE%E6%9C%8D%E5%8A%A1%E5%88%9D%E6%8E%A2/</url>
<content type="html"><![CDATA[<h1 id="初识为服务"><a href="#初识为服务" class="headerlink" title="初识为服务"></a>初识为服务</h1><h2 id="什么是微服务"><a href="#什么是微服务" class="headerlink" title="什么是微服务"></a>什么是微服务</h2><p>使用一套小服务来开发单个应用的方式，每个服务运行在<strong>独立的进程</strong>里。一般采用<strong>轻量级的通讯</strong>机制互联，并且他们可以通过<strong>自动化</strong>的方式部署。</p><h2 id="微服务特征"><a href="#微服务特征" class="headerlink" title="微服务特征"></a>微服务特征</h2><p><strong>单一职责</strong>(只把紧密相关的业务放在一起，无关的业务独立出来，比如订单和支付作为一个服务，登录注册作为一个服务，和其他服务不紧密的比如邮件服务，短信服务可以作为一个服务)</p><p><strong>轻量级的通信</strong>：微服务之间的访问和通信 (平台无关和语言无关，http就是轻量级的通信协议)</p><p><strong>隔离线</strong>：每个微服务运行在自己的进程中</p><p><strong>有自己的数据</strong>：微服务倾向都有自己的数据存储系统，这样可以降低数据结构的复杂度</p><p><strong>技术多样性</strong>：微服务可以由开发人员选择最适合的技术，只要提供应有的API就可以了</p><h2 id="微服务诞生背景"><a href="#微服务诞生背景" class="headerlink" title="微服务诞生背景"></a>微服务诞生背景</h2><ol><li>互联网行业的快速发展</li><li>敏捷开发，精益方法深入人心(说白了就是频繁的修改测试上线）</li><li>容器技术的成熟</li></ol><h2 id="微服务的优势"><a href="#微服务的优势" class="headerlink" title="微服务的优势"></a>微服务的优势</h2><p><strong>独立性</strong>:微服务从构建，部署，扩容，缩容，容错，数据库都是单独管理的，所以每个服务之间都是相互独立的</p><p><strong>敏捷性</strong>：对使用者来说，微服务暴露的接口相对简单，因为功能单一，并且有清晰的api,当有新需求时，也可以快速定位到是在哪个微服务中开发</p><p><strong>技术栈灵活</strong>：理论上每个微服务都可以有自己独立的技术栈，不会受到其他微服务的影响</p><p><strong>高效团队</strong>：微服务开发的人员不多，可以几个人开个小会就把需求定下来</p><h2 id="微服务的不足"><a href="#微服务的不足" class="headerlink" title="微服务的不足"></a>微服务的不足</h2><p><strong>额外的工作</strong>：服务的拆分，要把我们的服务拆解成微服务。tdd领域驱动设计</p><p><strong>数据一致性</strong>：单体架构只有一个数据库，可以使用事务来实现多表的级联的修改和删除很容易达到数据的一致性，微服务都有自己的数据库，拆分微服务的时候要尽量保证对数据库的连表操作，尽量在同一个微服务里面。但是很难保证意外的情况</p><p><strong>沟通成本</strong>：微服务的api的改变带来的沟通成本，因为可能需要改变的地方并不仅仅只是自己的服务，还涉及到其他服务，则这个时候推动项目的前进的沟通成本很高</p><h1 id="微服务架构引入的问题和解决方案"><a href="#微服务架构引入的问题和解决方案" class="headerlink" title="微服务架构引入的问题和解决方案"></a>微服务架构引入的问题和解决方案</h1><h2 id="微服务间如何通信？"><a href="#微服务间如何通信？" class="headerlink" title="微服务间如何通信？"></a>微服务间如何通信？</h2><h3 id="从通讯模式角度考虑"><a href="#从通讯模式角度考虑" class="headerlink" title="从通讯模式角度考虑"></a>从通讯模式角度考虑</h3><ul><li>一对一还是一对多？<br>同步还是异步？</li></ul><table><thead><tr><th></th><th>一对一</th><th>一对多</th></tr></thead><tbody><tr><td>同步</td><td>请求响应模式，最常见</td><td>———–（没有这样的场景）</td></tr><tr><td>异步</td><td>通知(不需要响应)/请求异步响应(不需要立即响应)</td><td>发布订阅/发布异步响应</td></tr></tbody></table><h3 id="从通讯协议角度考虑"><a href="#从通讯协议角度考虑" class="headerlink" title="从通讯协议角度考虑"></a>从通讯协议角度考虑</h3><p> REST API </p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">RPC : dubbo ,grpc, thrift, motan</span><br></pre></td></tr></table></figure><p> MQ : 消息队列，发布订阅的模式 </p><h3 id="如何选择RPC框架"><a href="#如何选择RPC框架" class="headerlink" title="如何选择RPC框架"></a>如何选择RPC框架</h3><p>I/O,线程调度模型:</p><p> 是同步的IO还是非阻塞的异步IO。是长连接还是短连接。 )</p><p> 是单线程还是多线程，线程的调度算法是怎么样的</p><p>序列化的方式——->通信的效率</p><p> 可读的</p><p> 二进制</p><p>多语言支持：是否都是一个语言开发</p><p>服务治理：服务发现和服务的监控</p><h2 id="服务发现、部署、更新"><a href="#服务发现、部署、更新" class="headerlink" title="服务发现、部署、更新"></a>服务发现、部署、更新</h2><p>不管微服务还是传统服务，绝大多数对外提供访问的方式是ip+port的方式.</p><p><strong>传统服务的 “服务发现”</strong></p><p>client—>dns服务器—>nginx轮训对应的ip和端口—->服务器</p><p>这也不算是服务发现，每次新增服务的时候，需要重新配置。</p><p><strong>微服务的服务发现</strong></p><ol><li>客户端的发现</li></ol><p><img src="/2018/07/12/微服务初探/客户端服务发现.png" alt=""></p><p>最下面有一个注册中心，当微服务启动之后，都会把自己所暴露的ip和port告诉给注册中心，然后客户端通过查询注册中心所注册的服务来得知微服务提供者的ip和port列表，然后通过本地的一些负载均衡等策略来实现对微服务的无差别访问，如果出现一些调用失败的情况，客户端也有自己的重试的规则。</p><p>dubbo和motan就是这种模式</p><p>服务端的发现</p><p><img src="/2018/07/12/微服务初探/服务端服务发现.png" alt=""></p><p>微服务还是同样的将自己的ip和port注册到注册中心，但是客户端不访问注册中心了，他不需通过注册中心指导微服务的列表，而是通过一个固有的ip去访问一个具有服务发现和负载均衡的服务，再由他将请求转发给后端的具体服务，并且将应答返回给客户端，这个服务在中间起到一个类似代理人的作用，他会从注册中心获取到具体的微服务列表，然后维护到自己的内部，然后当客户端请求一个服务的时候，他会知道客户端应该请求的是这个服务对应的哪些实例在运行，然后通过负载均衡算法去选择一个后端。</p><h2 id="服务编排"><a href="#服务编排" class="headerlink" title="服务编排"></a>服务编排</h2><p>服务编排包括，服务发现，服务更新和服务的扩缩融</p><p>流行的服务编排工具：</p><p>Mesos Docker Swarm kubernetes</p>]]></content>
<categories>
<category> 微服务 </category>
</categories>
<tags>
<tag> 微服务 </tag>
</tags>
</entry>
<entry>
<title>redis集群安装</title>
<link href="/2017/12/29/redis%E9%9B%86%E7%BE%A4%E5%AE%89%E8%A3%85/"/>
<url>/2017/12/29/redis%E9%9B%86%E7%BE%A4%E5%AE%89%E8%A3%85/</url>
<content type="html"><![CDATA[<h1 id="redis-Cluster-集群安装"><a href="#redis-Cluster-集群安装" class="headerlink" title="redis Cluster 集群安装"></a>redis Cluster 集群安装</h1><h2 id="集群部署"><a href="#集群部署" class="headerlink" title="集群部署"></a>集群部署</h2><p>测试部署方式，一台测试机多实例启动部署。</p><p>安装redis</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ wget http://download.redis.io/releases/redis-3.2.8.tar.gz</span><br><span class="line">$ tar xzf redis-3.2.8.tar.gz</span><br><span class="line">$ cd redis-3.2.8</span><br><span class="line">$ yum groupinstall -y "Development Tools"</span><br><span class="line">$ make && make install</span><br></pre></td></tr></table></figure><p>修改配置文件 redis.conf</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#redis.conf默认配置</span></span><br><span class="line"><span class="string">daemonize</span> <span class="literal">yes</span> <span class="comment">#后台运行开启</span></span><br><span class="line"><span class="string">pidfile</span> <span class="string">/var/run/redis/redis.pid</span> <span class="comment">#多实例情况下需修改，指定pid文件的路径 通过绝对路径指明文件存放的位置 自行创建相关的文件目录 例如redis_6380.pid</span></span><br><span class="line"><span class="string">port</span> <span class="number">6379</span> <span class="comment">#多实例情况下需要修改,修改端口号 例如6380</span></span><br><span class="line"><span class="string">tcp-backlog</span> <span class="number">511</span></span><br><span class="line"><span class="string">bind</span> <span class="number">0.0</span><span class="number">.0</span><span class="number">.0</span> </span><br><span class="line"><span class="string">timeout</span> <span class="number">0</span></span><br><span class="line"><span class="string">tcp-keepalive</span> <span class="number">0</span></span><br><span class="line"><span class="string">loglevel</span> <span class="string">notice</span></span><br><span class="line"><span class="string">logfile</span> <span class="string">/var/log/redis/redis.log</span> <span class="comment">#多实例情况下需要修改，例如6380.log</span></span><br><span class="line"><span class="string">databases</span> <span class="number">16</span></span><br><span class="line"><span class="string">save</span> <span class="number">900</span> <span class="number">1</span></span><br><span class="line"><span class="string">save</span> <span class="number">300</span> <span class="number">10</span></span><br><span class="line"><span class="string">save</span> <span class="number">60</span> <span class="number">10000</span></span><br><span class="line"><span class="string">stop-writes-on-bgsave-error</span> <span class="literal">yes</span></span><br><span class="line"><span class="string">rdbcompression</span> <span class="literal">yes</span></span><br><span class="line"><span class="string">rdbchecksum</span> <span class="literal">yes</span></span><br><span class="line"><span class="string">dbfilename</span> <span class="string">dump.rdb</span> <span class="comment">#多实例情况下需要修改，修改dump日志文件路径 果不修改dump文件那么每次的日志文件都是公用的 例如dump.6380.rdb</span></span><br><span class="line"><span class="string">slave-serve-stale-data</span> <span class="literal">yes</span></span><br><span class="line"><span class="string">slave-read-only</span> <span class="literal">yes</span></span><br><span class="line"><span class="string">repl-diskless-sync</span> <span class="literal">no</span></span><br><span class="line"><span class="string">repl-diskless-sync-delay</span> <span class="number">5</span></span><br><span class="line"><span class="string">repl-disable-tcp-nodelay</span> <span class="literal">no</span></span><br><span class="line"><span class="string">slave-priority</span> <span class="number">100</span></span><br><span class="line"><span class="string">appendonly</span> <span class="literal">yes</span> <span class="comment">#启用二进制日志</span></span><br><span class="line"><span class="string">appendfilename</span> <span class="string">"appendonly.aof"</span> <span class="comment">#多实例情况下需要修改,例如 appendonly_6380.aof</span></span><br><span class="line"><span class="string">appendfsync</span> <span class="string">everysec</span></span><br><span class="line"><span class="literal">no</span><span class="bullet">-appendfsync-on-rewrite</span> <span class="literal">no</span></span><br><span class="line"><span class="string">auto-aof-rewrite-percentage</span> <span class="number">100</span></span><br><span class="line"><span class="string">auto-aof-rewrite-min-size</span> <span class="number">64</span><span class="string">mb</span></span><br><span class="line"><span class="string">aof-load-truncated</span> <span class="literal">yes</span></span><br><span class="line"><span class="string">lua-time-limit</span> <span class="number">5000</span></span><br><span class="line"><span class="string">slowlog-log-slower-than</span> <span class="number">10000</span></span><br><span class="line"><span class="string">slowlog-max-len</span> <span class="number">128</span></span><br><span class="line"><span class="string">latency-monitor-threshold</span> <span class="number">0</span></span><br><span class="line"><span class="string">notify-keyspace-events</span> <span class="string">""</span></span><br><span class="line"><span class="string">hash-max-ziplist-entries</span> <span class="number">512</span></span><br><span class="line"><span class="string">hash-max-ziplist-value</span> <span class="number">64</span></span><br><span class="line"><span class="string">list-max-ziplist-entries</span> <span class="number">512</span></span><br><span class="line"><span class="string">list-max-ziplist-value</span> <span class="number">64</span></span><br><span class="line"><span class="string">set-max-intset-entries</span> <span class="number">512</span></span><br><span class="line"><span class="string">zset-max-ziplist-entries</span> <span class="number">128</span></span><br><span class="line"><span class="string">zset-max-ziplist-value</span> <span class="number">64</span></span><br><span class="line"><span class="string">hll-sparse-max-bytes</span> <span class="number">3000</span></span><br><span class="line"><span class="string">activerehashing</span> <span class="literal">yes</span></span><br><span class="line"><span class="string">client-output-buffer-limit</span> <span class="string">normal</span> <span class="number">0</span> <span class="number">0</span> <span class="number">0</span></span><br><span class="line"><span class="string">client-output-buffer-limit</span> <span class="string">slave</span> <span class="number">256</span><span class="string">mb</span> <span class="number">64</span><span class="string">mb</span> <span class="number">60</span></span><br><span class="line"><span class="string">client-output-buffer-limit</span> <span class="string">pubsub</span> <span class="number">32</span><span class="string">mb</span> <span class="number">8</span><span class="string">mb</span> <span class="number">60</span></span><br><span class="line"><span class="string">hz</span> <span class="number">10</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#################自定义配置</span></span><br><span class="line"><span class="comment">#系统配置</span></span><br><span class="line"><span class="comment">#vim /etc/sysctl.conf</span></span><br><span class="line"><span class="comment">#vm.overcommit_memory = 1</span></span><br><span class="line"></span><br><span class="line"><span class="string">aof-rewrite-incremental-fsync</span> <span class="literal">yes</span></span><br><span class="line"><span class="string">maxmemory</span> <span class="number">4096</span><span class="string">mb</span></span><br><span class="line"><span class="string">maxmemory-policy</span> <span class="string">allkeys-lru</span></span><br><span class="line"><span class="string">dir</span> <span class="string">/opt/redis/data</span> <span class="comment">#多实例情况下需要修改，例如/data/6380</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#集群配置</span></span><br><span class="line"><span class="string">cluster-enabled</span> <span class="literal">yes</span> <span class="comment">#启用集群</span></span><br><span class="line"><span class="string">cluster-config-file</span> <span class="string">/opt/redis/6380/nodes.conf</span> <span class="comment">#多实例情况下需要修改，例如/6380/</span></span><br><span class="line"><span class="string">cluster-node-timeout</span> <span class="number">5000</span> <span class="comment">#集群超时时间</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">#从ping主间隔默认10秒</span></span><br><span class="line"><span class="comment">#复制超时时间</span></span><br><span class="line"><span class="comment">#repl-timeout 60</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#远距离主从</span></span><br><span class="line"><span class="comment">#config set client-output-buffer-limit "slave 536870912 536870912 0"</span></span><br><span class="line"><span class="comment">#config set repl-backlog-size 209715200</span></span><br></pre></td></tr></table></figure><h3 id="拷贝redis-conf文件到文件夹中"><a href="#拷贝redis-conf文件到文件夹中" class="headerlink" title="拷贝redis.conf文件到文件夹中"></a>拷贝redis.conf文件到文件夹中</h3><p>cp redis.conf 7000/redis-7000.conf</p><p>mkdir 7000 7001 7002 7003 7004 7005</p><h3 id="将配置文件分别拷贝到7001-7008中，需要修改端口号即可"><a href="#将配置文件分别拷贝到7001-7008中，需要修改端口号即可" class="headerlink" title="将配置文件分别拷贝到7001-7008中，需要修改端口号即可."></a>将配置文件分别拷贝到7001-7008中，需要修改端口号即可.</h3><p>在vim下执行以下命令可以先将文件中的全部7000修改为7001</p><p>:%s/7000/7001/g 注：代表将当前文本的所有的7000替换成7001</p><p>分别将7002-7008的配置文件进行修改</p><p>.创建shell脚本文件启动多个redis服务从7000-7008</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">#!/bin/sh</span><br><span class="line">redis-server 7000/redis-7000.conf &</span><br><span class="line">redis-server 7001/redis-7001.conf &</span><br><span class="line">redis-server 7002/redis-7002.conf &</span><br><span class="line">redis-server 7003/redis-7003.conf &</span><br><span class="line">redis-server 7004/redis-7004.conf &</span><br><span class="line">redis-server 7005/redis-7005.conf &</span><br><span class="line">redis-server 7006/redis-7006.conf &</span><br><span class="line">redis-server 7007/redis-7007.conf &</span><br><span class="line">redis-server 7008/redis-7008.conf &</span><br></pre></td></tr></table></figure><h2 id="创建redis-cluster"><a href="#创建redis-cluster" class="headerlink" title="创建redis-cluster"></a>创建redis-cluster</h2><p>通过redis-trib.rb来创建redis集群，</p><p>1、安装的ruby，通过yum 源下载安装的ruby可能版本过低，导致:</p><p>输入命令 “ gem install redis “ 出现 “ ERROR: Error installing redis redis requires Ruby version >= 2.2.2. “</p><p>所以要安装ruby的RVM管理工具获取ruby的最新包</p><p>使用curl安装rvm ，输入命令 “ curl -L get.rvm.io | bash -s stable “ 进行安装，这个时候可能会出现</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">GPG signature verification failed for ‘/usr/local/rvm/archives/rvm-1.29.3.tgz‘ - ‘https://github.com/rvm/rvm/releases/download/1.29.3/1.29.3.tar.gz.asc‘! Try to install GPG v2 and then fetch the public key:</span><br></pre></td></tr></table></figure><p>这时候要输入密钥 <code>gpg2 --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3</code></p><p>然后在执行</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">curl -L get.rvm.io | bash -s stable</span><br><span class="line">source /usr/local/rvm/scripts/rvm</span><br></pre></td></tr></table></figure><p>查看rvm中管理的所有ruby版本，<br>输入命令 <code>rvm list known</code>进行查询，查询出来如下</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br></pre></td><td class="code"><pre><span class="line"># MRI Rubies</span><br><span class="line">[ruby-]1.8.6[-p420]</span><br><span class="line">[ruby-]1.8.7[-head] # security released on head</span><br><span class="line">[ruby-]1.9.1[-p431]</span><br><span class="line">[ruby-]1.9.2[-p330]</span><br><span class="line">[ruby-]1.9.3[-p551]</span><br><span class="line">[ruby-]2.0.0[-p648]</span><br><span class="line">[ruby-]2.1[.10]</span><br><span class="line">[ruby-]2.2[.7]</span><br><span class="line">[ruby-]2.3[.4]</span><br><span class="line">[ruby-]2.4[.1]</span><br><span class="line">ruby-head</span><br><span class="line"></span><br><span class="line"># for forks use: rvm install ruby-head-<name> --url https://github.com/github/ruby.git --branch 2.2</span><br><span class="line"></span><br><span class="line"># JRuby</span><br><span class="line">jruby-1.6[.8]</span><br><span class="line">jruby-1.7[.27]</span><br><span class="line">jruby[-9.1.13.0]</span><br><span class="line">jruby-head</span><br><span class="line"></span><br><span class="line"># Rubinius</span><br><span class="line">rbx-1[.4.3]</span><br><span class="line">rbx-2.3[.0]</span><br><span class="line">rbx-2.4[.1]</span><br><span class="line">rbx-2[.5.8]</span><br><span class="line">rbx-3[.84]</span><br><span class="line">rbx-head</span><br><span class="line"></span><br><span class="line"># Opal</span><br><span class="line">opal</span><br><span class="line"></span><br><span class="line"># Minimalistic ruby implementation - ISO 30170:2012</span><br><span class="line">mruby-1.0.0</span><br><span class="line">mruby-1.1.0</span><br><span class="line">mruby-1.2.0</span><br><span class="line">mruby-1[.3.0]</span><br><span class="line">mruby[-head]</span><br><span class="line"></span><br><span class="line"># Ruby Enterprise Edition</span><br><span class="line">ree-1.8.6</span><br><span class="line">ree[-1.8.7][-2012.02]</span><br><span class="line"></span><br><span class="line"># Topaz</span><br><span class="line">topaz</span><br><span class="line"></span><br><span class="line"># MagLev</span><br><span class="line">maglev[-head]</span><br><span class="line">maglev-1.0.0</span><br><span class="line"></span><br><span class="line"># Mac OS X Snow Leopard Or Newer</span><br><span class="line">macruby-0.10</span><br><span class="line">macruby-0.11</span><br><span class="line">macruby[-0.12]</span><br><span class="line">macruby-nightly</span><br></pre></td></tr></table></figure><p>选择一个你喜欢的版本进行安装，但首先提醒一下，你所选择的版本不能低于 “ 2.0.0 “ 就可以了，输入命令 “ rvm install 2.3.4 “ 进行安装，如下</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">[root@VM_0_12_centos ~]# rvm install 2.3.4</span><br><span class="line">Searching for binary rubies, this might take some time.</span><br><span class="line">Found remote file https://rvm_io.global.ssl.fastly.net/binaries/centos/7/x86_64/ruby-2.3.4.tar.bz2</span><br><span class="line">Checking requirements for centos.</span><br><span class="line">Installing requirements for centos.</span><br><span class="line">Installing required packages: libffi-devel, readline-devel, sqlite-devel, zlib-devel, libyaml-devel, openssl-devel.............</span><br><span class="line">Requirements installation successful.</span><br><span class="line">ruby-2.3.4 - #configure</span><br><span class="line">ruby-2.3.4 - #download</span><br><span class="line"> % Total % Received % Xferd Average Speed Time Time Time Current</span><br><span class="line"> Dload Upload Total Spent Left Speed</span><br><span class="line">100 25.2M 100 25.2M 0 0 461k 0 0:00:55 0:00:55 --:--:-- 225k</span><br><span class="line">No checksum for downloaded archive, recording checksum in user configuration.</span><br><span class="line">ruby-2.3.4 - #validate archive</span><br><span class="line">ruby-2.3.4 - #extract</span><br><span class="line">ruby-2.3.4 - #validate binary</span><br><span class="line">ruby-2.3.4 - #setup</span><br><span class="line">ruby-2.3.4 - #gemset created /usr/local/rvm/gems/ruby-2.3.4@global</span><br><span class="line">ruby-2.3.4 - #importing gemset /usr/local/rvm/gemsets/global.gems..............................</span><br><span class="line">ruby-2.3.4 - #generating global wrappers........</span><br><span class="line">ruby-2.3.4 - #gemset created /usr/local/rvm/gems/ruby-2.3.4</span><br><span class="line">ruby-2.3.4 - #importing gemsetfile /usr/local/rvm/gemsets/default.gems evaluated to empty gem list</span><br><span class="line">ruby-2.3.4 - #generating default wrappers........</span><br></pre></td></tr></table></figure><p>redis-trib.rb命令与redis-cli命令放置在同一个目录中，可全路径执行或者创建别名。</p><p>redis-trib.rb create –replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:70005</p><p>–replicas 1 表示一主一从，</p><h4 id="脚本使用帮助"><a href="#脚本使用帮助" class="headerlink" title="脚本使用帮助"></a>脚本使用帮助</h4><ul><li>查看脚本帮助</li></ul><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br></pre></td><td class="code"><pre><span class="line">$ ruby redis-trib.rb help</span><br><span class="line">Usage: redis-trib <command> <options> <arguments ...></span><br><span class="line"></span><br><span class="line"> create host1:port1 ... hostN:portN</span><br><span class="line"> --replicas <arg></span><br><span class="line"> check host:port</span><br><span class="line"> info host:port</span><br><span class="line"> fix host:port</span><br><span class="line"> --timeout <arg></span><br><span class="line"> reshard host:port</span><br><span class="line"> --from <arg></span><br><span class="line"> --to <arg></span><br><span class="line"> --slots <arg></span><br><span class="line"> --yes</span><br><span class="line"> --timeout <arg></span><br><span class="line"> --pipeline <arg></span><br><span class="line"> rebalance host:port</span><br><span class="line"> --weight <arg></span><br><span class="line"> --auto-weights</span><br><span class="line"> --use-empty-masters</span><br><span class="line"> --timeout <arg></span><br><span class="line"> --simulate</span><br><span class="line"> --pipeline <arg></span><br><span class="line"> --threshold <arg></span><br><span class="line"> add-node new_host:new_port existing_host:existing_port</span><br><span class="line"> --slave</span><br><span class="line"> --master-id <arg></span><br><span class="line"> del-node host:port node_id</span><br><span class="line"> set-timeout host:port milliseconds</span><br><span class="line"> call host:port command arg arg .. arg</span><br><span class="line"> import host:port</span><br><span class="line"> --from <arg></span><br><span class="line"> --copy</span><br><span class="line"> --replace</span><br><span class="line"> help (show this help)</span><br><span class="line"></span><br><span class="line">For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.</span><br></pre></td></tr></table></figure><ul><li>各选项详解</li></ul><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">1、create：创建集群</span><br><span class="line">2、check：检查集群</span><br><span class="line">3、info：查看集群信息</span><br><span class="line">4、fix：修复集群</span><br><span class="line">5、reshard：在线迁移slot</span><br><span class="line">6、rebalance：平衡集群节点slot数量</span><br><span class="line">7、add-node：将新节点加入集群</span><br><span class="line">8、del-node：从集群中删除节点</span><br><span class="line">9、set-timeout：设置集群节点间心跳连接的超时时间</span><br><span class="line">10、call：在集群全部节点上执行命令</span><br><span class="line">11、import：将外部redis数据导入集群</span><br></pre></td></tr></table></figure>]]></content>
<categories>
<category> redis </category>
</categories>
<tags>
<tag> redis </tag>
</tags>
</entry>
<entry>
<title>codis安装及配置</title>
<link href="/2017/12/12/codis%E5%AE%89%E8%A3%85%E5%8F%8A%E9%85%8D%E7%BD%AE/"/>
<url>/2017/12/12/codis%E5%AE%89%E8%A3%85%E5%8F%8A%E9%85%8D%E7%BD%AE/</url>
<content type="html"><![CDATA[<h1 id="codis安装及配置"><a href="#codis安装及配置" class="headerlink" title="codis安装及配置"></a>codis安装及配置</h1><h2 id="一、环境安装"><a href="#一、环境安装" class="headerlink" title="一、环境安装"></a>一、环境安装</h2><p><strong>系统环境centos</strong></p><ol><li><p>Golang 环境搭建: yum install go</p></li><li><p>codis 下载和编译:</p><p>go get -u -d github.com/CodisLabs/codis<br>cd $GOPATH/src/github.com/CodisLabs/codis<br>make<br>make gotest<br>mkdir -p /usr/local/codis/{logs,conf,scripts}<br>cp –rf bin /usr/local/codis/<br>cp config.ini /usr/local/codis/conf/</p></li></ol><ol start="3"><li><p>Codis-HA 编译<br>Codis-HA。这是一个通过 Codis 开放的 api 实现自动切换主从的工具。该工具会在检测<br>到 master 挂掉的时候将其下线并选择其中一个 slave 提升为 master 继续提供服务。<br>go get github.com/ngaut/codis-ha<br>cd codis-ha<br>go build<br>codis-ha –codis-config=localhost:18087 –productName=test</p></li><li><p>Zookeeper集群搭建</p><p>首先安装开发工具及openjdk，zookeeper是由Java语言开发的，所以需要openjdk环境。</p></li></ol><p>首先安装开发工具及openjdk，zookeeper是由Java语言开发的，所以需要openjdk环境。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">yum groupinstall "Development tools" "Compatibility libraries" -y </span><br><span class="line">yum install openssl-devel openssl -y</span><br><span class="line">yum install java-1.8.0-openjdk-devel java-1.8.0-openjdk -y</span><br></pre></td></tr></table></figure><p>确定Java运行环境正常</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">java -version</span><br><span class="line"> openjdk version "1.8.0101"</span><br><span class="line"> OpenJDK Runtime Environment (build 1.8.0101-b13)</span><br><span class="line"> OpenJDK 64-Bit Server VM (build 25.101-b13, mixed mode)</span><br></pre></td></tr></table></figure><p>安装二进制版本的zookeeper</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">tar xvf zookeeper-3.4.9.tar.gz -C /usr/local/ </span><br><span class="line">ln -s /usr/local/zookeeper-3.4.9/ /usr/local/zookeeper </span><br><span class="line">cd /usr/local/zookeeper/conf </span><br><span class="line">cp zoo_sample.cfg zoo.cfg</span><br></pre></td></tr></table></figure><p>编译zookeeper配置文件/usr/local/zookeeper/conf/zoo.cfg</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">maxClientCnxns=60</span><br><span class="line">tickTime=2000</span><br><span class="line">initLimit=10</span><br><span class="line">syncLimit=5</span><br><span class="line">dataDir=/data/zookeeper/db</span><br><span class="line">dataLogDir=/data/zookeeper/log</span><br><span class="line">clientPort=2181</span><br><span class="line"># cluster configure</span><br><span class="line">server.1=10.173.225.60:2888:3888</span><br><span class="line">server.2=10.174.33.81:2888:3888</span><br><span class="line">server.3=10.173.224.34:2888:3888</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">mkdir /data/zookeeper/{db,log} -p</span><br></pre></td></tr></table></figure><p>其中2888表示zookeeper程序监听端口，3888表示zookeeper选举通信端口。</p><p>下面需要生成ID，这里需要注意，myid对应的zoo.cfg的server.ID，比如第二台zookeeper主机对应的myid应该是2，以此类推，三个主机分别为：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">10.173.225.60 echo 1 > /data/zookeeper/db/myid</span><br><span class="line">10.174.33.81 echo 2 > /data/zookeeper/db/myid</span><br><span class="line">10.173.224.34 echo 3 > /data/zookeeper/db/myid</span><br></pre></td></tr></table></figure><p>然后输出环境变量。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">export PATH=$PATH:/usr/local/zookeeper/bin/</span><br><span class="line">source /etc/profile</span><br></pre></td></tr></table></figure><p>然后就可以启动zookeeper了。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">zkServer.sh start</span><br></pre></td></tr></table></figure><p>查看各个zookeeper节点的状态（会有一个leader节点，两个follower节点）。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root@node1 ~]# zkServer.sh status </span><br><span class="line">Mode: follower</span><br><span class="line">[root@node2 ~]# zkServer.sh status </span><br><span class="line">Mode: leader</span><br><span class="line">[root@node3 ~]# zkServer.sh status </span><br><span class="line">Mode: follower</span><br></pre></td></tr></table></figure><p>客户端连接，可以查看相关信息。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">zkCli.sh -server 127.0.0.1:2181</span><br></pre></td></tr></table></figure><p>至此，zookeeper已经搞定了。</p><h2 id="二、启动服务"><a href="#二、启动服务" class="headerlink" title="二、启动服务"></a>二、启动服务</h2><p>再编译后的codis文件夹下有启动服务的脚本</p><h3 id="启动codis-dashboard"><a href="#启动codis-dashboard" class="headerlink" title="启动codis-dashboard"></a>启动codis-dashboard</h3><p>使用 <code>codis-dashboard-admin.sh</code> 脚本启动 dashboard，并查看 dashboard 日志确认启动是否有异常。</p><p>dashboard只需要启动一个</p><p>配置文件dashboard.toml</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"># Set Coordinator, only accept "zookeeper" & "etcd" & "filesystem".</span><br><span class="line"># for zookeeper/etcd, coorinator_auth accept "user:password"</span><br><span class="line"># Quick Start</span><br><span class="line">coordinator_name = "zookeeper"</span><br><span class="line">coordinator_addr = "10.173.225.60:2181,10.174.33.81:2181,10.173.224.34:2181"</span><br><span class="line">#coordinator_name = "zookeeper"</span><br><span class="line">#coordinator_addr = "127.0.0.1:2181"</span><br><span class="line">#coordinator_auth = ""</span><br><span class="line"></span><br><span class="line"># Set Codis Product Name/Auth.</span><br><span class="line">product_name = "codis-demo"</span><br><span class="line">product_auth = ""</span><br><span class="line"></span><br><span class="line"># Set bind address for admin(rpc), tcp only.</span><br><span class="line">admin_addr = "0.0.0.0:18080"</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">./admin/codis-dashboard-admin.sh start</span><br><span class="line"> tail -100 ./log/codis-dashboard.log.2017-04-08</span><br></pre></td></tr></table></figure><h3 id="启动codis-fe"><a href="#启动codis-fe" class="headerlink" title="启动codis-fe"></a>启动codis-fe</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./bin/codis-admin --dashboard-list --zookeeper=127.0.0.1:2181 | tee $Gopath/src/github.com/CodisLabs/codis/config/codis.json</span><br></pre></td></tr></table></figure><p>codis.json</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[</span><br><span class="line">{</span><br><span class="line"> "name": "codis-demo",</span><br><span class="line"> "dashboard": "10.174.33.81:18080"</span><br><span class="line"> }</span><br><span class="line"> ]</span><br></pre></td></tr></table></figure><p>启动codis-fe</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">nohup `which codis-fe` --ncpu=2 --log=/data/codis/log/fe.log --log-level=WARN </span><br><span class="line">--dashboard-list=$Gopath/src/github.com/CodisLabs/codis/config/codis.json --listen=0.0.0.0:8080 &</span><br></pre></td></tr></table></figure><h3 id="启动codis-proxy"><a href="#启动codis-proxy" class="headerlink" title="启动codis-proxy"></a>启动codis-proxy</h3><p>使用 <code>codis-proxy-admin.sh</code> 脚本启动 codis-proxy，并查看 proxy 日志确认启动是否有异常。(代理可以启动一个，也可以启动多个，但是启动的多个代理的配置必须是一样的，是同一个dashboard的地址)</p><p>配置文件proxy.toml</p><blockquote><p>19 # Set bind address for admin(rpc), tcp only.</p><p>20 admin_addr = “10.173.225.60:11080”<br>21<br>22 # Set bind address for proxy, proto_type can be “tcp”, “tcp4”, “tcp6”, “unix” or “unixpacket”.<br>23 proto_type = “tcp4”<br>24 proxy_addr = “10.173.225.60:19000”<br>26 # Set jodis address & session timeout<br>27 # 1. jodis_name is short for jodis_coordinator_name, only accept “zookeeper” & “etcd”.<br>28 # 2. jodis_addr is short for jodis_coordinator_addr<br>29 # 3. jodis_auth is short for jodis_coordinator_auth, for zookeeper/etcd, “user:password” is accepted.<br>30 # 4. proxy will be registered as node:<br>31 # if jodis<em>compatible = true (not suggested):32 # /zk/codis/db</em>{PRODUCT_NAME}/proxy-{HASHID} (compatible with Codis2.0)<br>33 # or else<br>34 # /jodis/{PRODUCT_NAME}/proxy-{HASHID}<br>35 jodis_name = “”<br>3 # jodis_addr不能写地址，不然启动报错，不知道为啥，可能没有安装jodis,<br>36 jodis_addr = “”<br>37 jodis_auth = “”<br>38 jodis_timeout = “20s”</p></blockquote><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">./admin/codis-proxy-admin.sh start</span><br><span class="line">tail -100 ./log/codis-proxy.log.2017-04-08</span><br></pre></td></tr></table></figure><p>要启动多个代理需要修改脚本</p><p>vi codis-proxy-admin.sh</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">CODIS_DASHBOARD_ADDR="10.173.225.60:18080"</span><br></pre></td></tr></table></figure><h3 id="启动codis-server"><a href="#启动codis-server" class="headerlink" title="启动codis-server"></a>启动codis-server</h3><p>使用 <code>codis-server-admin.sh</code> 脚本启动 codis-server，并查看 redis 日志确认启动是否有异常。</p><p>配置redis.conf 和sentinel.conf</p><p><strong>redis.conf</strong></p><blockquote><p>48 # bind 127.0.0.1 ::1<br>49 #<br>50 # WARNING If the computer running Redis is directly exposed to the<br>51 # internet, binding to all the interfaces is dangerous and will expose the<br>52 # instance to everybody on the internet. So by default we uncomment the<br>53 # following bind directive, that will force Redis to listen only into<br>54 # the IPv4 lookback interface address (this means Redis will be able to<br>55 # accept connections only from clients running into the same computer it<br>56 # is running).<br>57 #<br>58 # IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES<br>59 # JUST COMMENT THE FOLLOWING LINE.<br>60 # ~~<br>61 bind 0.0.0.0<br>…….<br>…….<br>82 # Accept connections on the specified port, default is 6379 (IANA #815344).<br>83 # If port 0 is specified Redis will not listen on a TCP socket.<br>84 port 6379</p></blockquote><p><strong>sentinel.conf</strong></p><blockquote><p>48 # bind 127.0.0.1 ::1<br>49 #<br>50 # WARNING If the computer running Redis is directly exposed to the<br>51 # internet, binding to all the interfaces is dangerous and will expose the<br>52 # instance to everybody on the internet. So by default we uncomment the<br>53 # following bind directive, that will force Redis to listen only into<br>54 # the IPv4 lookback interface address (this means Redis will be able to<br>55 # accept connections only from clients running into the same computer it<br>56 # is running).<br>57 #<br>58 # IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES<br>59 # JUST COMMENT THE FOLLOWING LINE.<br>60 #~~<br>61 bind 0.0.0.0<br>…….<br>…….<br>82 # Accept connections on the specified port, default is 6379 (IANA #815344).<br>83 # If port 0 is specified Redis will not listen on a TCP socket.<br>84 port 26379</p></blockquote><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">./admin/codis-server-admin.sh start</span><br><span class="line">tail -100 /tmp/redis_6379.log</span><br></pre></td></tr></table></figure><p>redis.conf 配置中 pidfile、logfile 默认保存在 <code>/tmp</code> 目录，若启动失败，请检查当前用户是否有该目录的读写权限。</p><h3 id="通过fe添加group"><a href="#通过fe添加group" class="headerlink" title="通过fe添加group"></a>通过fe添加group</h3><p>通过web浏览器访问集群管理页面(fe地址:127.0.0.1:8080) 选择我们刚搭建的集群 codis-demo，在 Proxy 栏可看到我们已经启动的 Proxy， 但是 Group 栏为空，因为我们启动的 codis-server 并未加入到集群 添加 <code>NEW GROUP</code>，<code>NEW GROUP</code> 行输入 1，再点击 <code>NEW GROUP</code> 即可 添加 Codis Server，<code>Add Server</code> 行输入我们刚刚启动的 codis-server 地址，添加到我们刚新建的 Group，然后再点击 <code>Add Server</code> 按钮即可，如下图所示<br><img src="/2017/12/12/codis安装及配置/addgroup.jpg" alt=""></p><h3 id="通过fe初始化slot"><a href="#通过fe初始化slot" class="headerlink" title="通过fe初始化slot"></a>通过fe初始化slot</h3><p>新增的集群 slot 状态是 offline，因此我们需要对它进行初始化（将 1024 个 slot 分配到各个 group），而初始化最快的方法可通过 fe 提供的 <code>rebalance all slots</code> 按钮来做，如下图所示，点击此按钮，我们即快速完成了一个集群的搭建。<br><img src="/2017/12/12/codis安装及配置/rebalance_slots.jpg" alt=""><br>每次增加组之后就需要重新执行<strong>Rebalance All Slots</strong></p>]]></content>
<categories>
<category> redis </category>
</categories>
<tags>
<tag> codis </tag>
</tags>
</entry>
<entry>
<title>Go性能分析工具</title>
<link href="/2017/08/23/Go%E6%80%A7%E8%83%BD%E5%88%86%E6%9E%90%E5%B7%A5%E5%85%B7/"/>
<url>/2017/08/23/Go%E6%80%A7%E8%83%BD%E5%88%86%E6%9E%90%E5%B7%A5%E5%85%B7/</url>
<content type="html"><![CDATA[<h1 id="为什么需要性能分析"><a href="#为什么需要性能分析" class="headerlink" title="为什么需要性能分析"></a>为什么需要性能分析</h1><p>作为开发，一般在开发过程大多是为了功能的实现和单元测试，当业务量不大的时候，也不会说去过早的分析代码的性能。但是一旦业务量上来，原有开发中代码写的不太好的地方会造成性能瓶颈，这个时候就得需要性能分析工具帮助分析程序在哪块代码上出现了性能瓶颈。才可以有针对性的优化代码。</p><p>很幸运的，go语言官方有提供的性能工具pprof，我们可以很方便的分析程序运行过程中造成瓶颈的地方</p><h1 id="pprof"><a href="#pprof" class="headerlink" title="pprof"></a>pprof</h1><h2 id="什么是pprof"><a href="#什么是pprof" class="headerlink" title="什么是pprof"></a>什么是pprof</h2><p>pprof是Go语言内置的标准方法用来调试Go程序性能。golang官方有提供两种pprof启动的方式，分别是 <a href="https://golang.org/pkg/runtime/pprof/" target="_blank" rel="noopener">runtime/pprof</a>，<a href="https://golang.org/pkg/net/http/pprof/" target="_blank" rel="noopener">net/http/pprof</a> ，它能提取出来应用程序的CPU和内存数据，此外还有运行的代码行数和内容信息。</p><h2 id="pprof文件生成"><a href="#pprof文件生成" class="headerlink" title="pprof文件生成"></a>pprof文件生成</h2><h3 id="runtime-pprof"><a href="#runtime-pprof" class="headerlink" title="runtime/pprof"></a>runtime/pprof</h3><p>此包是方便不是提供web服务的后端程序来进行分析性能</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line">f, err := os.Create(fmt.Sprintf(<span class="string">"cpu-%s.pprof"</span>, time.Now().Format(<span class="string">"20060102"</span>)))</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> {</span><br><span class="line">log.Fatal(<span class="string">"create CPU profile error: "</span>, err)</span><br><span class="line">}</span><br><span class="line"><span class="keyword">if</span> err := pprof.StartCPUProfile(f); err != <span class="literal">nil</span> {</span><br><span class="line">log.Fatal(<span class="string">"start CPU profile error: "</span>, err)</span><br><span class="line">}</span><br><span class="line"><span class="keyword">defer</span> pprof.StopCPUProfile()</span><br><span class="line"></span><br><span class="line"><span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line">doSth()</span><br><span class="line">}()</span><br><span class="line"><span class="comment">//优雅退出</span></span><br><span class="line">sigChan := <span class="built_in">make</span>(<span class="keyword">chan</span> os.Signal)</span><br><span class="line">exitChan := <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">struct</span>{})</span><br><span class="line">signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)</span><br><span class="line">log.Printf(<span class="string">"signal received"</span>, <-sigChan)</span><br><span class="line"></span><br><span class="line"><span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line"><span class="keyword">if</span> err := server.Stop(); err != <span class="literal">nil</span> {</span><br><span class="line">log.Fatal(err)</span><br><span class="line">}</span><br><span class="line">exitChan <- <span class="keyword">struct</span>{}{}</span><br><span class="line">}()</span><br><span class="line"><span class="keyword">select</span> {</span><br><span class="line"><span class="keyword">case</span> <-exitChan:</span><br><span class="line"><span class="keyword">case</span> s := <-sigChan:</span><br><span class="line">log.Panicln(<span class="string">"signal received, stopping immediately"</span>, s)</span><br><span class="line">}</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">doSth</span><span class="params">()</span></span> {</span><br><span class="line"><span class="keyword">for</span> {</span><br><span class="line">rand.Float32()</span><br><span class="line">time.Sleep(<span class="number">500</span> * time.Millisecond)</span><br><span class="line">}</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>此时运行程序会生成关于cpu统计的文件cpu-20170823.pprof</p><p>然后通过命令执行</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">~ <span class="keyword">go</span> tool pprof .\cpu<span class="number">-20170823.</span>pprof</span><br><span class="line">Type: cpu</span><br><span class="line">Time: Jul <span class="number">19</span>, <span class="number">2018</span> at <span class="number">5</span>:<span class="number">31</span>pm (CST)</span><br><span class="line">Duration: <span class="number">58.32s</span>, Total samples = <span class="number">0</span></span><br><span class="line">Entering interactive mode (<span class="keyword">type</span> <span class="string">"help"</span> <span class="keyword">for</span> commands, <span class="string">"o"</span> <span class="keyword">for</span> options)</span><br><span class="line">(pprof)</span><br></pre></td></tr></table></figure><p>然后就可以在pprof下使用命令了</p><p>可以通过go tool pprof命令查看pprof支持哪些命令。</p><p><a href="https://golang.org/pkg/runtime/pprof/" target="_blank" rel="noopener">runtime/pprof</a>的缺点是必须将程序关闭或者设置信号量来停止pprof的输出，这样才可以使用生产的pprof文件</p><h3 id="net-http-pprof"><a href="#net-http-pprof" class="headerlink" title="net/http/pprof"></a>net/http/pprof</h3><p>对专门提供web服务的程序可以使用此包，可以方便的测试应用程序的性能</p><p>要使用<code>net/http/pprof</code>包很简单，在main.go文件导入包的时候，通过<code>_ "net/http/pprof"</code>方式导入，其实看pprof.go的文件就能知道，实现的原理。</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">init</span><span class="params">()</span></span> {</span><br><span class="line">http.HandleFunc(<span class="string">"/debug/pprof/"</span>, Index)</span><br><span class="line">http.HandleFunc(<span class="string">"/debug/pprof/cmdline"</span>, Cmdline)</span><br><span class="line">http.HandleFunc(<span class="string">"/debug/pprof/profile"</span>, Profile)</span><br><span class="line">http.HandleFunc(<span class="string">"/debug/pprof/symbol"</span>, Symbol)</span><br><span class="line">http.HandleFunc(<span class="string">"/debug/pprof/trace"</span>, Trace)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="通过浏览器分析"><a href="#通过浏览器分析" class="headerlink" title="通过浏览器分析"></a>通过浏览器分析</h4><p>pprof.go中最开始先申明一个init函数，这里申明了五个<code>HandleFunc</code>对应的可以在浏览器中可以打开着五个页面</p><p><code>/debug/pprof/</code>页面是首页，可以查看go程序的堆栈、goroutine、线程等信息</p><p><img src="/2017/08/23/Go性能分析工具/debug_pprof.png" alt=""></p><p>一般如果要获取cpu的信息，生成pprof文件，则直接访问<code>/debug/pprof/profile</code>，</p><p>通过代码:</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">Profile</span><span class="params">(w http.ResponseWriter, r *http.Request)</span></span> {</span><br><span class="line">sec, _ := strconv.ParseInt(r.FormValue(<span class="string">"seconds"</span>), <span class="number">10</span>, <span class="number">64</span>)</span><br><span class="line"><span class="keyword">if</span> sec == <span class="number">0</span> {</span><br><span class="line">sec = <span class="number">30</span></span><br><span class="line">}</span><br><span class="line">········</span><br><span class="line"><span class="keyword">if</span> err := pprof.StartCPUProfile(w); err != <span class="literal">nil</span> {</span><br><span class="line"><span class="comment">// StartCPUProfile failed, so no writes yet.</span></span><br><span class="line"><span class="comment">// Can change header back to text content</span></span><br><span class="line"><span class="comment">// and send error code.</span></span><br><span class="line">w.Header().Set(<span class="string">"Content-Type"</span>, <span class="string">"text/plain; charset=utf-8"</span>)</span><br><span class="line">w.Header().Set(<span class="string">"X-Go-Pprof"</span>, <span class="string">"1"</span>)</span><br><span class="line">w.WriteHeader(http.StatusInternalServerError)</span><br><span class="line">fmt.Fprintf(w, <span class="string">"Could not enable CPU profiling: %s\n"</span>, err)</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">}</span><br><span class="line">sleep(w, time.Duration(sec)*time.Second)</span><br><span class="line">pprof.StopCPUProfile()</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>可以看出<code>Profile</code>函数接收一个cpu收集时间，是按秒为单位收集的，如果，不填的话，默认是30秒的收集时间.</p><p>所以想自定义程序收集cpu的时间的话就可以自己传入手机时间的数值</p><p>比如我要手机1分钟的数据，则只需要<code>localhost:8080/debug/pprof/profile?seconds=60</code>.这样的话，程序就会进入60秒的cpu收集时间，等到收集完成后，会返回一个profile的二进制文件，我们可以给重命名为<code>cpu.pprof</code>，然后就可以使用go tool pprof cpu.pprof 来进行性能分析了。</p><p>同理内存分析可以通过访问<code>localhost:8080/debug/pprof/heap</code></p><h4 id="通过命令行来分析"><a href="#通过命令行来分析" class="headerlink" title="通过命令行来分析"></a>通过命令行来分析</h4><p>可以通过命令收集cpu</p><p><code>go tool pprof http://localhost:8080/debug/pprof/profile</code></p><p>同样可以进行数据收集，当然，可以后面设置参数(<code>--seconds 25</code>表示设置25秒)，默认是30秒的收集时间。收集完成后悔进入pprof模式下</p><p><img src="/2017/08/23/Go性能分析工具/client_pprof.png" alt=""></p><p>也可以通过命令收集内存<code>go tool pprof http://localhost:8080/debug/pprof/heap</code></p><h2 id="pprof文件分析"><a href="#pprof文件分析" class="headerlink" title="pprof文件分析"></a>pprof文件分析</h2><p>接下来就是重点，如何分析我们pprof文件：</p><p>在进入pprof状态之后，可以使用top命令来查看，最耗费资源的是哪些函数</p><p><img src="/2017/08/23/Go性能分析工具/analyze.png" alt=""></p><p>这里分析下各个参数的意思</p><h3 id="flat和cum"><a href="#flat和cum" class="headerlink" title="flat和cum:"></a>flat和cum:</h3><p>Flat表示给定函数的持续时间，cum表示当前函数的累加调用。 比如有一个函数a()调用函数b()和函数c(),</p><p>函数b()耗时1秒，函数b()耗时两秒，那么cum就是1+2=3s</p><p>flat表示的是a()函数自己耗费的时间</p><p>如果a()函数是这样的</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">b() <span class="comment">// takes 2s</span></span><br><span class="line">do something directly <span class="comment">// takes 3s</span></span><br><span class="line">c() <span class="comment">// takes 2s</span></span><br></pre></td></tr></table></figure><p>那么a函数的cum值是6秒，flat是3秒（假设do something directly里面没有函数调用）</p><h3 id="sum"><a href="#sum" class="headerlink" title="sum:"></a>sum:</h3><p>要理解sum需要看上图，第一个sum是25%和flat的25%是相同的，然后第二个sum是50%,是第一个flat的25%加上第二个flat的25%，以此类推。</p><p>具体可以参考：</p><p><a href="https://www.reddit.com/r/golang/comments/7ony5f/what_is_the_meaning_of_flat_and_cum_in_golang/" target="_blank" rel="noopener">What is the meaning of “flat” and “cum” in golang pprof output</a></p><h3 id="图形分析"><a href="#图形分析" class="headerlink" title="图形分析"></a>图形分析</h3><p>只是通过top命令来查看分析数据的话，太过抽象也不好分析，go tool pprof中也有工具可以把生成的pprof文件转换成图形工具，但是需要事先安装 graphviz 。</p><p>安装好之后可以直接使用命令来生成图片</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">[root@iZbp14ouog5ocoeakj39q1Z guess]# <span class="keyword">go</span> tool pprof cpu.pprof</span><br><span class="line">Entering interactive mode (<span class="keyword">type</span> <span class="string">"help"</span> <span class="keyword">for</span> commands)</span><br><span class="line">(pprof) svg</span><br><span class="line">Generating report in profile001.svg</span><br><span class="line">(pprof)</span><br></pre></td></tr></table></figure><p>这样就生成了svg图片<code>profile001.svg</code></p><p><img src="/2017/08/23/Go性能分析工具/sum&cum&flat.png" alt=""></p><p>由于我代码中并没有写太多的业务逻辑，所以这里可以看到大部分的耗时多事发生在运行时，四个耗时25%的函数是<code>runtime.memmove</code> <code>runtime.memeqbody</code> <code>runtimecgocall</code> <code>runtime.stdcall</code>.接下来，可以根据图形具体分析程序在哪里耗费资源然后进行优化</p><h2 id="火焰图"><a href="#火焰图" class="headerlink" title="火焰图"></a>火焰图</h2><p>上图的结构给我们的是晦涩难懂的感觉，我们需要寻求更直观，更简单的分析工具。而且使用火焰图不需要安装graphviz</p><p>go-torch是<code>Uber</code>公司开源的一款针对Go语言程序的火焰图生成工具，能收集 stack traces,并把它们整理成火焰图，直观地程序给开发人员。</p><p>go-torch是基于使用BrendanGregg创建的火焰图工具生成直观的图像，很方便地分析Go的各个方法所占用的CPU的时间， 火焰图是一个新的方法来可视化CPU的使用情况，本文中我会展示如何使用它辅助我们排查问题。</p><h3 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h3><p>1.首先，我们要配置<code>FlameGraph</code>的脚本</p><blockquote><p>FlameGraph 是profile数据的可视化层工具，已被广泛用于Python和Node</p></blockquote><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">git clone https://github.com/brendangregg/FlameGraph.git</span><br></pre></td></tr></table></figure><p>2.检出完成后，把<code>flamegraph.pl</code>拷到我们机器环境变量$PATH的路径中去，例如：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">cp flamegraph.pl /usr/local/bin</span><br></pre></td></tr></table></figure><p>3.在终端输入 <code>flamegraph.pl -h</code> 是否安装FlameGraph成功</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line">$ flamegraph.pl -h</span><br><span class="line">Option h is ambiguous (hash, height, help)</span><br><span class="line">USAGE: /usr/local/bin/flamegraph.pl [options] infile > outfile.svg</span><br><span class="line"></span><br><span class="line"> --title # change title text</span><br><span class="line"> --width # width of image (default 1200)</span><br><span class="line"> --height # height of each frame (default 16)</span><br><span class="line"> --minwidth # omit smaller functions (default 0.1 pixels)</span><br><span class="line"> --fonttype # font type (default "Verdana")</span><br><span class="line"> --fontsize # font size (default 12)</span><br><span class="line"> --countname # count type label (default "samples")</span><br><span class="line"> --nametype # name type label (default "Function:")</span><br><span class="line"> --colors # set color palette. choices are: hot (default), mem, io,</span><br><span class="line"> # wakeup, chain, java, js, perl, red, green, blue, aqua,</span><br><span class="line"> # yellow, purple, orange</span><br><span class="line"> --hash # colors are keyed by function name hash</span><br><span class="line"> --cp # use consistent palette (palette.map)</span><br><span class="line"> --reverse # generate stack-reversed flame graph</span><br><span class="line"> --inverted # icicle graph</span><br><span class="line"> --negate # switch differential hues (blue<->red)</span><br><span class="line"> --help # this message</span><br><span class="line"></span><br><span class="line"> eg,</span><br><span class="line"> /usr/local/bin/flamegraph.pl --title="Flame Graph: malloc()" trace.txt > graph.svg</span><br></pre></td></tr></table></figure><p>4.安装go-torch</p><p>有了flamegraph的支持，我们接下来要使用go-torch展示profile的输出，而安装go-torch很简单，我们使用下面的命令即可完成安装</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">go get -v github.com/uber/go-torch</span><br></pre></td></tr></table></figure><p>5.使用go-torch -h命令：可以查看go-torch的帮助文档，这里我们根据生产的cpu.pprof文件，通过使用go-torch 命令来生成火焰图</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ go-torch -b cpu.pprof -f cpu.svg</span><br><span class="line">INFO[12:38:16] Run pprof command: go tool pprof -raw cpu.pprof</span><br><span class="line">INFO[12:38:16] Writing svg to cpu.svg</span><br></pre></td></tr></table></figure><ul><li><strong>-b：</strong>表示需要被转换成svg的二进制文件</li><li><strong>-f：</strong>表示要生成的svg图片名称</li></ul><p>此时已经将cpu.pprof生成了cpu.svg的火焰图了，可以通过浏览器查看</p><p><img src="/2017/08/23/Go性能分析工具/fire1.png" alt=""></p><p>这就是go-torch生成的火焰图，看起来是不是舒服多了。</p><blockquote><ul><li>火焰图的y轴表示cpu调用方法的先后，比如：<code>bufio.(*Writer).Flush</code>是由<code>net/http.(*chunkWriter).write</code>和<code>net/http.CheckConnErrorWriter.Writer</code>两个函数组成的。</li><li>x轴表示在每个采样调用时间内，方法所占的时间百分比，越宽代表占据cpu时间越多</li></ul></blockquote><p>有了火焰图，我们就可以更清楚的看到哪个方法调用耗时长了，然后不断的修正代码，重新采样，不断优化。</p><h1 id="参考："><a href="#参考：" class="headerlink" title="参考："></a>参考：</h1><p><a href="http://lihaoquan.me/2017/1/1/Profiling-and-Optimizing-Go-using-go-torch.html" target="_blank" rel="noopener">Go代码调优利器-火焰图</a></p><p><a href="https://blog.csdn.net/WaltonWang/article/details/54019891" target="_blank" rel="noopener">Golang性能调优(go-torch, go tool pprof)</a></p>]]></content>
<categories>
<category> Golang </category>
</categories>
<tags>
<tag> golang </tag>
</tags>
</entry>
<entry>
<title>golang-http包</title>
<link href="/2017/08/16/golang-http%E5%8C%85/"/>
<url>/2017/08/16/golang-http%E5%8C%85/</url>
<content type="html"><![CDATA[<h1 id="http服务"><a href="#http服务" class="headerlink" title="http服务"></a>http服务</h1><p>http包包含http客户端和服务端的实现,利用Get,Head,Post,以及PostForm实现HTTP或者HTTPS的请求.</p><h2 id="1-函数"><a href="#1-函数" class="headerlink" title="1. 函数"></a>1. 函数</h2><h3 id="1-1-服务端函数"><a href="#1-1-服务端函数" class="headerlink" title="1.1 服务端函数"></a>1.1 服务端函数</h3><ol><li><p><code>Handle</code>将handler按照指定的格式注册到<code>DefaultServeMux</code>,<code>ServeMux</code>解释了模式匹配规则</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">Handle</span><span class="params">(pattern <span class="keyword">string</span>, handler Handler)</span></span></span><br></pre></td></tr></table></figure></li><li><p><code>HandleFunc</code>同上，主要用来实现动态文件内容的展示，这点与<code>ServerFile()</code>不同的地方。</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">HandleFunc</span><span class="params">(pattern <span class="keyword">string</span>, handler <span class="keyword">func</span>(ResponseWriter, *Request)</span>)</span></span><br></pre></td></tr></table></figure></li><li><p><code>ServeFile</code>利用指定的文件或者目录的内容来响应相应的请求.</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">ServeFile</span><span class="params">(w ResponseWriter, r *Request, name <span class="keyword">string</span>)</span></span></span><br></pre></td></tr></table></figure></li><li><p><code>ListenAndServe</code>监听TCP网络地址addr然后调用具有handler的Serve去处理连接请求.通常情况下Handler是nil,使用默认的DefaultServeMux</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">ListenAndServe</span><span class="params">(addr <span class="keyword">string</span>, handler Handler)</span> <span class="title">error</span></span></span><br></pre></td></tr></table></figure></li></ol><h3 id="1-2最简单的http服务"><a href="#1-2最简单的http服务" class="headerlink" title="1.2最简单的http服务"></a>1.2最简单的http服务</h3><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> <span class="string">"net/http"</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> http.ListenAndServe(<span class="string">":8080"</span>, <span class="literal">nil</span>)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>访问网页后会发现，提示的不是“无法访问”，而是”页面没找到“，说明http已经开始服务了，只是没有找到页面<br>由此可以看出，访问什么路径显示什么网页 这件事情，和ListenAndServe的第2个参数有关</p><p>由1.1的解析可知，第2个参数是一个 <strong>Hander</strong></p><p>在http包中看到这个 <strong>Hander</strong>接口只有一个方法ServeHTTP</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> Handler <span class="keyword">interface</span> {</span><br><span class="line">ServeHTTP(ResponseWriter, *Request)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>所以只要实现了ServeHTTP(ResponseWriter, *Request)这个方法的struct,那么就可以将这个struct方法放进去，然后被调用</p><p>ServeHTTP方法，他需要2个参数，</p><ol><li>一个是http.ResponseWriter， 往http.ResponseWriter写入什么内容，浏览器的网页源码就是什么内容</li><li>另一个是<em>http.Request，</em>http.Request里面是封装了浏览器发过来的请求（包含路径、浏览器类型等等）</li></ol><h3 id="example"><a href="#example" class="headerlink" title="example"></a>example</h3><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"> <span class="string">"io"</span></span><br><span class="line"> <span class="string">"net/http"</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> test <span class="keyword">struct</span>{}</span><br><span class="line"><span class="comment">//结构体a实现了ServeHTTP</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(*test)</span> <span class="title">ServeHTTP</span><span class="params">(w http.ResponseWriter, r *http.Request)</span></span> {</span><br><span class="line"> io.WriteString(w, <span class="string">"hello world"</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> http.ListenAndServe(<span class="string">":8080"</span>, &test{})<span class="comment">//第2个参数需要实现Hander的struct，a满足</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>现在<br>访问localhost:8080的话，可以看到“hello world”<br>访问localhost:8080/abc的话，可以看到“hello world”<br>访问localhost:8080/123的话，可以看到“hello world”<br>事实上访问任何路径都是“hello world”</p><p>当 http.ListenAndServe(“:8080”, &test{})后，开始等待有访问请求</p><p>一旦有访问请求过来，http包回去调用test的ServeHTTP这个方法。并把自己已经处理好的<code>http.ResponseWriter, *http.Request</code>传进去</p><p>而test的ServeHTTP这个方法，拿到<code>*http.ResponseWriter</code>后，并往里面写东西，客户端的网页就显示出来了</p><p>一、从源码可以理解:这里会将Handler赋值给Server</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// ListenAndServe always returns a non-nil error.</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">ListenAndServe</span><span class="params">(addr <span class="keyword">string</span>, handler Handler)</span> <span class="title">error</span></span> {</span><br><span class="line">server := &Server{Addr: addr, Handler: handler}</span><br><span class="line"><span class="keyword">return</span> server.ListenAndServe()</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>二、</p><p>这里是server.ListenAndServe()———–>回去调用go c.server(ctx)</p><p>其中c是<code>c := srv.newConn(rw)</code></p><p>然后<code>c.server(ctx)</code>这个函数中会调用<code>serverHandler{c.server}.ServeHTTP(w, w.req)</code>这个方法</p><p>这里serverHandler组合了Server结构体</p><p>这里当handler为空的时候就调用默认的DefaultServeMux，当不为空的时候就会去调用handler.ServeHTTP</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// serverHandler delegates to either the server's Handler or</span></span><br><span class="line"><span class="comment">// DefaultServeMux and also handles "OPTIONS *" requests.</span></span><br><span class="line"><span class="keyword">type</span> serverHandler <span class="keyword">struct</span> {</span><br><span class="line">srv *Server</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(sh serverHandler)</span> <span class="title">ServeHTTP</span><span class="params">(rw ResponseWriter, req *Request)</span></span> {</span><br><span class="line">handler := sh.srv.Handler</span><br><span class="line"><span class="keyword">if</span> handler == <span class="literal">nil</span> {</span><br><span class="line">handler = DefaultServeMux</span><br><span class="line">}</span><br><span class="line"><span class="keyword">if</span> req.RequestURI == <span class="string">"*"</span> && req.Method == <span class="string">"OPTIONS"</span> {</span><br><span class="line">handler = globalOptionsHandler{}</span><br><span class="line">}</span><br><span class="line">handler.ServeHTTP(rw, req)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>通过上面的解析就可以知道网页解析是通过调用ServeHTTP方法来的</p><p>###2.3、ServeMux的作用</p><p>先看ServeMux的结构体:</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> ServeMux <span class="keyword">struct</span> {</span><br><span class="line"> mu sync.RWMutex</span><br><span class="line"> m <span class="keyword">map</span>[<span class="keyword">string</span>]muxEntry</span><br><span class="line"> hosts <span class="keyword">bool</span> <span class="comment">// whether any patterns contain hostnames</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> muxEntry <span class="keyword">struct</span> {</span><br><span class="line"> explicit <span class="keyword">bool</span></span><br><span class="line"> h Handler</span><br><span class="line"> pattern <span class="keyword">string</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>从结构体中可以看出ServeMux有一个map属性，map属性的value是个muxEntry类型，这个类型中有一个Handler属性，可以推测看看此ServerMux的m属性的key保存的是url，muxEntry是一个Handler方法</p><p>然后看代码</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"> <span class="string">"net/http"</span></span><br><span class="line"> <span class="string">"io"</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> b <span class="keyword">struct</span>{}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(*b)</span> <span class="title">ServeHTTP</span><span class="params">(w http.ResponseWriter, r *http.Request)</span></span> {</span><br><span class="line"> io.WriteString(w, <span class="string">"hello"</span>)</span><br><span class="line">}</span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> mux := http.NewServeMux()<span class="comment">//新建一个ServeMux。</span></span><br><span class="line"> mux.Handle(<span class="string">"/h"</span>, &b{})<span class="comment">//注册路由，把"/"注册给b这个实现Handler接口的struct，注册到map表中。</span></span><br><span class="line"> http.ListenAndServe(<span class="string">":8080"</span>, mux)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><strong>mux.Handle内部</strong></p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//Handle registers the handler for the given pattern.</span></span><br><span class="line"><span class="comment">// If a handler already exists for pattern, Handle panics.</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(mux *ServeMux)</span> <span class="title">Handle</span><span class="params">(pattern <span class="keyword">string</span>, handler Handler)</span></span> {</span><br><span class="line"> mux.mu.Lock()</span><br><span class="line"> <span class="keyword">defer</span> mux.mu.Unlock()</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> pattern == <span class="string">""</span> {</span><br><span class="line"> <span class="built_in">panic</span>(<span class="string">"http: invalid pattern "</span> + pattern)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> handler == <span class="literal">nil</span> {</span><br><span class="line"> <span class="built_in">panic</span>(<span class="string">"http: nil handler"</span>)</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> mux.m[pattern].explicit {</span><br><span class="line"> <span class="built_in">panic</span>(<span class="string">"http: multiple registrations for "</span> + pattern)</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> mux.m == <span class="literal">nil</span> {</span><br><span class="line"> mux.m = <span class="built_in">make</span>(<span class="keyword">map</span>[<span class="keyword">string</span>]muxEntry)</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">//将路由作为key，然后将handler和路由以及显示调用设置为true</span></span><br><span class="line"> mux.m[pattern] = muxEntry{explicit: <span class="literal">true</span>, h: handler, pattern: pattern}</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> pattern[<span class="number">0</span>] != <span class="string">'/'</span> {</span><br><span class="line"> mux.hosts = <span class="literal">true</span></span><br><span class="line"> }</span><br><span class="line"> ....</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p>所以可以看出ServeMux是通过一个map将路由以及函数存起来的。</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(mux *ServeMux)</span> <span class="title">Handle</span><span class="params">(pattern <span class="keyword">string</span>, handler Handler)</span></span> {}</span><br></pre></td></tr></table></figure><p>这个函数接收的第一个参数是<strong>路由</strong>，第二个参数是一个<strong>Handler</strong>。这个Handler和上面ListenAndServe的第二个参数是一样的是只有一个ServeHTTP(ResponseWriter, *Request)方法的接口。所以此处的handler需要实现ServeHTTP方法。</p><p>运行时，因为第二个参数是mux，所以http会调用mux的ServeHTTP方法。ServeHTTP方法执行时，会检查map表（表里有一条数据，key是“/h”，value是&b{}的ServeHTTP方法）</p><p>如果用户访问<code>/h</code>的话，mux因为匹配上了，mux的ServeHTTP方法会去调用&b{}的 ServeHTTP方法，从而打印hello<br>如果用户访问<code>/abc</code>的话，mux因为没有匹配上，从而打印404 page not found</p><h3 id="2-4、ServeMux的HandleFunc方法"><a href="#2-4、ServeMux的HandleFunc方法" class="headerlink" title="2.4、ServeMux的HandleFunc方法"></a>2.4、ServeMux的HandleFunc方法</h3><p>ServeMux有一个HandleFunc方法，此方法直接调用handle函数并实现了ServeHTTP</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(mux *ServeMux)</span> <span class="title">HandleFunc</span><span class="params">(pattern <span class="keyword">string</span>, handler <span class="keyword">func</span>(ResponseWriter, *Request)</span>)</span> {</span><br><span class="line"> mux.Handle(pattern, HandlerFunc(handler))</span><br><span class="line">}</span><br></pre></td></tr></table></figure><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> HandlerFunc <span class="function"><span class="keyword">func</span><span class="params">(ResponseWriter, *Request)</span></span></span><br><span class="line"><span class="function"></span></span><br><span class="line"><span class="function">// <span class="title">ServeHTTP</span> <span class="title">calls</span> <span class="title">f</span><span class="params">(w, r)</span>.</span></span><br><span class="line"><span class="function"><span class="title">func</span> <span class="params">(f HandlerFunc)</span> <span class="title">ServeHTTP</span><span class="params">(w ResponseWriter, r *Request)</span></span> {</span><br><span class="line"> f(w, r)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>所以使用HandlerFunc的时候</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"> <span class="string">"net/http"</span></span><br><span class="line"> <span class="string">"io"</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"> mux := http.NewServeMux()</span><br><span class="line"> mux.HandleFunc(<span class="string">"/h"</span>, <span class="function"><span class="keyword">func</span><span class="params">(w http.ResponseWriter, r *http.Request)</span></span> {</span><br><span class="line"> io.WriteString(w, <span class="string">"hello"</span>)</span><br><span class="line"> })</span><br><span class="line"> mux.HandleFunc(<span class="string">"/bye"</span>, <span class="function"><span class="keyword">func</span><span class="params">(w http.ResponseWriter, r *http.Request)</span></span> {</span><br><span class="line"> io.WriteString(w, <span class="string">"byebye"</span>)</span><br><span class="line"> })</span><br><span class="line"> mux.HandleFunc(<span class="string">"/hello"</span>, sayhello)</span><br><span class="line"> http.ListenAndServe(<span class="string">":8080"</span>, mux)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">sayhello</span><span class="params">(w http.ResponseWriter, r *http.Request)</span></span> {</span><br><span class="line"> io.WriteString(w, <span class="string">"hello world"</span>)</span><br><span class="line">}</span><br></pre></td></tr></table></figure>]]></content>
<categories>
<category> Golang </category>
</categories>
<tags>
<tag> golang </tag>
</tags>
</entry>
<entry>
<title>Go单元测试</title>
<link href="/2017/06/23/Go%E5%8D%95%E5%85%83%E6%B5%8B%E8%AF%95/"/>
<url>/2017/06/23/Go%E5%8D%95%E5%85%83%E6%B5%8B%E8%AF%95/</url>
<content type="html"><![CDATA[<h1 id="为什么需要单元测试"><a href="#为什么需要单元测试" class="headerlink" title="为什么需要单元测试"></a>为什么需要单元测试</h1><p> 开发程序其中很重要的一点是测试，我们如何保证代码的质量，如何保证每个函数是可运行，运行结果是正确的，又如何保证写出来的代码性能是好的，我们知道单元测试的重点在于发现程序设计或实现的逻辑错误，使问题及早暴露，便于问题的定位解决。对一个包做（单元）测试，需要写一些可以频繁（每次更新后）执行的小块测试单元来检查代码的正确性。于是我们必须写一些 Go 源文件来测试代码。测<strong>试程序必须属于被测试的包</strong>，并且文件名满足这种形式 <code>*_test.go</code>，所以测试代码和包中的业务代码是分开的。</p><p> Go语言中自带有一个轻量级的测试框架<code>testing</code>和自带的<code>go test</code>命令来实现单元测试和性能测试，<code>testing</code>框架和其他语言中的测试框架类似，你可以基于这个框架写针对相应函数的测试用例。</p><p><code>_test</code> 程序不会被普通的 Go 编译器编译，所以当放应用部署到生产环境时它们不会被部署；只有 gotest 会编译所有的程序：普通程序和测试程序。</p><p>另外建议安装<a href="https://github.com/cweill/gotests" target="_blank" rel="noopener">gotests</a>插件自动生成测试代码:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">go get -u -v github.com/cweill/gotests/...</span><br></pre></td></tr></table></figure><p>go单元测试的命令是</p><p><code>go test</code></p><p>go test 只会输出错误的信息，想要看详细的信息使用<code>go test -v</code></p><h1 id="Go的自带单元测试的编写"><a href="#Go的自带单元测试的编写" class="headerlink" title="Go的自带单元测试的编写"></a>Go的自带单元测试的编写</h1><h2 id="1、如何编写测试用例"><a href="#1、如何编写测试用例" class="headerlink" title="1、如何编写测试用例"></a>1、如何编写测试用例</h2><p>接下来我们在该目录下面创建两个文件：even.go和oddeven_test.go<br><strong>even.go:</strong></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">/*</span><br><span class="line">用来测试前 100 个整数是否是偶数。这个函数属于 even 包。</span><br><span class="line">*/</span><br><span class="line">package even</span><br><span class="line">func Even(i int) bool {</span><br><span class="line">return i%2 == 0</span><br><span class="line">}</span><br><span class="line">func Odd(i int) bool {</span><br><span class="line">return i%2 != 0</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><strong>oddeven_test.go:这是我们的单元测试文件，</strong></p><h3 id="1-1测试文件的规则："><a href="#1-1测试文件的规则：" class="headerlink" title="1.1测试文件的规则："></a><strong>1.1测试文件的规则：</strong></h3><ol><li>文件名必须是<code>_test.go</code>结尾的，这样在执行<code>go test</code>的时候才会执行到相应的代码</li><li>你必须import <code>testing</code>这个包</li><li>所有的测试用例函数必须是<code>Test</code>开头</li><li>测试用例会按照源代码中写的顺序依次执行</li><li>测试函数<code>TestXxx()</code>的参数是<code>testing.T</code>，我们可以使用该类型来记录错误或者是测试状态</li><li>测试格式：<code>func TestXxx (t *testing.T)</code>,<code>Xxx</code>部分可以为任意的字母数字的组合，但是首字母不能是小写字母[a-z]，例如<code>Testintdiv</code>是错误的函数名。</li><li>函数中通过调用<code>testing.T</code>的<code>Error</code>, <code>Errorf</code>, <code>FailNow</code>, <code>Fatal</code>, <code>FatalIf</code>方法，说明测试不通过，调用<code>Log</code>方法用来记录测试的信息。</li></ol><p><strong>oddeven_test.go</strong></p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> even</span><br><span class="line"><span class="keyword">import</span> <span class="string">"testing"</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">TestEven</span><span class="params">(t *testing.T)</span></span> {</span><br><span class="line"><span class="keyword">if</span> !Even(<span class="number">10</span>) { <span class="comment">//!Even(10)==false,不会向下执行</span></span><br><span class="line">t.Log(<span class="string">"10 must be even"</span>)</span><br><span class="line">t.Fail()</span><br><span class="line">}</span><br><span class="line"><span class="keyword">if</span> Even(<span class="number">7</span>) {</span><br><span class="line">t.Log(<span class="string">"7 is not even"</span>)</span><br><span class="line">t.Fatal()</span><br><span class="line">}</span><br><span class="line"><span class="keyword">if</span> Even(<span class="number">10</span>) { <span class="comment">//Even(10)==true,但是为了测试，让它执行t.Log()和 t.Fail()</span></span><br><span class="line">t.Log(<span class="string">"Everything OK: 10 is even, just a test to see failed output!"</span>)</span><br><span class="line">t.Fail()</span><br><span class="line">}</span><br><span class="line">}</span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">TestOdd</span><span class="params">(t *testing.T)</span></span> {</span><br><span class="line"><span class="keyword">if</span> !Odd(<span class="number">11</span>) {</span><br><span class="line">t.Log(<span class="string">" 11 must be odd!"</span>)</span><br><span class="line">t.Fail()</span><br><span class="line">}</span><br><span class="line"><span class="keyword">if</span> Odd(<span class="number">10</span>) {</span><br><span class="line">t.Log(<span class="string">" 10 is not odd!"</span>)</span><br><span class="line">t.Fail()</span><br><span class="line">}</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><strong>使用go test 运行oddeven_test.go,只会出现错误信息</strong></p><blockquote><p>— FAIL: TestEven (0.00s)</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">> oddeven_test.go:16: Everything OK: 10 is even, just a test to see failed output!</span><br><span class="line">></span><br></pre></td></tr></table></figure></blockquote><blockquote><p>FAIL<br>exit status 1<br>FAIL MyStudy/GolangTest/even 0.364s</p></blockquote><p><strong>使用go test -v，会将详细的信息都打印出来，通过的会有pass标识</strong></p><blockquote><p>=== RUN TestEven<br>— FAIL: TestEven (0.00s)</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">> oddeven_test.go:16: Everything OK: 10 is even, just a test to see failed output!</span><br><span class="line">></span><br></pre></td></tr></table></figure></blockquote><blockquote><p>=== RUN TestOdd<br>— PASS: TestOdd (0.00s)<br>FAIL<br>exit status 1<br>FAIL MyStudy/GolangTest/even 0.397s</p></blockquote><h3 id="1-2单元测试的一些通知失败的方法"><a href="#1-2单元测试的一些通知失败的方法" class="headerlink" title="1.2单元测试的一些通知失败的方法"></a>1.2单元测试的一些通知失败的方法</h3><p>1）<code>func (t *T) Fail()</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">标记测试函数为失败，然后继续执行（剩下的测试）。</span><br></pre></td></tr></table></figure><p>2）<code>func (t *T) FailNow()</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">标记测试函数为失败并中止执行；文件中别的测试也被略过，继续执行下一个文件。</span><br></pre></td></tr></table></figure><p>3）<code>func (t *T) Log(args ...interface{})</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">args 被用默认的格式格式化并打印到错误日志中。</span><br></pre></td></tr></table></figure><p>4）<code>func (t *T) Fatal(args ...interface{})</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">结合 先执行 3），然后执行 2）的效果。</span><br></pre></td></tr></table></figure><h1 id="GoGonvey框架写单元测试"><a href="#GoGonvey框架写单元测试" class="headerlink" title="GoGonvey框架写单元测试"></a>GoGonvey框架写单元测试</h1><h2 id="1、GoGonvey框架介绍"><a href="#1、GoGonvey框架介绍" class="headerlink" title="1、GoGonvey框架介绍"></a>1、GoGonvey框架介绍</h2><p> Go 语言虽然自带单元测试功能，在 GoConvey 诞生之前也出现了许多第三方辅助库。但没有一个辅助库能够像 GoConvey 这样优雅地书写代码的单元测试，简洁的语法和舒适的界面能够让一个不爱书写单元测试的开发人员从此爱上单元测试。</p><h3 id="1-1、GoGonvey的优点"><a href="#1-1、GoGonvey的优点" class="headerlink" title="1.1、GoGonvey的优点"></a>1.1、GoGonvey的优点</h3><ol><li>GoConvey支持Go的本机<a href="http://golang.org/pkg/testing/" target="_blank" rel="noopener"><code>testing</code></a>包。无论是网页界面还是DSL都不需要; 你可以独立使用任何一个。</li><li>GoConvey集成后<code>**go test**</code>，您可以<a href="https://github.com/smartystreets/goconvey/wiki/Execution" target="_blank" rel="noopener">在终端中</a>继续运行测试<a href="https://github.com/smartystreets/goconvey/wiki/Execution" target="_blank" rel="noopener">，</a>或者使用自动更新Web UI进行测试。</li><li>GoConvey还有web UI可以来方便查看代码覆盖率，以及单元测试错误信息</li></ol><h3 id="1-2、安装GoGonvey"><a href="#1-2、安装GoGonvey" class="headerlink" title="1.2、安装GoGonvey"></a>1.2、安装GoGonvey</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">go get github.com/smartystreets/goconvey</span><br></pre></td></tr></table></figure><h3 id="1-3、快速开始一个例子"><a href="#1-3、快速开始一个例子" class="headerlink" title="1.3、快速开始一个例子"></a>1.3、快速开始一个例子</h3><p>写一个oddeven_goconvey_test.go 文件,将test.go改写成convey形式的</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> even</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line">. <span class="string">"github.com/smartystreets/goconvey/convey"</span></span><br><span class="line"><span class="string">"testing"</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">TestEvenConvey</span><span class="params">(t *testing.T)</span></span> {</span><br><span class="line">Convey(<span class="string">"Given some integer with a starting value"</span>, t, <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line">Convey(<span class="string">"When the integer is even"</span>, <span class="function"><span class="keyword">func</span><span class="params">()</span></span> { <span class="comment">//第一个Convey需要t参数，以后的不需要了</span></span><br><span class="line">Convey(<span class="string">"10 must be even"</span>, <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line">b := Even(<span class="number">10</span>)</span><br><span class="line">So(b, ShouldBeTrue)</span><br><span class="line">})</span><br><span class="line"></span><br><span class="line">Convey(<span class="string">"7 is not even"</span>, <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line">b := Even(<span class="number">7</span>)</span><br><span class="line">So(b, ShouldBeFalse)</span><br><span class="line">})</span><br><span class="line"></span><br><span class="line">Convey(<span class="string">"Everything OK: 10 is even, just a test to see failed output!"</span>, <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line">b := Even(<span class="number">10</span>)</span><br><span class="line">So(b, ShouldBeFalse)</span><br><span class="line">})</span><br><span class="line"></span><br><span class="line">})</span><br><span class="line">})</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p> 使用 GoConvey 书写单元测试，每个测试用例需要使用Convey函数包裹起来。它接受的第一个参数为 string 类型的描述；第二个参数一般为*testing.T，即本例中的变量 t；第三个参数为不接收任何参数也不返回任何值的函数（习惯以闭包的形式书写）。</p><p>Convey语句同样可以无限嵌套，以体现各个测试用例之间的关系，例如TestEvenConvey函数就采用了嵌套的方式体现它们之间的关系。需要注意的是，<strong>只有最外层的Convey需要传入变量 t，内层的嵌套均不需要传入</strong>。</p><p>最后，需要使用So语句来对条件进行判断。在本例中，我们只使用了 2 个不同类型的条件判断：ShouldBeTrue和ShouldBeFalse，分别表示值应该为 true、值应该false。</p><p>常用的有</p><ol><li><code>ShouldEqual</code>: 表示值应该想等</li><li><code>ShouldResemble</code>: 进行深度相同检查，要有两个值<code>So(b, ShouldResemble, true)</code></li><li><code>ShouldBeTrue</code>: 表示值应该为true</li><li><code>ShouldBeZeroValue</code>:表示值应该为0</li><li><code>ShouldNotContainSubstring</code>:接收2字符串参数并确保第一个不包含第二个字符串。</li><li><code>ShouldPanic</code>: 表示值应该panic</li></ol><p>有关详细的条件列表，可以参见<a href="https://github.com/smartystreets/goconvey/wiki/Assertions" target="_blank" rel="noopener">官方文档</a>。</p><h3 id="1-4、运行测试"><a href="#1-4、运行测试" class="headerlink" title="1.4、运行测试"></a>1.4、运行测试</h3><h4 id="1-4-1、go-test-v运行"><a href="#1-4-1、go-test-v运行" class="headerlink" title="1.4.1、go test -v运行"></a>1.4.1、go test -v运行</h4><p>现在，可以打开命令行，然后输入<code>go test -v</code>来进行测试。由于 GoConvey 兼容 Go 原生的单元测试，因此我们可以直接使用 Go 的命令来执行测试。 </p><blockquote><p>=== RUN TestEvenConvey</p><p>Given some integer with a starting value<br>When the integer is even<br>10 must be even .<br>7 is not even .<br>Everything OK: 10 is even, just a test to see failed output! x</p><p>Failures: </p><p>D:/gopath/src/MyStudy/GolangTest/even/oddeven_goconvey_test.go</p><p>Line 23:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">> Expected: false</span><br><span class="line">> Actual: true</span><br><span class="line">></span><br></pre></td></tr></table></figure></blockquote><blockquote><p> 3 total assertions</p><p>— FAIL: TestEvenConvey (0.00s)<br>FAIL<br>exit status 1<br>FAIL MyStudy/GolangTest/even 0.401s</p></blockquote><p>可以看到给出的信息比go自带的测试包多很多信息</p><h4 id="1-4-2、goconvey-运行测试"><a href="#1-4-2、goconvey-运行测试" class="headerlink" title="1.4.2、goconvey 运行测试"></a>1.4.2、goconvey 运行测试</h4><p>web界面查看数据</p><p>GoConvey 不仅支持在命令行进行人工调用调试命令，还有非常舒适的 Web 界面提供给开发者来进行自动化的编译测试工作。在项目目录下执行<code>goconvey</code> 同时可以打开浏览器查看localhost:8080</p><p><img src="/2017/06/23/Go单元测试/带有错误的convey.png" alt=""></p><p>解析上面图片</p><h3 id="1-5、GoConvey的一些功能"><a href="#1-5、GoConvey的一些功能" class="headerlink" title="1.5、GoConvey的一些功能"></a>1.5、GoConvey的一些功能</h3><h4 id="测试自动运行"><a href="#测试自动运行" class="headerlink" title="测试自动运行"></a>测试自动运行</h4><p> 只需保存<code>.go</code><img src="/2017/06/23/Go单元测试/更新.png" alt="文件或单击图标执行测试">。您的浏览器页面将自动更新。</p><h4 id="覆盖报告"><a href="#覆盖报告" class="headerlink" title="覆盖报告"></a>覆盖报告</h4><p>可以点击左上角的软件包名称来查看<a href="http://blog.golang.org/cover#TOC_5." target="_blank" rel="noopener">Go的详细HTML覆盖率报告</a>。</p><h4 id="黑暗，光和自定义主题"><a href="#黑暗，光和自定义主题" class="headerlink" title="黑暗，光和自定义主题"></a>黑暗，光和自定义主题</h4><p>GoConvey内置了两个主题。（在此页面上<a href="javascript:" target="_blank" rel="noopener">切换主题</a>以尝试！）您还可以使用第三方主题或自己滚动。</p><h4 id="暂停，恢复和审查"><a href="#暂停，恢复和审查" class="headerlink" title="暂停，恢复和审查"></a>暂停，恢复和审查</h4><p>可以暂停自动测试执行，并查看最近的测试历史，以查看代码在何处，何时以及为什么中断。</p><p>在 Web 界面中，您可以设置界面主题，查看完整的测试结果，使用浏览器提醒等实用功能。</p><h4 id="其它功能："><a href="#其它功能：" class="headerlink" title="其它功能："></a>其它功能：</h4><ol><li>自动检测代码变动并编译测试</li><li>半自动化书写测试用例：<a href="http://localhost:8080/composer.html" target="_blank" rel="noopener">http://localhost:8080/composer.html</a></li><li>查看测试覆盖率：<a href="http://localhost:8080/reports/" target="_blank" rel="noopener">http://localhost:8080/reports/</a></li><li>临时屏蔽某个包的编译测试</li></ol>]]></content>
<categories>
<category> Golang </category>
</categories>
<tags>
<tag> golang </tag>
</tags>
</entry>
<entry>
<title>defer和追踪</title>
<link href="/2017/06/15/defer%E5%92%8C%E8%BF%BD%E8%B8%AA/"/>
<url>/2017/06/15/defer%E5%92%8C%E8%BF%BD%E8%B8%AA/</url>
<content type="html"><![CDATA[<h2 id="defer关键字"><a href="#defer关键字" class="headerlink" title="defer关键字"></a>defer关键字</h2><p>关键字 defer 允许我们推迟到函数返回之前（或任意位置执行 <code>return</code> 语句之后）一刻才执行某个语句或函数（为什么要在返回之后才执行这些语句？因为 <code>return</code> 语句同样可以包含一些操作，而不是单纯地返回某个值）。</p><p>关键字 defer 的用法类似于面向对象编程语言 Java 和 C# 的 <code>finally</code> 语句块，它一般用于释放某些已分配的资源。</p><h3 id="示例-6-8-defer-go："><a href="#示例-6-8-defer-go：" class="headerlink" title="示例 6.8 defer.go："></a>示例 6.8 <a href="https://zhqqqy.github.io/2017/06/15/my-first-blog/examples/chapter_6/defer.go">defer.go</a>：</h3><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"><span class="keyword">import</span> <span class="string">"fmt"</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line">function1()</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">function1</span><span class="params">()</span></span> {</span><br><span class="line">fmt.Printf(<span class="string">"In function1 at the top\n"</span>)</span><br><span class="line"><span class="keyword">defer</span> function2()</span><br><span class="line">fmt.Printf(<span class="string">"In function1 at the bottom!\n"</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">function2</span><span class="params">()</span></span> {</span><br><span class="line">fmt.Printf(<span class="string">"function2: Deferred until the end of the calling function!"</span>)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>输出：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">In Function1 at the top</span><br><span class="line">In Function1 at the bottom!</span><br><span class="line">Function2: Deferred until the end of the calling function!</span><br></pre></td></tr></table></figure><p>请将 defer 关键字去掉并对比输出结果。</p><p>使用 defer 的语句同样可以接受参数，下面这个例子就会在执行 defer 语句时打印 <code>0</code>：</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">a</span><span class="params">()</span></span> {</span><br><span class="line">i := <span class="number">0</span></span><br><span class="line"><span class="keyword">defer</span> fmt.Println(i)</span><br><span class="line">i++</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>当有多个 defer 行为被注册时，它们会以逆序执行（类似栈，即后进先出）：</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">f</span><span class="params">()</span></span> {</span><br><span class="line"><span class="keyword">for</span> i := <span class="number">0</span>; i < <span class="number">5</span>; i++ {</span><br><span class="line"><span class="keyword">defer</span> fmt.Printf(<span class="string">"%d "</span>, i)</span><br><span class="line">}</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>上面的代码将会输出：<code>4 3 2 1 0</code>。</p><h3 id="关键字-defer-允许我们进行一些函数执行完成后的收尾工作，"><a href="#关键字-defer-允许我们进行一些函数执行完成后的收尾工作，" class="headerlink" title="关键字 defer 允许我们进行一些函数执行完成后的收尾工作，"></a>关键字 defer 允许我们进行一些函数执行完成后的收尾工作，</h3><p>例如：</p><ol><li><p>关闭文件流：</p><p>// open a file<br>defer file.Close() （详见第 12.2 节）</p></li><li><p>解锁一个加锁的资源</p><p>mu.Lock()<br>defer mu.Unlock() （详见第 9.3 节）</p></li><li><p>打印最终报告</p><p>printHeader()<br>defer printFooter()</p></li><li><p>关闭数据库链接</p><p>// open a database connection<br>defer disconnectFromDB()</p></li></ol><p>合理使用 defer 语句能够使得代码更加简洁。</p><p>以下代码模拟了上面描述的第 4 种情况：</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> <span class="string">"fmt"</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line">doDBOperations()</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">connectToDB</span><span class="params">()</span></span> {</span><br><span class="line">fmt.Println(<span class="string">"ok, connected to db"</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">disconnectFromDB</span><span class="params">()</span></span> {</span><br><span class="line">fmt.Println(<span class="string">"ok, disconnected from db"</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">doDBOperations</span><span class="params">()</span></span> {</span><br><span class="line">connectToDB()</span><br><span class="line">fmt.Println(<span class="string">"Defering the database disconnect."</span>)</span><br><span class="line"><span class="keyword">defer</span> disconnectFromDB() <span class="comment">//function called here with defer</span></span><br><span class="line">fmt.Println(<span class="string">"Doing some DB operations ..."</span>)</span><br><span class="line">fmt.Println(<span class="string">"Oops! some crash or network error ..."</span>)</span><br><span class="line">fmt.Println(<span class="string">"Returning from function here!"</span>)</span><br><span class="line"><span class="keyword">return</span> <span class="comment">//terminate the program</span></span><br><span class="line"><span class="comment">// deferred function executed here just before actually returning, even if</span></span><br><span class="line"><span class="comment">// there is a return or abnormal termination before</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>输出：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">ok, connected to db</span><br><span class="line">Defering the database disconnect.</span><br><span class="line">Doing some DB operations ...</span><br><span class="line">Oops! some crash or network error ...</span><br><span class="line">Returning from function here!</span><br><span class="line">ok, disconnected from db</span><br></pre></td></tr></table></figure><h3 id="使用-defer-语句实现代码追踪"><a href="#使用-defer-语句实现代码追踪" class="headerlink" title="使用 defer 语句实现代码追踪"></a>使用 defer 语句实现代码追踪</h3><p>一个基础但十分实用的实现代码执行追踪的方案就是在进入和离开某个函数打印相关的消息，即可以提炼为下面两个函数：</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">trace</span><span class="params">(s <span class="keyword">string</span>)</span></span> { fmt.Println(<span class="string">"entering:"</span>, s) }</span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">untrace</span><span class="params">(s <span class="keyword">string</span>)</span></span> { fmt.Println(<span class="string">"leaving:"</span>, s) }</span><br></pre></td></tr></table></figure><p>以下代码展示了何时调用两个函数：</p><p>示例 6.10 <a href="https://zhqqqy.github.io/2017/06/15/my-first-blog/examples/chapter_6/defer_tracing.go">defer_tracing.go</a>:</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> <span class="string">"fmt"</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">trace</span><span class="params">(s <span class="keyword">string</span>)</span></span> { fmt.Println(<span class="string">"entering:"</span>, s) }</span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">untrace</span><span class="params">(s <span class="keyword">string</span>)</span></span> { fmt.Println(<span class="string">"leaving:"</span>, s) }</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">a</span><span class="params">()</span></span> {</span><br><span class="line">trace(<span class="string">"a"</span>)</span><br><span class="line"><span class="keyword">defer</span> untrace(<span class="string">"a"</span>)</span><br><span class="line">fmt.Println(<span class="string">"in a"</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">b</span><span class="params">()</span></span> {</span><br><span class="line">trace(<span class="string">"b"</span>)</span><br><span class="line"><span class="keyword">defer</span> untrace(<span class="string">"b"</span>)</span><br><span class="line">fmt.Println(<span class="string">"in b"</span>)</span><br><span class="line">a()</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line">b()</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>输出：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">entering: b</span><br><span class="line">in b</span><br><span class="line">entering: a</span><br><span class="line">in a</span><br><span class="line">leaving: a</span><br><span class="line">leaving: b</span><br></pre></td></tr></table></figure><p>上面的代码还可以修改为更加简便的版本（示例 6.11 <a href="https://zhqqqy.github.io/2017/06/15/my-first-blog/examples/chapter_6/defer_tracing2.go">defer_tracing2.go</a>）：</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> <span class="string">"fmt"</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">trace</span><span class="params">(s <span class="keyword">string</span>)</span> <span class="title">string</span></span> {</span><br><span class="line">fmt.Println(<span class="string">"entering:"</span>, s)</span><br><span class="line"><span class="keyword">return</span> s</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">un</span><span class="params">(s <span class="keyword">string</span>)</span></span> {</span><br><span class="line">fmt.Println(<span class="string">"leaving:"</span>, s)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">a</span><span class="params">()</span></span> {</span><br><span class="line"><span class="keyword">defer</span> un(trace(<span class="string">"a"</span>))</span><br><span class="line">fmt.Println(<span class="string">"in a"</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">b</span><span class="params">()</span></span> {</span><br><span class="line"><span class="keyword">defer</span> un(trace(<span class="string">"b"</span>))</span><br><span class="line">fmt.Println(<span class="string">"in b"</span>)</span><br><span class="line">a()</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line">b()</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="使用-defer-语句来记录函数的参数与返回值"><a href="#使用-defer-语句来记录函数的参数与返回值" class="headerlink" title="使用 defer 语句来记录函数的参数与返回值"></a>使用 defer 语句来记录函数的参数与返回值</h3><p>下面的代码展示了另一种在调试时使用 defer 语句的手法（示例 6.12 <a href="https://zhqqqy.github.io/2017/06/15/my-first-blog/examples/chapter_6/defer_logvalues.go">defer_logvalues.go</a>）：</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"><span class="string">"io"</span></span><br><span class="line"><span class="string">"log"</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">func1</span><span class="params">(s <span class="keyword">string</span>)</span> <span class="params">(n <span class="keyword">int</span>, err error)</span></span> {</span><br><span class="line"><span class="keyword">defer</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line">log.Printf(<span class="string">"func1(%q) = %d, %v"</span>, s, n, err)</span><br><span class="line">}()</span><br><span class="line"><span class="keyword">return</span> <span class="number">7</span>, io.EOF</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line">func1(<span class="string">"Go"</span>)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>输出：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">Output: 2011/10/04 10:46:11 func1("Go") = 7, EOF</span><br></pre></td></tr></table></figure>]]></content>
<categories>
<category> Golang </category>
</categories>
<tags>
<tag> golang </tag>
</tags>
</entry>
</search>